When a challenge object is instantiated with a multi-task data set and the parameter "by" is not specified, the data set is interpreted as a single-task data set. This can lead to subsequent errors.
Depending on the composition of the data set, the message "Performance of not all algorithms is observed for all cases in task x" or the error "Case(s) (y) appear(s) more than once for the same algorithm in task x" is shown. This makes sense, but is not intuitive at first glance.
The user can be warned of that scenario (including a suggestion on how to resolve the situation: specify the task column via parameter "by"). Further computation could also be stopped.