Page MenuHomePhabricator
Create Task
Maniphest T27253

Notify user when multi-task data set is interpreted as single-task data set
Closed, ResolvedPublic
Actions

Description

When a challenge object is instantiated with a multi-task data set and the parameter "by" is not specified, the data set is interpreted as a single-task data set. This can lead to subsequent errors.

Depending on the composition of the data set, the message "Performance of not all algorithms is observed for all cases in task x" or the error "Case(s) (y) appear(s) more than once for the same algorithm in task x" is shown. This makes sense, but is not intuitive at first glance.

The user can be warned of that scenario (including a suggestion on how to resolve the situation: specify the task column via parameter "by"). Further computation could also be stopped.

Related Objects

Event Timeline

eisenman created this task.Mar 25 2020, 1:33 PM
eisenman added a comment.Apr 2 2020, 11:21 AM

At the meeting on 30/03/2020 we agreed that the messages should be extended to warn the user of that scenario. Since the column containing the tasks is unknown when the parameter "by" is not specified, an automatic detection can only be realized with a lot of effort.

eisenman edited projects, added challengeR (v1.0); removed challengeR.Apr 2 2020, 11:21 AM
wiesenfa claimed this task.Apr 2 2020, 11:32 AM
wiesenfa added a comment.Aug 31 2020, 3:36 PM

if at least one quarter of cases have a duplicate, the error message is extended by "Or are you considering a multi-task challenge and forgot to specify argument 'by'? "
Otherwise the error message is now further improved, e.g. if it is a single task, the error message doesn't mention "in task dummyTask any more".
Example error message:

  The following cases appear more than once. Or are you considering a multi-task challenge and forgot to specify argument 'by'? 
Please revise. Cases : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 32, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 74, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 128, 129, 130, 131, 132, 135, 136, 137, 138, 139, 140, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 21
wiesenfa moved this task from Backlog to In Progress on the challengeR (v1.0) board.Aug 31 2020, 3:37 PM
wiesenfa added a comment.Aug 31 2020, 3:48 PM

@eisenman could affect tests. please close when finished

eisenman added a comment.Sep 10 2020, 11:43 AM

Tests will be fixed in T27694.

eisenman moved this task from In Progress to Done on the challengeR (v1.0) board.Sep 10 2020, 11:47 AM
eisenman closed this task as Resolved.Dec 20 2020, 9:31 PM
MITK · MIC · IMSY · DKFZ