Page MenuHomePhabricator

When missing algorithm performances are added as NAs, task is also NA
Open, HighPublic

Description

example from #T27252 (besides the problem to be solved there)

data=rbind(
    data.frame(alg_name="A1", value=0.8, case="C1"),
    data.frame(alg_name="A1", value=0.8, case="C1"),
    data.frame(alg_name="A2", value=0.6, case="C2"),
    data.frame(alg_name="A2", value=0.6, case="C2")
  )

as.challenge(data, algorithm="alg_name", case="case", value="value", smallBetter=FALSE)

gives as output

Performance of not all algorithms is observed for all cases in task 'dummyTask'. Inserted as missings in following cases:
  alg_name case
1       A1   C2
2       A2   C1
$dummyTask
  alg_name case      task value
1       A1   C1 dummyTask   0.8
2       A1   C1 dummyTask   0.8
3       A1   C2      <NA>    NA
4       A2   C1      <NA>    NA
5       A2   C2 dummyTask   0.6
6       A2   C2 dummyTask   0.6

but the task "dummyTask" is artificial. this might be confusing. also in rows 3 and 4 task is empty.
@eisenman you are more into the change of artificial tasks, can you solve this?

Event Timeline

@eisenman could affect tests. please close when finished

eisenman renamed this task from single task challenge treated as multi task in sanity checks gives unuseful warnings to When missing algorithm performances are added as NAs, task is also NA.Sep 10 2020, 12:09 PM
eisenman raised the priority of this task from Normal to High.
eisenman added subscribers: reinkea, aguilera.

Task is added as NA for both specified task name and dummy task name.