taskSubset is deprecated
there is now
subset(x, top, tasks)
- if top is specified subset to top algorithms (only applicable to single task challenges)
- if tasks is specified subset of tasks (only applicable to multi task challenges)
taskSubset is deprecated
there is now
subset(x, top, tasks)
20 might be a sensible number
the sentence can be dropped. But wasn't it planned to revise the text in line with the paper in total?
In T27774#210754, @wiesenfa wrote:In T27774#210741, @eisenman wrote:Ok, weird. For me it says: "The top 5 out of 5 algorithms are considered."
your result is strange given the fact that not the correct things are actually counted
In T27774#210741, @eisenman wrote:Ok, weird. For me it says: "The top 5 out of 5 algorithms are considered."
this is more or less what I tried, ugliness is T27326. I think a ggplot solution mentioned there would be better
report gains new argument "dpi" to change resolution
In T27774#210731, @wiesenfa wrote:
- I think it will be confusing if it writes "top 5 of 5 algorithms", I would put an if
- which changes? as.challenge does not have a full data attribute. The changes in subset have been done by you, see my comments in T27685.
Ok, weird. For me it says: "The top 5 out of 5 algorithms are considered."
set.seed(4) strip=runif(n,.9,1) c_ideal=cbind(task="c_ideal", rbind( data.frame(alg_name="A1",value=runif(n,.9,1),case=1:n), data.frame(alg_name="A2",value=runif(n,.8,.89),case=1:n), data.frame(alg_name="A3",value=runif(n,.7,.79),case=1:n), data.frame(alg_name="A4",value=runif(n,.6,.69),case=1:n), data.frame(alg_name="A5",value=runif(n,.5,.59),case=1:n) )) challenge=as.challenge(c_ideal, algorithm="alg_name", case="case", value="value", smallBetter = T) ranking=challenge%>%aggregateThenRank(FUN = "mean", # aggregation function, na.treat="na.rm", # either "na.rm" to remove missing data, ties.method = "min" # a character string specifying )
In T27774#210733, @wiesenfa wrote:currently It says "The top 0 out of 0 algorithms are considered." in my latest example
currently It says "The top 0 out of 0 algorithms are considered." in my latest example
yes I think so. maybe you have purrr (or dplyr) and ggplot loaded in the workspace.
packages in depends are loaded at startup (and we need these package to be available by the user). imports does not load the packages, functions of challengeR can however call them.
it is true that packages in depends should be minimize, but these are needed
Are you using the latest develop branch? There everything works for me.
This issue is fixed in T27677.
In T27677#210406, @eisenman wrote:In section 3.2.1 "Visualizing bootstrap results", two visualization methods are mentioned, but only one is contained in the report: "To investigate which tasks separate algorithms well (i.e., lead to a stable ranking), two visualization methods
are recommended."Do we add a second plot or adapt the text?