kept select.if(), winner(), extract.workfolow and compareRanks()
and removed everything not supported anymore.
as.warehouse (benchmarkUtils) is not exported, recommend to leave because this may come handy for specific situations

removed second()

What is the problem?
in the extend of the vignette it would be simply

legend now always at bottom and dynamically adjusts to number of algorithms and algorithm name length.
Took me almost the day...

max number of tasks / algorithms now 20 in stacked frequency and line plots, respectively for legend appear on right, otherwise put on bottom of plot. could also be put to lower number

only podium plots and line plots actually require algorithms in legend, otherwise algorithm is identifiable from x-axis or facet label. Now, these redundant legends are removed.
besides that stacked frequency plots have colored tasks and respective legend

works again for single task visualizations. in case multiple plots (a list of plots) is created %++% instead of + must be used for scale_*_() etc
mention in vignette

ok. just an idea, thought this might help issues with the server because people can run stuff by themselves and still seize the user interface.
can be closed or considered later

It has to be
ordering= names(meanRanks)
i.e. a vector of algorithm names in the ranking order
see reportMultiple.Rmd: ordering_consensus=names(params$consensus)

will check old documentation/nature communications example

will set a message in as.challenge() that na.treat needed for report and sanity check in report()

should be working but hasn't been really tested lately. only applies to rank then aggregate. originates from nature communications paper and should be explained there.
For testing I could again compare to Annette's code used at that time.
Wouldn't drop it but you are right that documentation is missing

I'm using R 4.0.2 since summer without issues

also roxygen import and also intitialization of S3 methods have to be written in roxygen then and the namespace file is then entirely written by roxygenization (ideally indcluding help)

I think this may be closed. There have not been any problems with R 4 and R 4 is generally unproblematic.
Agreed?

hm, how do you propose to deal with that? Adapt font size of algorithm names? These might get tiny then....

stabilityByAlgorithmStacked is now replaced by stabilityByAlgorithm(....., stacked=TRUE)

disadvantage would be dependency on another package facilitating ggplot2 dendrograms

current implementation is fine. However, we said it would be nice if all plots (if directly called by users and not within the report) would have the same syntax and could be layouted with ggplot2 syntax. current implementation is using basic R graphics.

turned off in v1.0, everything else postponed

This was a tricky bug, thanks!
alpha is now respected, argument FUN is removed. significance ranking is considered an aggregation step, so testThenRank() is nothing else than aggregateThenRank(FUN="significance").
Note that testThenRank will pass all arguments to decision.challenge(), i.e. it will respect arguments:

redundant

stacked frequency plots are quite neglected. They are not mentioned in the paper. Are they interesting or should they be removed entirely?