will check old documentation/nature communications example

will set a message in as.challenge() that na.treat needed for report and sanity check in report()

should be working but hasn't been really tested lately. only applies to rank then aggregate. originates from nature communications paper and should be explained there.
For testing I could again compare to Annette's code used at that time.
Wouldn't drop it but you are right that documentation is missing

I'm using R 4.0.2 since summer without issues

also roxygen import and also intitialization of S3 methods have to be written in roxygen then and the namespace file is then entirely written by roxygenization (ideally indcluding help)

I started with the roxygen documentation and found that the NAMESPACE file can be generated automatically by using the "export" tag.

I did not try it yet. Did you? Currently, I don't want to take the risk of breaking my running development environment.

The legend could be moved to the bottom.

I think this may be closed. There have not been any problems with R 4 and R 4 is generally unproblematic.
Agreed?

hm, how do you propose to deal with that? Adapt font size of algorithm names? These might get tiny then....

stabilityByAlgorithmStacked is now replaced by stabilityByAlgorithm(....., stacked=TRUE)

disadvantage would be dependency on another package facilitating ggplot2 dendrograms

current implementation is fine. However, we said it would be nice if all plots (if directly called by users and not within the report) would have the same syntax and could be layouted with ggplot2 syntax. current implementation is using basic R graphics.

People that are not from the statistics community might find them easier to read and interpret. I would be in favour of keeping them in the reports. We used the plots in one of our papers.

What is the issue with the current implementation?

This was a tricky bug, thanks!
alpha is now respected, argument FUN is removed. significance ranking is considered an aggregation step, so testThenRank() is nothing else than aggregateThenRank(FUN="significance").
Note that testThenRank will pass all arguments to decision.challenge(), i.e. it will respect arguments:

redundant

stacked frequency plots are quite neglected. They are not mentioned in the paper. Are they interesting or should they be removed entirely?

what about
keep S3, compareRanks and extract.workflow
but do not export (i.e. remove from namespace) compareRanks and extract.workflow. they can then only be accessed e.g. by challengeR:::compareRanks(). These might be of practical use.
?

In my opinion, the functionality that we want to keep should also have unit tests to (1) indicate that it is maintained and (2) to demonstrate how to use it.

The idea is to raise an error if the user tries to extract a task that is not contained in the set of tasks.

In T27282#211185, @eisenman wrote:

The function testThenRank has an argument FUN that is set to "significance" internally. The value that is actually passed is not used.
@wiesenfa Should "significance" be the default value in the signature already or should the argument be removed?

The function testThenRank has an argument FUN that is set to "significance" internally. The value that is actually passed is not used.
@wiesenfa Should "significance" be the default value in the signature already or should the argument be removed?

We talked about this in the meeting this morning. It makes sense to proceed stepwise: First, having consistent descriptions and Lena's feedback (T27677) integrated for release v1.0. We should not reuse text from the paper until it is accepted. Once it is accepted, we can still decide on the revision of the report text (T27420).

I propose to turn off networks and exclude them from the report for now.
if this is solved, we can reintroduce them.
they are not essential I think. what do you think?

It's important to mention that the subset of algorithms should be drawn from the final ranking to avoid wrong results. So if bootstrapping should be performed, create the subset from the bootstrapped ranking, not from the initial ranking that is passed to perform bootstrapping.

this will require a further package, put on hold for the moment

taskSubset is deprecated
there is now
subset(x, top, tasks)

• if top is specified subset to top algorithms (only applicable to single task challenges)
• if tasks is specified subset of tasks (only applicable to multi task challenges)

20 might be a sensible number