Page MenuHomePhabricator

Variables holding the aggregated value are named differently for single- and multi-task challenges
Open, NormalPublic

Description

The returned ranking object of rankThenAggregate contains different variables holding the result of the aggregation depending on the data set.

For single-task challenges the name is rank_mean (if mean is chosen as aggregation function). For multi-task challenges the name is rank_FUN (independent from the chosen aggregation function). In aggregateThenRank the variable is named value_FUN for both single- and multi-task challenges.

The variables should be named consistently to avoid confusion.

Related Objects

Event Timeline

eisenman created this task.May 8 2020, 9:42 AM

@wiesenfa: Do you have any preference regarding the naming scheme?

Did you take care that FUN="mean" and FUN=mean is handled differently (in the former case it is a name, in the latter case it is a function)?

probably

  • "aggregated value" possibly with underscore in case aggregation name cannot be identified (e.g. if it is a function)
  • otherwise e.g. if FUN="mean" it would be nice if it would be "mean value", to simplify it could be still "aggregated value"
  • if FUN="significance" something possibly nicer than prop.sign. , but not "aggregated value"
wiesenfa claimed this task.Sep 4 2020, 12:00 PM

now
e.g. if FUN="mean" (i.e. a character) or if FUN=mean (i.e. a function)
the column name will be "value_mean"

e.g. if FUN=function(x) quantile(x, prob = 0.5)
the column name will be "value_quantile(x, prob = 0.5)", i.e. carrying the function name.

@eisenman several tests unrelated to the fix fail (everything with "(reverse order)" is in the wrong order, which surprises me, should have failed already before, but I am unsure about this) could you please check?

The tests in test-rankThenAggregate.R are failing because the expectation is "rank_FUN" but should be "rank_mean" or "rank_median" respectively after the changes. This will be fixed in T27694.

eisenman moved this task from In Progress to Done on the challengeR (v1.0) board.Sep 10 2020, 11:47 AM