Page MenuHomePhabricator

challengeRProject
ActivePublic

Details

Description

Methods and open-source toolkit for analyzing and visualizing challenge results

Recent Activity

May 14 2020

reinkea moved T27412: Description of variability of achieved rankings across tasks is missing in short report from v1.0 to Done on the challengeR board.
May 14 2020, 8:47 AM · challengeR (v1.0)

Apr 3 2020

reinkea created T27292: Fix toolkit authors.
Apr 3 2020, 4:12 PM · challengeR (v1.0)

Apr 2 2020

eisenman added a comment to T27253: Notify user when multi-task data set is interpreted as single-task data set.

At the meeting on 30/03/2020 we agreed that the messages should be extended to warn the user of that scenario. Since the column containing the tasks is unknown when the parameter "by" is not specified, an automatic detection can only be realized with a lot of effort.

Apr 2 2020, 11:21 AM · challengeR (v1.0)
eisenman closed T27254: Challenge attributes are arranged differently depending on sanity check as Wontfix.

In the meeting on 30/03/2020 we agreed that this issue won't be fixed. Cases can appear more than once per algorithm in bootstrapping and thus the sanity check has to be turned off. Performance reasons were also mentioned.

Apr 2 2020, 11:10 AM · challengeR
eisenman closed T27256: Changes during sanity check can constradict NA handling option as Wontfix.

In the meeting on 30/03/2020 we agreed that this issue won't be fixed. Parameters have to be described properly to raise awareness of this scenario to the user.

Apr 2 2020, 11:06 AM · challengeR

Mar 31 2020

eisenman moved T27237: Implement tests for ranking method "aggregate-then-rank" from Backlog to In Progress on the challengeR board.
Mar 31 2020, 9:15 PM · challengeR (v1.0)
eisenman triaged T27282: Implement tests for ranking method "test-then-rank" based on Wilcoxon signed rank test as High priority.
Mar 31 2020, 9:14 PM · challengeR (v1.0)
eisenman triaged T27281: Implement tests for ranking method "rank-then-aggregate" as High priority.
Mar 31 2020, 9:13 PM · challengeR (v1.0)
eisenman renamed T27237: Implement tests for ranking method "aggregate-then-rank" from Implement unit tests to Implement tests for ranking method "aggregate-then-rank".
Mar 31 2020, 9:12 PM · challengeR (v1.0)
eisenman added a comment to T27252: Multiple cases per algorithm are not detected when missing data was added before.

The given example is already available as a unit test.

Mar 31 2020, 12:33 PM · challengeR (v1.0)
eisenman moved T27244: Implement tests for the "challenge" class from In Progress to Done on the challengeR board.
Mar 31 2020, 12:28 PM · challengeR (v1.0)

Mar 30 2020

aguilera added a comment to T27241: Readme improvements.

After discussing, the next steps have been decided:

  1. @wiesenfa: look into a way to output a the readme.md in another way (probably in another folder)
  2. @aguilera: create scripts to extend the code from the readme, in a different folder
  3. @aguilera: crete a troubleshooting section
Mar 30 2020, 4:51 PM · challengeR (v1.0)
eisenman moved T27252: Multiple cases per algorithm are not detected when missing data was added before from Backlog to In Progress on the challengeR board.
Mar 30 2020, 4:28 PM · challengeR (v1.0)
eisenman assigned T27252: Multiple cases per algorithm are not detected when missing data was added before to wiesenfa.
Mar 30 2020, 4:28 PM · challengeR (v1.0)
eisenman triaged T27278: Writing documentation as High priority.
Mar 30 2020, 4:26 PM · challengeR (v1.0)

Mar 27 2020

wiesenfa added a comment to T27241: Readme improvements.

There should be a team section where all of you are acknowledged

Mar 27 2020, 10:48 AM · challengeR (v1.0)
wiesenfa added a comment to T27256: Changes during sanity check can constradict NA handling option.

in my opinion this is a desirable feature

Mar 27 2020, 10:43 AM · challengeR

Mar 26 2020

eisenman created T27256: Changes during sanity check can constradict NA handling option.
Mar 26 2020, 3:57 PM · challengeR

Mar 25 2020

eisenman created T27254: Challenge attributes are arranged differently depending on sanity check.
Mar 25 2020, 4:46 PM · challengeR
eisenman created T27253: Notify user when multi-task data set is interpreted as single-task data set.
Mar 25 2020, 1:33 PM · challengeR (v1.0)

Mar 24 2020

eisenman updated subscribers of T27252: Multiple cases per algorithm are not detected when missing data was added before.
Mar 24 2020, 6:35 PM · challengeR (v1.0)
eisenman updated the task description for T27252: Multiple cases per algorithm are not detected when missing data was added before.
Mar 24 2020, 6:02 PM · challengeR (v1.0)
eisenman triaged T27252: Multiple cases per algorithm are not detected when missing data was added before as High priority.
Mar 24 2020, 5:59 PM · challengeR (v1.0)
eisenman added a project to T27244: Implement tests for the "challenge" class: challengeR.
Mar 24 2020, 3:10 PM · challengeR (v1.0)

Mar 23 2020

eisenman added a member for challengeR: onogur.
Mar 23 2020, 11:52 AM

Mar 20 2020

aguilera updated the task description for T27241: Readme improvements.
Mar 20 2020, 9:15 AM · challengeR (v1.0)

Mar 19 2020

eisenman reopened T27236: Set up test infrastructure as "Open".

Make the task visible in the workboard column "Done".

Mar 19 2020, 6:13 PM · challengeR (v1.0)
eisenman moved T27244: Implement tests for the "challenge" class from Backlog to In Progress on the challengeR board.
Mar 19 2020, 5:53 PM · challengeR (v1.0)
eisenman created T27244: Implement tests for the "challenge" class.
Mar 19 2020, 5:53 PM · challengeR (v1.0)
eisenman closed T27236: Set up test infrastructure as Resolved.
Mar 19 2020, 2:47 PM · challengeR (v1.0)

Mar 18 2020

aguilera updated subscribers of T27241: Readme improvements.
Mar 18 2020, 11:57 AM · challengeR (v1.0)
aguilera created T27241: Readme improvements.
Mar 18 2020, 11:56 AM · challengeR (v1.0)

Mar 17 2020

eisenman triaged T27237: Implement tests for ranking method "aggregate-then-rank" as Normal priority.
Mar 17 2020, 5:18 PM · challengeR (v1.0)
eisenman updated the task description for T27236: Set up test infrastructure.
Mar 17 2020, 4:41 PM · challengeR (v1.0)
eisenman moved T27236: Set up test infrastructure from Backlog to In Progress on the challengeR board.
Mar 17 2020, 4:41 PM · challengeR (v1.0)
eisenman triaged T27236: Set up test infrastructure as High priority.
Mar 17 2020, 4:39 PM · challengeR (v1.0)
eisenman set the image for challengeR to F1532360: profile.
Mar 17 2020, 4:18 PM
eisenman edited Description on challengeR.
Mar 17 2020, 4:16 PM
eisenman added a member for challengeR: aguilera.
Mar 17 2020, 1:10 PM
kislinsk created challengeR.
Mar 17 2020, 12:40 PM