Page MenuHomePhabricator

feature request: subset of top performing algorithms according to consensus
Open, WishlistPublic

Description

feature request by user:
in case of multiple tasks

subset.bootstrap.list(ranking_bootstrapped, top = 10)

could provide top 10 performing algorithms according to consensus ranking. Currently an error is given
"Subset of algorithms only sensible for single-task challenges. Otherwise no consensus ranking is possible."

the complication arises because in the task specific plots, the selected algorithms should provide the task specific rank, such that for a specific task there may be no rank 1 for example because the best algorithm for that task has been omitted because it is not among the best 10 according to consensus.

Event Timeline

wiesenfa lowered the priority of this task from Normal to Wishlist.May 4 2021, 5:43 PM
wiesenfa created this task.