Page MenuHomePhabricator

wiesenfa (Manuel Wiesenfarth)
User

Projects

User Details

User Since
Mar 2 2020, 3:23 PM (73 w, 5 d)

Recent Activity

May 4 2021

wiesenfa moved T28453: [external] Error when all metric values are the same from In Progress to Done on the challengeR board.
May 4 2021, 5:45 PM · challengeR
wiesenfa lowered the priority of T28477: feature request: subset of top performing algorithms according to consensus from Normal to Wishlist.
May 4 2021, 5:44 PM · challengeR
wiesenfa created T28477: feature request: subset of top performing algorithms according to consensus.
May 4 2021, 5:43 PM · challengeR

Apr 26 2021

wiesenfa added a comment to T28453: [external] Error when all metric values are the same.

@eisenman the change in develop branch has not been uploaded to GitHub, is this not automatically synchronized?. So the user who reported the bug still has the same problem. It would be good to merge into master as well

Apr 26 2021, 9:32 AM · challengeR
wiesenfa moved T28453: [external] Error when all metric values are the same from Done to In Progress on the challengeR board.
Apr 26 2021, 9:29 AM · challengeR

Apr 23 2021

wiesenfa added a comment to T28453: [external] Error when all metric values are the same.

now test case in test-report.R

Apr 23 2021, 1:01 PM · challengeR
wiesenfa committed rCHALLENGER00085cd910f5: T28453 (authored by wiesenfa).
T28453
Apr 23 2021, 1:00 PM

Apr 22 2021

wiesenfa added a comment to T28453: [external] Error when all metric values are the same.

very simple fix in rankingHeatmap.challenge
@eisenman can this be merged into master branch?

Apr 22 2021, 10:30 PM · challengeR
wiesenfa moved T28453: [external] Error when all metric values are the same from In Progress to Done on the challengeR board.
Apr 22 2021, 10:28 PM · challengeR
wiesenfa moved T28453: [external] Error when all metric values are the same from Backlog to In Progress on the challengeR board.
Apr 22 2021, 10:27 PM · challengeR
wiesenfa committed rCHALLENGER6ceb955c61a3: Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into… (authored by wiesenfa).
Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into…
Apr 22 2021, 10:27 PM
wiesenfa committed rCHALLENGER3f5ea60c7f28: hotfix T28453 (authored by wiesenfa).
hotfix T28453
Apr 22 2021, 10:27 PM

Apr 19 2021

wiesenfa created T28453: [external] Error when all metric values are the same.
Apr 19 2021, 9:26 AM · challengeR

Mar 3 2021

wiesenfa added a comment to T28346: Package version mismatch during upgrade.

graph had only been used for networks, I guess version 1.62 is sufficient

Mar 3 2021, 7:45 PM · challengeR

Jan 27 2021

wiesenfa created T28210: update citation of paper to scientific reports.
Jan 27 2021, 12:25 PM · challengeR
wiesenfa added a comment to T28202: Error in stabilityByAlgorithm if only one case is used.

additional test in test-blobPlotStabilityByAlgorithm for the case "one task out of 3 tasks contains >1 test cases". in this case bootstrap() gives result for this remaining task and stability() only produces plot with this task. @eisenman could you please check and close?

Jan 27 2021, 11:49 AM · challengeR
wiesenfa committed rCHALLENGER8ebf10fba330: test only 1 tast >1 test case (authored by wiesenfa).
test only 1 tast >1 test case
Jan 27 2021, 11:47 AM
wiesenfa added a comment to T28193: bug in stabilityByAlgorithm.bootstrap.list.

@eisenman yes, should be sufficient
only one case per task covered in bootstrap() now see T28202

Jan 27 2021, 11:38 AM · challengeR
wiesenfa added a comment to T28202: Error in stabilityByAlgorithm if only one case is used.

bootstrap() now gives error if all tasks only contain 1 test case and a message if some tasks contain only 1 test case (tasks with 1 test case are excluded from bootstrapping)
test-bootstrap.R contains tests. @eisenman could you please check test

Jan 27 2021, 11:32 AM · challengeR
wiesenfa committed rCHALLENGER6bd398e8f0c1: T28202 - bootstrapping 1 test case message/error (authored by wiesenfa).
T28202 - bootstrapping 1 test case message/error
Jan 27 2021, 11:30 AM
wiesenfa committed rCHALLENGERaf8b7633ba1f: tests for bootstrapping with 1 test case (authored by wiesenfa).
tests for bootstrapping with 1 test case
Jan 27 2021, 11:30 AM
wiesenfa committed rCHALLENGER07dfc6359329: error/message if bootstrapping only 1 test case (authored by wiesenfa).
error/message if bootstrapping only 1 test case
Jan 27 2021, 11:30 AM
wiesenfa claimed T28202: Error in stabilityByAlgorithm if only one case is used.
Jan 27 2021, 10:08 AM · challengeR
wiesenfa moved T28202: Error in stabilityByAlgorithm if only one case is used from Backlog to In Progress on the challengeR board.
Jan 27 2021, 10:08 AM · challengeR

Jan 25 2021

wiesenfa added a comment to T28193: bug in stabilityByAlgorithm.bootstrap.list.

task names ( names(x$matlist) ) had been used for facet titles instead of algorithm names ( stored in "ordering").

Jan 25 2021, 7:40 PM · challengeR
wiesenfa moved T28194: drop restriction at least 3 algorithms required for bootstrapping from In Progress to Done on the challengeR board.
Jan 25 2021, 12:02 PM · challengeR
wiesenfa moved T28193: bug in stabilityByAlgorithm.bootstrap.list from In Progress to Done on the challengeR board.
Jan 25 2021, 12:02 PM · challengeR
wiesenfa committed rCHALLENGERf89ab5d8b1e1: revert (authored by wiesenfa).
revert
Jan 25 2021, 12:01 PM
wiesenfa committed rCHALLENGERfb258fbf1b25: allow only 2 algorithms in bootstrapping (authored by wiesenfa).
allow only 2 algorithms in bootstrapping
Jan 25 2021, 12:01 PM
wiesenfa committed rCHALLENGER4430defcf69d: T28194 allow 2 algorithms for bootstrapping (authored by wiesenfa).
T28194 allow 2 algorithms for bootstrapping
Jan 25 2021, 12:01 PM
wiesenfa committed rCHALLENGER83bcd8eeb603: fix facet names (authored by wiesenfa).
fix facet names
Jan 25 2021, 12:01 PM
wiesenfa moved T28194: drop restriction at least 3 algorithms required for bootstrapping from Backlog to In Progress on the challengeR board.
Jan 25 2021, 11:55 AM · challengeR
wiesenfa created T28194: drop restriction at least 3 algorithms required for bootstrapping.
Jan 25 2021, 11:55 AM · challengeR
wiesenfa moved T28193: bug in stabilityByAlgorithm.bootstrap.list from Backlog to In Progress on the challengeR board.
Jan 25 2021, 11:53 AM · challengeR
wiesenfa created T28193: bug in stabilityByAlgorithm.bootstrap.list.
Jan 25 2021, 11:53 AM · challengeR

Dec 18 2020

wiesenfa created T28124: add line plot (like methods plot) across tasks to report.
Dec 18 2020, 6:28 PM · challengeR

Dec 17 2020

wiesenfa added a comment to T28119: Unit tests are failing on Ubuntu due to mismatching messages.

platform is output with Sys.info(); linux and Mac systems are equal and windows may differ

Dec 17 2020, 5:55 PM · challengeR (v1.0)
wiesenfa added a comment to T28119: Unit tests are failing on Ubuntu due to mismatching messages.

good idea!
otherwise you could use a platform dependent expect_equal but I think your idea is sufficient

Dec 17 2020, 5:53 PM · challengeR (v1.0)
wiesenfa added a comment to T28121: Remove parallel bootstrapping from quick-start scripts.

would mention it there at least though because bootstrapping may be time consuming

Dec 17 2020, 5:14 PM · challengeR (v1.0)
wiesenfa added a comment to T28097: test errors in aggregateThenRank and rankThenAggregate.

don't think its necessary. there is anyway ordering applied to it in later stages.
otherwise ranking step (rank.aggregated.list()) could always do the ordering directly (then the test would check whether ordering is working). As mentioned before I avoided such change before the release because of (very unlikely) risk of breaking something in the first level hierarchy

Dec 17 2020, 5:13 PM · challengeR (v1.0)
wiesenfa added a comment to T28119: Unit tests are failing on Ubuntu due to mismatching messages.

I do see those as well on Mac

Dec 17 2020, 4:46 PM · challengeR (v1.0)
wiesenfa added a comment to T28097: test errors in aggregateThenRank and rankThenAggregate.

R 4.0.2.
testthat previously 2.3.2, now 3.0.1.
the issue is however not connected to test_that I see that in the first 2 tests in aggregate-then-rank the ordering in the data set is not used but sorted

Dec 17 2020, 4:45 PM · challengeR (v1.0)

Dec 14 2020

wiesenfa added a comment to T28097: test errors in aggregateThenRank and rankThenAggregate.

seems to not fail on windows machines. but does on my Mac (and thus might also on linux). minor issue, defer to post 1.0.0

Dec 14 2020, 6:47 PM · challengeR (v1.0)
wiesenfa moved T28112: colors in box plots multitask not consistent from In Progress to Done on the challengeR (v1.0) board.
Dec 14 2020, 2:09 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER018edb100eae: T28112: respect algo colors in boxplots (authored by wiesenfa).
T28112: respect algo colors in boxplots
Dec 14 2020, 2:08 PM
wiesenfa moved T28112: colors in box plots multitask not consistent from Backlog to In Progress on the challengeR (v1.0) board.
Dec 14 2020, 2:07 PM · challengeR (v1.0)
wiesenfa created T28112: colors in box plots multitask not consistent.
Dec 14 2020, 1:33 PM · challengeR (v1.0)

Dec 10 2020

wiesenfa moved T27292: Fix toolkit authors from In Progress to Done on the challengeR (v1.0) board.
Dec 10 2020, 1:25 PM · challengeR (v1.0)
wiesenfa moved T27292: Fix toolkit authors from Backlog to In Progress on the challengeR (v1.0) board.
Dec 10 2020, 1:25 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER9217054b61ae: fix authors (authored by wiesenfa).
fix authors
Dec 10 2020, 1:25 PM
wiesenfa created T28097: test errors in aggregateThenRank and rankThenAggregate.
Dec 10 2020, 11:11 AM · challengeR (v1.0)
wiesenfa moved T28085: Report generation is broken from In Progress to Done on the challengeR (v1.0) board.
Dec 10 2020, 11:09 AM · challengeR (v1.0)
wiesenfa moved T28085: Report generation is broken from Backlog to In Progress on the challengeR (v1.0) board.
Dec 10 2020, 11:09 AM · challengeR (v1.0)
wiesenfa claimed T28085: Report generation is broken.
Dec 10 2020, 11:09 AM · challengeR (v1.0)
wiesenfa committed rCHALLENGER484d1c685a70: bug fixes (authored by wiesenfa).
bug fixes
Dec 10 2020, 11:08 AM
wiesenfa committed rCHALLENGER8fe6dd4d9574: ensure english warnings while testing (authored by wiesenfa).
ensure english warnings while testing
Dec 10 2020, 11:08 AM

Dec 7 2020

wiesenfa added a comment to T27925: Use cases of annotator option.

would defer to after release

Dec 7 2020, 3:58 PM · challengeR
wiesenfa added a comment to T27494: Check which files are still relevant.

please close if ok

Dec 7 2020, 3:58 PM · challengeR
wiesenfa committed rCHALLENGER77411271cd7a: fix podium (authored by wiesenfa).
fix podium
Dec 7 2020, 3:51 PM
wiesenfa committed rCHALLENGERd7c27e2a6dad: fix (authored by wiesenfa).
fix
Dec 7 2020, 3:51 PM
wiesenfa committed rCHALLENGER4a677bed212f: style (authored by wiesenfa).
style
Dec 7 2020, 3:51 PM
wiesenfa committed rCHALLENGER38ac623e8adc: add annotor (authored by wiesenfa).
add annotor
Dec 7 2020, 3:51 PM
wiesenfa added a comment to T27925: Use cases of annotator option.

added to documentation of as.challenge():
(arg annotator:) If multiple annotators annotated the test cases, a string specifying the name of the column that contains the annotator identifiers. Only applies to rang-then-aggregate. Use with caution: Currently not tested.

Dec 7 2020, 3:18 PM · challengeR
wiesenfa moved T27925: Use cases of annotator option from Backlog to In Progress on the challengeR (v1.0) board.
Dec 7 2020, 3:16 PM · challengeR
wiesenfa committed rCHALLENGER17905fc80c85: add man for annotator (authored by wiesenfa).
add man for annotator
Dec 7 2020, 3:16 PM
wiesenfa added a comment to T27971: Provide code for podium plot in vignette "Visualizations".

added

podium(ranking)

to vignette.
podium() does have a different syntax for layouting (it is not ggplot2 but base graphics) but vignette does not describe layouting for ggplot2 plots either

Dec 7 2020, 2:20 PM · challengeR (v1.0)
wiesenfa moved T27971: Provide code for podium plot in vignette "Visualizations" from Backlog to Done on the challengeR (v1.0) board.
Dec 7 2020, 2:18 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER6267c97aeb1b: T27971 podium in visualizations vignette (authored by wiesenfa).
T27971 podium in visualizations vignette
Dec 7 2020, 2:13 PM
wiesenfa committed rCHALLENGERf899ce83619a: add euclidean consensus man (authored by wiesenfa).
add euclidean consensus man
Dec 7 2020, 2:13 PM
wiesenfa committed rCHALLENGER269f169422a0: update man (authored by wiesenfa).
update man
Dec 7 2020, 2:13 PM
wiesenfa committed rCHALLENGERccd930a457fd: add details for euclidean consensus (authored by wiesenfa).
add details for euclidean consensus
Dec 7 2020, 2:13 PM
wiesenfa placed T27998: Interpretation of consensus ranking values needs further explanation up for grabs.
Dec 7 2020, 2:03 PM · challengeR (v1.0)
wiesenfa moved T27998: Interpretation of consensus ranking values needs further explanation from Backlog to In Progress on the challengeR (v1.0) board.
Dec 7 2020, 2:03 PM · challengeR (v1.0)
wiesenfa added a comment to T27998: Interpretation of consensus ranking values needs further explanation.

report() does not know what type of consensus ranking method had been used.
I added sentence
"Consensus ranking according to mean ranks across tasks if method="euclidean" where in case of ties (equal ranks for multiple algorithms) the average rank is used, i.e. ties.method="average"."
to help for consensus() and would suggest to also add this to the vignette/readme, but not to report. Could someone of you please take care of this?

Dec 7 2020, 2:03 PM · challengeR (v1.0)
wiesenfa added a comment to T27775: only export functions to be used by user.

@eisenman do you want to do this with roxygen?

Dec 7 2020, 1:10 PM · challengeR
wiesenfa assigned T27494: Check which files are still relevant to eisenman.
Dec 7 2020, 1:09 PM · challengeR
wiesenfa assigned T28016: embed web app in package to eisenman.
Dec 7 2020, 1:09 PM · challengeR (v1.0)
wiesenfa claimed T27998: Interpretation of consensus ranking values needs further explanation.
Dec 7 2020, 1:08 PM · challengeR (v1.0)
wiesenfa moved T28016: embed web app in package from Backlog to In Progress on the challengeR (v1.0) board.
Dec 7 2020, 1:08 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER2ade85d374f7: fix tests (authored by wiesenfa).
fix tests
Dec 7 2020, 1:07 PM
wiesenfa committed rCHALLENGER3578ab9be39c: allow consensus named numeric vector or ordered character vector (authored by wiesenfa).
allow consensus named numeric vector or ordered character vector
Dec 7 2020, 1:07 PM
wiesenfa committed rCHALLENGER6fc162a3bcde: remove compare.rank (authored by wiesenfa).
remove compare.rank
Dec 7 2020, 1:07 PM
wiesenfa committed rCHALLENGER019ec7bb9392: remove winner.ranked (authored by wiesenfa).
remove winner.ranked
Dec 7 2020, 1:07 PM
wiesenfa committed rCHALLENGERb925ab764dba: fix (authored by wiesenfa).
fix
Dec 7 2020, 1:07 PM
wiesenfa committed rCHALLENGERdeb1db150dac: T27943 allow ordering numeric (authored by wiesenfa).
T27943 allow ordering numeric
Dec 7 2020, 1:07 PM
wiesenfa moved T27943: Plot function stability() raises error when used with default values from In Progress to Done on the challengeR (v1.0) board.
Dec 7 2020, 1:07 PM · challengeR (v1.0)
wiesenfa added a comment to T27494: Check which files are still relevant.

kept select.if(), winner(), extract.workfolow and compareRanks()
and removed everything not supported anymore.
as.warehouse (benchmarkUtils) is not exported, recommend to leave because this may come handy for specific situations

Dec 7 2020, 1:07 PM · challengeR
wiesenfa moved T27494: Check which files are still relevant from Backlog to In Progress on the challengeR (v1.0) board.
Dec 7 2020, 12:56 PM · challengeR
wiesenfa moved T27943: Plot function stability() raises error when used with default values from Backlog to In Progress on the challengeR (v1.0) board.
Dec 7 2020, 11:21 AM · challengeR (v1.0)
wiesenfa added a comment to T27494: Check which files are still relevant.

removed second()

Dec 7 2020, 11:19 AM · challengeR

Dec 4 2020

wiesenfa committed rCHALLENGER6144decf510d: delete funciton second() (unmaintained) (authored by wiesenfa).
delete funciton second() (unmaintained)
Dec 4 2020, 8:49 PM
wiesenfa placed T27971: Provide code for podium plot in vignette "Visualizations" up for grabs.
Dec 4 2020, 8:46 PM · challengeR (v1.0)
wiesenfa added a comment to T27971: Provide code for podium plot in vignette "Visualizations".

What is the problem?
in the extend of the vignette it would be simply

Dec 4 2020, 8:46 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER2e04538d9e1d: podium default topright legend (authored by wiesenfa).
podium default topright legend
Dec 4 2020, 8:44 PM
wiesenfa moved T27915: Podium plot shrinks for long algorithm names from In Progress to Done on the challengeR (v1.0) board.
Dec 4 2020, 8:40 PM · challengeR (v1.0)
wiesenfa added a comment to T27915: Podium plot shrinks for long algorithm names.

legend now always at bottom and dynamically adjusts to number of algorithms and algorithm name length.
Took me almost the day...

Dec 4 2020, 8:40 PM · challengeR (v1.0)
wiesenfa moved T27748: Decide on limit for algorithms in plot legends from In Progress to Done on the challengeR (v1.0) board.
Dec 4 2020, 8:39 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER26eb63cdef48: report podium plot dynamic legend bottom (authored by wiesenfa).
report podium plot dynamic legend bottom
Dec 4 2020, 8:38 PM
wiesenfa committed rCHALLENGER019b63757493: bottom legend in line plot if n.algorithms>20 (authored by wiesenfa).
bottom legend in line plot if n.algorithms>20
Dec 4 2020, 8:38 PM
wiesenfa committed rCHALLENGERdad94cd28be0: bottom legend in stacked frequency if n.tasks>20 (authored by wiesenfa).
bottom legend in stacked frequency if n.tasks>20
Dec 4 2020, 8:38 PM