Page MenuHomePhabricator
Feed Advanced Search

Oct 1 2020

wiesenfa added a comment to T27494: Check which files are still relevant.

what about
keep S3, compareRanks and extract.workflow
but do not export (i.e. remove from namespace) compareRanks and extract.workflow. they can then only be accessed e.g. by challengeR:::compareRanks(). These might be of practical use.
?

Oct 1 2020, 12:03 PM · challengeR

Sep 25 2020

wiesenfa moved T27447: significanceMap function raises warning about deprecated function call from In Progress to Done on the challengeR (v1.0) board.
Sep 25 2020, 12:08 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER2691d1ba70a6: fix grid in significancePlot (authored by wiesenfa).
fix grid in significancePlot
Sep 25 2020, 12:06 PM
wiesenfa committed rCHALLENGER0ab46fb7c444: Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into… (authored by wiesenfa).
Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into…
Sep 25 2020, 12:06 PM
wiesenfa moved T27447: significanceMap function raises warning about deprecated function call from Backlog to In Progress on the challengeR (v1.0) board.
Sep 25 2020, 10:17 AM · challengeR (v1.0)
wiesenfa claimed T27447: significanceMap function raises warning about deprecated function call.
Sep 25 2020, 9:47 AM · challengeR (v1.0)
wiesenfa created T27778: turn off networks.
Sep 25 2020, 9:45 AM · challengeR (v1.0)

Sep 24 2020

wiesenfa added a comment to T27326: autolayout networks in multi task report ugly.

I propose to turn off networks and exclude them from the report for now.
if this is solved, we can reintroduce them.
they are not essential I think. what do you think?

Sep 24 2020, 5:24 PM · challengeR
wiesenfa placed T27326: autolayout networks in multi task report ugly up for grabs.
Sep 24 2020, 4:26 PM · challengeR
wiesenfa moved T27777: Failing unit tests for challenge from Backlog to Done on the challengeR (v1.0) board.
Sep 24 2020, 4:24 PM · challengeR (v1.0)
wiesenfa claimed T27777: Failing unit tests for challenge.
Sep 24 2020, 4:24 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER083f7c671f05: fix test for as.challenge (authored by wiesenfa).
fix test for as.challenge
Sep 24 2020, 4:24 PM
wiesenfa moved T27776: Failing unit tests for subset of tasks from Backlog to Done on the challengeR (v1.0) board.
Sep 24 2020, 4:18 PM · challengeR (v1.0)
wiesenfa claimed T27776: Failing unit tests for subset of tasks.
Sep 24 2020, 4:18 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER08deba103b9e: fix subset (authored by wiesenfa).
fix subset
Sep 24 2020, 4:17 PM
wiesenfa moved T27708: ggplot2 dendrogram from In Progress to Backlog on the challengeR (v1.0) board.
Sep 24 2020, 3:28 PM · challengeR
wiesenfa added a comment to T27708: ggplot2 dendrogram .

this will require a further package, put on hold for the moment

Sep 24 2020, 3:28 PM · challengeR
wiesenfa moved T27685: changes in subset() from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 3:23 PM · challengeR (v1.0)
wiesenfa added a comment to T27685: changes in subset().

taskSubset is deprecated
there is now
subset(x, top, tasks)

  • if top is specified subset to top algorithms (only applicable to single task challenges)
  • if tasks is specified subset of tasks (only applicable to multi task challenges)
Sep 24 2020, 3:22 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGERaf8f561d7b94: fix subset(), remove taskSubset() (authored by wiesenfa).
fix subset(), remove taskSubset()
Sep 24 2020, 3:20 PM
wiesenfa updated subscribers of T27775: only export functions to be used by user.
Sep 24 2020, 3:17 PM · challengeR
wiesenfa triaged T27775: only export functions to be used by user as Low priority.
Sep 24 2020, 3:17 PM · challengeR
wiesenfa claimed T27326: autolayout networks in multi task report ugly.
Sep 24 2020, 2:43 PM · challengeR
wiesenfa moved T27708: ggplot2 dendrogram from Backlog to In Progress on the challengeR (v1.0) board.
Sep 24 2020, 2:42 PM · challengeR
wiesenfa added a comment to T27748: Decide on limit for algorithms in plot legends.

20 might be a sensible number

Sep 24 2020, 2:42 PM · challengeR (v1.0)
wiesenfa moved T27714: Report contains weird R output from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 2:40 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGERd34e4efe8ea5: fix output classes (authored by wiesenfa).
fix output classes
Sep 24 2020, 2:40 PM
wiesenfa placed T27288: separate readme.md and readme.Rmd up for grabs.
Sep 24 2020, 2:27 PM · challengeR (v1.0)
wiesenfa claimed T27708: ggplot2 dendrogram .
Sep 24 2020, 2:26 PM · challengeR
wiesenfa moved T27714: Report contains weird R output from Backlog to In Progress on the challengeR (v1.0) board.
Sep 24 2020, 2:26 PM · challengeR (v1.0)
wiesenfa claimed T27714: Report contains weird R output.
Sep 24 2020, 2:26 PM · challengeR (v1.0)
wiesenfa added a comment to T27770: Inconsistency in report description.

the sentence can be dropped. But wasn't it planned to revise the text in line with the paper in total?

Sep 24 2020, 2:23 PM · challengeR (v1.0)
wiesenfa moved T27685: changes in subset() from Backlog to In Progress on the challengeR (v1.0) board.
Sep 24 2020, 1:16 PM · challengeR (v1.0)
wiesenfa claimed T27685: changes in subset().
Sep 24 2020, 1:16 PM · challengeR (v1.0)
wiesenfa added a comment to T27774: wrong text in report on algorithm subsets.

Ok, weird. For me it says: "The top 5 out of 5 algorithms are considered."

Sep 24 2020, 1:13 PM · challengeR (v1.0)
wiesenfa added a comment to T27774: wrong text in report on algorithm subsets.
  1. it won't be more that I needed for the missing data description I guess;-) I still think that this sentence should be avoided if no subset is in fact used
Sep 24 2020, 1:11 PM · challengeR (v1.0)
wiesenfa moved T27453: Create complete network plot in its function from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 1:09 PM · challengeR (v1.0)
wiesenfa added a comment to T27453: Create complete network plot in its function.

this is more or less what I tried, ugliness is T27326. I think a ggplot solution mentioned there would be better

Sep 24 2020, 1:08 PM · challengeR (v1.0)
wiesenfa moved T27452: Provide plot function for dendrogram from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 1:07 PM · challengeR (v1.0)
wiesenfa moved T27337: Report contains wrong ranking method when rank-then-aggregate is used from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 1:06 PM · challengeR (v1.0)
wiesenfa moved T27294: add multiple figure output formats in report with option clean=TRUE from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 1:05 PM · challengeR (v1.0)
wiesenfa added a comment to T27294: add multiple figure output formats in report with option clean=TRUE.

report gains new argument "dpi" to change resolution

Sep 24 2020, 1:05 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER88b673042288: set dpi default to 150 (authored by wiesenfa).
set dpi default to 150
Sep 24 2020, 1:05 PM
wiesenfa committed rCHALLENGER88e1de7d8cc3: set dpi default to 300 (authored by wiesenfa).
set dpi default to 300
Sep 24 2020, 1:05 PM
wiesenfa committed rCHALLENGERb2891c3b4f02: Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into… (authored by wiesenfa).
Merge branch 'develop' of https://phabricator.mitk.org/source/challenger into…
Sep 24 2020, 1:00 PM
wiesenfa committed rCHALLENGERc66c18f1fbcd: allow setting dpi (authored by wiesenfa).
allow setting dpi
Sep 24 2020, 1:00 PM
wiesenfa added a comment to T27774: wrong text in report on algorithm subsets.
set.seed(4)
strip=runif(n,.9,1)
c_ideal=cbind(task="c_ideal",
              rbind(
                      data.frame(alg_name="A1",value=runif(n,.9,1),case=1:n),
                      data.frame(alg_name="A2",value=runif(n,.8,.89),case=1:n),
                      data.frame(alg_name="A3",value=runif(n,.7,.79),case=1:n),
                      data.frame(alg_name="A4",value=runif(n,.6,.69),case=1:n),
                      data.frame(alg_name="A5",value=runif(n,.5,.59),case=1:n)
              ))
challenge=as.challenge(c_ideal, 
                       algorithm="alg_name", case="case", value="value",
                       smallBetter = T)
ranking=challenge%>%aggregateThenRank(FUN = "mean", # aggregation function, 
                                      na.treat="na.rm", # either "na.rm" to remove missing data, 
                                      ties.method = "min" # a character string specifying 
)
Sep 24 2020, 12:18 PM · challengeR (v1.0)
wiesenfa moved T27338: Report contains misleading information on missing values from In Progress to Done on the challengeR (v1.0) board.
Sep 24 2020, 12:07 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER0e1a1e5dae75: description of missings in reports+as.challenge (authored by wiesenfa).
description of missings in reports+as.challenge
Sep 24 2020, 12:06 PM
wiesenfa added a comment to T27774: wrong text in report on algorithm subsets.

currently It says "The top 0 out of 0 algorithms are considered." in my latest example

Sep 24 2020, 11:46 AM · challengeR (v1.0)
wiesenfa added a comment to T27774: wrong text in report on algorithm subsets.
  1. I think it will be confusing if it writes "top 5 of 5 algorithms", I would put an if
  2. which changes? as.challenge does not have a full data attribute. The changes in subset have been done by you, see my comments in T27685.
Sep 24 2020, 11:45 AM · challengeR (v1.0)
wiesenfa added a comment to T27773: package dependencies.

yes I think so. maybe you have purrr (or dplyr) and ggplot loaded in the workspace.
packages in depends are loaded at startup (and we need these package to be available by the user). imports does not load the packages, functions of challengeR can however call them.
it is true that packages in depends should be minimize, but these are needed

Sep 24 2020, 11:38 AM · challengeR (v1.0)
wiesenfa triaged T27774: wrong text in report on algorithm subsets as High priority.
Sep 24 2020, 11:32 AM · challengeR (v1.0)
wiesenfa created T27773: package dependencies.
Sep 24 2020, 11:21 AM · challengeR (v1.0)

Sep 11 2020

wiesenfa added a comment to T27338: Report contains misleading information on missing values.

the first sentence refers to the fact that no missings have been found in the data set. would rephrasing to "0 missing cases entered in the data set have been found" help? the next sentence is about cases which have been inserted by the sanity check. Would adding "However, ..." in between help?
If you have any other suggestions, I would be happy to include them

Sep 11 2020, 11:42 AM · challengeR (v1.0)

Sep 7 2020

wiesenfa moved T27287: is warning given if na.treat unspecified in rank-first and obligatory otherwise from Backlog to Done on the challengeR (v1.0) board.
Sep 7 2020, 7:06 PM · challengeR (v1.0)
wiesenfa added a comment to T27456: Extraction of task subset from (bootstrap) ranking does not raise an error for invalid task name.

@eisenman I dont understand this task, could you give an example?

Sep 7 2020, 7:05 PM · challengeR (v1.0)
wiesenfa added a comment to T27287: is warning given if na.treat unspecified in rank-first and obligatory otherwise.

solved in T27338

Sep 7 2020, 7:03 PM · challengeR (v1.0)
wiesenfa created T27708: ggplot2 dendrogram .
Sep 7 2020, 7:01 PM · challengeR
wiesenfa added a comment to T27326: autolayout networks in multi task report ugly.

networks should be replaced by some ggplot2 implementation, see
https://journal.r-project.org/archive/2017/RJ-2017-023/RJ-2017-023.pdf

Sep 7 2020, 6:57 PM · challengeR
wiesenfa moved T27449: Consistent handling of plots created for multi-task data sets from In Progress to Done on the challengeR (v1.0) board.
Sep 7 2020, 6:55 PM · challengeR (v1.0)
wiesenfa added a comment to T27449: Consistent handling of plots created for multi-task data sets.

lists of ggplot objects can now be handled like single ggplot objects, e.g. boxbplot(...) + xlab(...) will apply xlab() to every ggplot object

Sep 7 2020, 6:54 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGERffd45a0c13bb: layouting of lists of ggplot objects (authored by wiesenfa).
layouting of lists of ggplot objects
Sep 7 2020, 6:49 PM
wiesenfa added a comment to T27447: significanceMap function raises warning about deprecated function call.

comes from significanceMap.data.frame

Sep 7 2020, 6:34 PM · challengeR (v1.0)
wiesenfa moved T27449: Consistent handling of plots created for multi-task data sets from Backlog to In Progress on the challengeR (v1.0) board.
Sep 7 2020, 6:21 PM · challengeR (v1.0)
wiesenfa added a comment to T27338: Report contains misleading information on missing values.

now as.challenge() as well as report contain extended information on missings and automatically inserted test cases.
@eisenman this changes the tests for as.challenge. Could you please 1. check whether the messages are well phrased and then 2. adapt tests, then 3. close task. Thanks!

Sep 7 2020, 2:40 PM · challengeR (v1.0)
wiesenfa moved T27338: Report contains misleading information on missing values from Backlog to In Progress on the challengeR (v1.0) board.
Sep 7 2020, 2:37 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGERc8b0952b6941: extend na.treat messages (authored by wiesenfa).
extend na.treat messages
Sep 7 2020, 2:37 PM
wiesenfa created T27707: na.treat obligatory for report .
Sep 7 2020, 2:24 PM · challengeR (v1.0)

Sep 4 2020

wiesenfa committed rCHALLENGER24782c58e8e3: fix column names (authored by wiesenfa).
fix column names
Sep 4 2020, 2:38 PM
wiesenfa added a comment to T27385: Variables holding the aggregated value are named differently for single- and multi-task challenges.

now
e.g. if FUN="mean" (i.e. a character) or if FUN=mean (i.e. a function)
the column name will be "value_mean"

Sep 4 2020, 2:32 PM · challengeR (v1.0)
wiesenfa moved T27385: Variables holding the aggregated value are named differently for single- and multi-task challenges from Backlog to In Progress on the challengeR (v1.0) board.
Sep 4 2020, 2:10 PM · challengeR (v1.0)
wiesenfa added a comment to T27252: Multiple cases per algorithm are not detected when missing data was added before.

@eisenman error message might have changed, please check tests

Sep 4 2020, 12:30 PM · challengeR (v1.0)
wiesenfa reassigned T27658: LaTeX Warning: There were undefined references from wiesenfa to eisenman.
Sep 4 2020, 12:22 PM · challengeR (v1.0)
wiesenfa added a comment to T27658: LaTeX Warning: There were undefined references.

@eisenman seems to come from "Blob plot similar to the one shown in Section \ref{blobByTask}" in visualizationAcrossTasks.Rmd. Could you please take care of it, you are more familiar with the restructured report

Sep 4 2020, 12:21 PM · challengeR (v1.0)
wiesenfa claimed T27658: LaTeX Warning: There were undefined references.
Sep 4 2020, 12:00 PM · challengeR (v1.0)
wiesenfa claimed T27385: Variables holding the aggregated value are named differently for single- and multi-task challenges.
Sep 4 2020, 12:00 PM · challengeR (v1.0)

Aug 31 2020

wiesenfa moved T27327: warning in docx report from In Progress to Done on the challengeR (v1.0) board.
Aug 31 2020, 6:33 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER4462382e4272: fixes (authored by wiesenfa).
fixes
Aug 31 2020, 6:25 PM
wiesenfa committed rCHALLENGERb3f747f61e4b: fix figure.format conflict with networks (authored by wiesenfa).
fix figure.format conflict with networks
Aug 31 2020, 6:25 PM
wiesenfa committed rCHALLENGER29fcaabf46e0: fix (authored by wiesenfa).
fix
Aug 31 2020, 6:25 PM
wiesenfa committed rCHALLENGERa5465970a236: add dendrogram (authored by wiesenfa).
add dendrogram
Aug 31 2020, 6:25 PM
wiesenfa committed rCHALLENGERb286e03e1e80: new func dendrogram() (authored by wiesenfa).
new func dendrogram()
Aug 31 2020, 6:25 PM
wiesenfa committed rCHALLENGER834c265c2168: fixes (authored by wiesenfa).
fixes
Aug 31 2020, 6:06 PM
wiesenfa moved T27327: warning in docx report from Backlog to In Progress on the challengeR (v1.0) board.
Aug 31 2020, 5:39 PM · challengeR (v1.0)
wiesenfa added a comment to T27327: warning in docx report.

should be solved with T27294

Aug 31 2020, 5:39 PM · challengeR (v1.0)
wiesenfa moved T27294: add multiple figure output formats in report with option clean=TRUE from Backlog to In Progress on the challengeR (v1.0) board.
Aug 31 2020, 5:37 PM · challengeR (v1.0)
wiesenfa added a comment to T27294: add multiple figure output formats in report with option clean=TRUE.

report() gains new argument fig.format to specify figure output formats in case of clean=FALSE.
can be vector, e.g. fig.format=c('jpeg','png', 'pdf'), then 3 files are stored for each figure

Aug 31 2020, 5:37 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER7303c65b21cc: allow specificaton of figure format in report (authored by wiesenfa).
allow specificaton of figure format in report
Aug 31 2020, 5:32 PM
wiesenfa updated the task description for T27658: LaTeX Warning: There were undefined references.
Aug 31 2020, 4:58 PM · challengeR (v1.0)
wiesenfa committed rCHALLENGER21f6bb076a05: ranking scheme description (authored by wiesenfa).
ranking scheme description
Aug 31 2020, 4:49 PM
wiesenfa moved T27337: Report contains wrong ranking method when rank-then-aggregate is used from Backlog to In Progress on the challengeR (v1.0) board.
Aug 31 2020, 4:42 PM · challengeR (v1.0)
wiesenfa added a comment to T27337: Report contains wrong ranking method when rank-then-aggregate is used.

actually it was

Aug 31 2020, 4:42 PM · challengeR (v1.0)
wiesenfa added a comment to T27452: Provide plot function for dendrogram.

@eisenman worked at least for one test data set. I guess this can be closed

Aug 31 2020, 4:29 PM · challengeR (v1.0)
wiesenfa added a comment to T27452: Provide plot function for dendrogram.

new function dendrogram(), also handles more informative titles ( T27411 ).
also updated in report

Aug 31 2020, 4:28 PM · challengeR (v1.0)
wiesenfa moved T27411: Improve labels in cluster dendrogram from In Progress to Done on the challengeR (v1.0) board.
Aug 31 2020, 4:24 PM · challengeR (v1.0)
wiesenfa moved T27411: Improve labels in cluster dendrogram from Backlog to In Progress on the challengeR (v1.0) board.
Aug 31 2020, 4:24 PM · challengeR (v1.0)
wiesenfa moved T27452: Provide plot function for dendrogram from Backlog to In Progress on the challengeR (v1.0) board.
Aug 31 2020, 4:24 PM · challengeR (v1.0)
wiesenfa added a comment to T27657: When missing algorithm performances are added as NAs, task is also NA.

@eisenman could affect tests. please close when finished

Aug 31 2020, 3:49 PM · challengeR (v1.0)
wiesenfa added a comment to T27253: Notify user when multi-task data set is interpreted as single-task data set.

@eisenman could affect tests. please close when finished

Aug 31 2020, 3:48 PM · challengeR (v1.0)