Page MenuHomePhabricator

Error in stabilityByAlgorithm if only one case is used
Closed, ResolvedPublic

Description

The unit test code provided below leads to the following error: no applicable method for 'rename_' applied to an object of class "list"

If a second case is present in the data, the error does not occur.

test_that("blob plot for visualizing ranking stability by algorithm returns one plot if #algorithms < #tasks", {
  dataTask1 <- cbind(task="T1",
                     rbind(
                       data.frame(algo="A1", value=0.8, case="C1"),
                       data.frame(algo="A2", value=0.6, case="C1")
                     ))
  dataTask2 <- cbind(task="T2",
                     rbind(
                       data.frame(algo="A1", value=0.2, case="C1"),
                       data.frame(algo="A2", value=0.3, case="C1")
                     ))
  dataTask3 <- cbind(task="T3",
                     rbind(
                       data.frame(algo="A1", value=0.1, case="C1"),
                       data.frame(algo="A2", value=0.8, case="C1")
                     ))

  data <- rbind(dataTask1, dataTask2, dataTask3)

  challenge <- as.challenge(data, by="task", algorithm="algo", case="case", value="value", smallBetter=FALSE)

  ranking <- challenge%>%aggregateThenRank(FUN=median, ties.method="min")

  set.seed(1)
  rankingBootstrapped <- ranking%>%bootstrap(nboot=10)

  meanRanks <- ranking%>%consensus(method = "euclidean")

  actualPlot <- stabilityByAlgorithm(rankingBootstrapped, ordering = names(meanRanks), single = FALSE)
  expect_is(actualPlot, "ggplot")
})

Event Timeline

eisenman created this task.

bootstrap() now gives error if all tasks only contain 1 test case and a message if some tasks contain only 1 test case (tasks with 1 test case are excluded from bootstrapping)
test-bootstrap.R contains tests. @eisenman could you please check test

additional test in test-blobPlotStabilityByAlgorithm for the case "one task out of 3 tasks contains >1 test cases". in this case bootstrap() gives result for this remaining task and stability() only produces plot with this task. @eisenman could you please check and close?

Seems fine!

A multi-task bootstrapping report will contain a warning for each task having only one case in section 3.4 (line plots). But I think that's OK as well.

This issue will be fixed in v1.0.1.