Warning messages when there are missing values in the data were reviewed as below:

I added tests in current future branch for checking class of “algorithm” column in challenge object.

First, I tried the fix with R 3.6 and can confirm that it does not break the functionality there.

I have added object[[algorithm]] <- as.factor(object[[algorithm]]) to challengeR.R as you suggested. Now everything works without any problem. No need of stating stringsAsFactors anymore during CSV read.

Thank you so much @aekavur ! It helps a lot to understand the reason finally!

Hi again :)

if the output is NULL, object[[by]] is not a factor, i.e. class(object[[by]]) is "character", in this case you need to use use unique() and probably your solution

Hi again,

Thanks Emre!
Thats a weird change. I didn't find any mention in R changelog.
probably instead of

algorithms=factor(unique(object[[by]]))

it will be preferred

Congrats for tracing this down!

Finally, I could find the source of the bug. 😊 It is caused by changed output type of unique() function in R:Base from R-3 to R-4.

Update: I go deeper with debugging and some reverse engineering. The source of the problem is the miss-ranking of the algorithms. When we use testThenRank and rankThenAggregate methods, the ranking of the algorithms seems wrong, as it is seen on the screenshot (marked with red rectangular). There is no problem with aggregateThenRank method so far.

In T29167#238218, @wiesenfa wrote:

From the change log for R 4.2.0

GRAPHICS:

• The graphics engine version, R_GE_version, has been bumped to 15 and so packages that provide graphics devices should be reinstalled.

Maybe just reinstalling the package helps?

From the change log for R 4.2.0

I have detected the same problem with R 4.2.0. It seems, it is caused by major version change of R. We need to go deeper and find the source of this change.

Could someone with R v4 double-check locally as well?

I checked on my local Windows system (R 3.6.3) and the plots seem to be correct.

The problem is almost fixed by giving na.treat parameter in both as.challenge and ranking methods (except rankThenAggregate). Now we can generate reports for all ranking methods.

I guess na.treat it is only needed for the line plot for comparing to other ranking methods?
In this case, a message could be thrown when compiling the report saying something like "line plot comparing ranking methods omitted since na.treat is not specified. Specify na.treat in as.challenge() if inclusion of line plot is desired" and allow compilation of the report (excluding line plot).
(Note that you can define na.treat both in as.challenge() as well as in the ranking functions).

Thank you for investigating this! In challengeR it is covered in the way that a message is emitted saying "na.treat obligatory if report is intended to be compiled". In order to solve the mentioned issue 2, a strategy for the preferred way to handle it in VISSART should be defined. Should the user be guided to specify the NaN handling strategy? Should the user be able to generate a report but without the plots that require numeric values?

Current status of the issue:

visart@dkfz-heidelberg.de mail list has been configured by ITCF. The mails that sent to this address will be forwarded to people on the list.

Configuration summary on report generation page has been added. Users can check their preferences just before report generation.

For upload process, dash-uploader library is used. This library offers built-in progress bar support. Users can see the uploading stage.

It seems there is no practical way to handle this. Dash offers loading_state property for each component. This is useable (callable) via CSS or JS codes.

Descriptions for each ranking method were added.

Deployment configurations were completed. A user guide was added to the Readme file in the root folder.

One solution to approach this issue is to check if NaN values are present in the data. If so, the user should be able to select the NaN handling strategy if necessary (e.g., not for case-based ranking). So what about providing the NaN handling options after the chosen ranking method is known?

The reason for the problem is that the server is configured to accept connections from only authorized users. ITFC added Ali Emre Kavur and Fabian Isensee as authorized users.