Q1. When will the test set be released? How much time do the participants have before submitting their solutions?

A1.  The test set will be released at 10 am CET on 6th September. The participants will be asked to submit their segmentations within 48 hours. The participants should upload the segmentations to Mega, WeTransfer, or Google Drive and send the download link to the organizer. 

Test data can be downloaded here. Please note that you are required to send us the segmentation map within 48 hours (Deadline: 10 am CET, 08 Sept). The submission format is the same as the one submitted to the leaderboard. Please upload the segmentation map to Google Drive, WeTransfer or MEGA, and share the download link with us. 

Q2. The annotation of Rater 3 in sub-task 02 of brain tumor segmentation seems noisy. How should I deal with it? 

A2. We claim that the annotation board for brain tumor segmentation is consistent across the train/validation/test sets. In sub-task 02, Rater 3 consistently segments the same structure (tumor core + enhancing tumor). Since one of the main objectives of the QUBIQ challenge is to model the distribution of multi-rater annotations while the behavior/knowledge of Rater 3 is consistent in train/validation/test sets, we consider that the annotation of Rater 3 is not erroneous in this regard.

Q3. Can we have a look at the evaluation codes? 

A3: Sure. Evaluation codes can be downloaded here.