Dear organisers, We are writing to ask for some clarification regarding the evaluation of our submission: - It seems unclear to us which metrics will be used to evaluate our submission. We understood that the F1-score would be used at the pixel-level and the AP would be used at the sample-level, is that correct? In such case, can you confirm that the implementation of the AP at the sample-level is given by `process_file_samplewise` from https://github.com/MIC-DKFZ/mood/blob/master/scripts/evalresults.py#L64 and then sklearn.metrics.average_precision_score? And that the pixel-level F1-score is computed using `get_f1_score_clean_list` from https://www.synapse.org/Synapse:syn21343101/discussion/threadId=10485, meaning that `process_file_pixelwise` from https://github.com/MIC-DKFZ/mood/blob/master/scripts/evalresults.py#L19 is never used? - Could you please explain how we can obtain ?X * Y * Z scores per sample (where X * Y * Z is the dimensionality of the data sample)? for pixel-level as stated on Synapse? From our understanding, the given code does not compute X * Y * Z scores per sample, but rather a score depending on the number of ground truth and predicted objects. - Should we add _Medical Out of Distribution Analysis Challenge 2020 Organizers_ as viewers to our project or is there a specific team for 2024? - Lastly, can you confirm that we are allowed an unlimited number of submissions to the task ?Toy Examples - Pixel-Level? and ?Toy Examples - Sample-Level? to test the docker on the submission system and obtain the toy-test scores, whereas we are only allowed 3 submissions to the entire testing set? Many thanks in advance, Best wishes, CB, TDV, HR & MS