Hello,
I am currently participating in the BraTS challenge and working on the Pathology task. I am ready to submit my predictions, but I do not see an option to submit them on Synapse for this specific task.
Could anyone provide information on when it will be possible to evaluate our submissions? Any updates on the timeline would be greatly appreciated.
Thank you!
Created by Luis Carlos Rivera Monroy luiscarm9 Hey @luiscarm9,
You can find details on how to generate the input file [here](https://docs.mlcommons.org/GaNDLF/usage/#generate-metrics). Please note that you need to be using a [different branch](https://github.com/mlcommons/GaNDLF/tree/new-apis_v0.1.0-dev) for the full functionality.
Cheers,
Sarthak Hi @sarthakpati,
Thanks for your answer. I still have some problems with the validation.
I'm not sure if my file is incorrectly formatted or if there is another issue, because I always get similar performance (a horrible one, haha), even when using different models. Could you maybe provide a full example of how the CSV file should look?
Best regards,
Luis Carlos Hi @luiscarm9,
This depends on which metrics you are talking about. It is definitely possible that multiple metrics will have the same value (we have seen this ourselves during testing).
There are no custom implementations for the metrics. Everything is taken from Torchmetrics [[ref1](https://github.com/mlcommons/GaNDLF/blob/new-apis_v0.1.0-dev/GANDLF/metrics/classification.py#L51-L81), [ref2](https://github.com/mlcommons/GaNDLF/blob/new-apis_v0.1.0-dev/GANDLF/metrics/classification.py#L94-L97)].
Cheers,
Sarthak
@vchung Thanks for the update!
I have a question regarding the validation metrics. From my submission, I got 3 metrics with the same small value (14%). I guess this is unlikely but still possible. However, what is a bit contradictory is the high specificity (85%+). Could you maybe look into how the metrics are calculated or give me some insights on it?
Thanks in advance! @luiscarm9 ,
Thank you for your patience! The submission queue for the Pathology task is now available starting today, July 15th. Please submit your CSV predictions to the queue: **BraTS 2024: Pathology**
As a reminder, you are allowed up to 2 scored submissions a day (invalid submissions do not count towards your quota), where the daily submission limit will reset every day at 00:00:00 UTC.