For Task 1, the data is the same as Brats 2021 right? My question is what is the label format that we need to submit? Is it similar to the segmentation labels provided, or do we provide segmentation labels corresponding to ET, TC, WT in the submission? Thanks.
Created by Verena Chung vchung The Validation phase will launch next week with more details. But to answer your question briefly, evaluation will be the same across all challenges, which will be across the combination of ground truth labels (WT, TC and ET). And performance will be lesionwise (Dice and Hausdorff for each lesion), averaged across lesions for each subject, although different challenges will define individual lesions differently . OK thank you, that leads me onto my next question. Reading through the manuscripts, it seems like different regions are used for evaluation in different challenges.
For Challenges 1, 4 and 5, I read that the regions considered for evaluation are the partially overlapping ones (i.e. combinations of ground truth labels): WT, TC and ET.
However, for Challenges 2 and 3, I read that the regions considered for evaluation are ET, NETC (NCR) and SNFH (ED), i.e. they correspond exactly to the ground truth labels.
Will this be the case or will all Challenges be subject to the same evaluation? This is a subtle detail but it will slightly change how we train our models in each challenge, so please let us know.
Thanks,
Ethan Yes. Thank you for pointing that out. It is a typo. I have notified the Pediatric Challenge organizers.
Jeff OK, so I think at the bottom of page 4 in the manuscript it means to say TC in brackets instead of NC?
"As described earlier and exemplified in Figure 2 (bottom panel), for the BraTS-PEDs 2023 challenge, the
regions to evaluate the performances are: i) the ?enhancing tumor? (ET), ii) the ?enhancing tumor/cystic
component/necrosis? (**NC**) and iii) the entire tumorous region, or the so-called ?whole tumor? (WT)."
Let me know. Thanks,
Ethan NC in Peds challenge equivalent to NCR/NETC, label 1
Jeff @jeffrudie
OK thank you.
I am confused about the label NC in Challenge 5, BraTS-PED.
In the challenge manuscript, it says that NC is "a combination of **nonenhancing** tumor, cystic component, and necrosis" in the middle of page 4, yet at the bottom of the page NC is described as "**enhancing** tumor/cystic
component/necrosis". Which one should it be?
Another way of thinking about this: is NC equivalent to NCR (label 1) or TC (label 1 + label 3)?
Thanks,
Ethan Yes- the labels are the same across challenges. NETC=NCR and SNFH=ED. The names have just be updated (NETC and SNFH are the updated names, previoulsy NCR and ED) to more accurately reflect tumor pathology/imaging characteristics.
Jeff @jeffrudie
Thank you for clarifying that. For some of the other Challenges (2,3,4), I see that there are different names for labels 1 and 2. For all intents and purposes, can we consider NETC to be equivalent to NCR, and NSFH to be equivalent to ED?
Ethan The segmentations you provide should be identical format to the training segmentations provided (1 for NCR, 2 for ED and 3 for ET).
The creation of the combinations of different masks (TC = NCR + ET and WT = NCR + ED + ET) is done automatically by the evaluation scripts. You do not need to generate your own TC/WT masks/labels.
Jeff @vchung @jeffrudie
Hi there,
Our team was also confused about this. I am hoping you can clarify.
In this thread and in [this one](https://www.synapse.org/#!Synapse:syn51156910/discussion/threadId=10243&replyId=30038), you say that the segmentation output should be the same as the provided segmentations i.e. 1 for NCR, 2 for ED and 3 for ET.
However, under the subsection ** Task: Tumor Sub-region Segmentation** on the page for Challenge 1, it states that 'The sub-regions considered for evaluation are the "enhancing tumor" (ET), the "tumor core" (TC), and the "whole tumor" (WT)' and proceeds to describe how these are defined. My understanding of this is the same as @naumanbashir 's above, where ET is as is, TC = NCR + ET and WT = NCR + ED + ET.
I believe both of these approaches to segmentation convey equivalent information, however, please do let us know what approach we should take for the challenge submissions.
Thanks,
Ethan
@vchung
I have a confusion regarding the segmentation labels for the submission. On the homepage, it is mentioned:
"The sub-regions considered for evaluation are the "enhancing tumor" (ET), the "tumor core" (TC), and the "whole tumor" (WT)."
According to the provided information:
1 NCR + 2 ED + 3 ET = **WT**
1 NCR + 3 ET = **TC**
= **ET**
However, based on your previous message, it appears that we need to provide the following segmentation:
1 for NCR (necrotic parts of the tumor)
2 for ED (peritumoral edematous/invaded tissue)
3 for ET (enhancing tumor)
0 for everything else
Thanks! @rohitrango ,
> For Task 1, the data is the same as Brats 2021 right?
Yes, the BraTS-GLI training and validation data are the same at BraTS 2021, **with one major change**: enhancing tumors (ET) are labeled with '3', not '4'.
> My question is what is the label format that we need to submit? Is it similar to the segmentation labels provided, or do we provide segmentation labels corresponding to ET, TC, WT in the submission?
Great question. In your segmentation files, you are expected to provide the following labels:
* 1 for NCR (necrotic parts of the tumor)
* 2 for ED (peritumoral edematous/invaded tissue)
* 3 for ET (enhancing tumor)
* 0 for everything else
Hope this helps!