Thank you all for participating in this year's BraTS Challenge. This year we are introducing some additional features to our lesion-wise dice score and lesion-wise Hausdorff distance-95 (HD95) performance metrics. Additionally, based on the challenge, teams will be evaluated on new tissues to make it more clinically relevant. Code: https://github.com/rachitsaluja/BraTS-2024-Metrics Here is an outline of how we perform this analysis - 1. First, we isolate the Lesion Tissue Sub-regions. 2. We perform a dilation on the Ground Truth (GT) labels to understand the extent of the lesion. This is mainly done so that when we do a connected component analysis; we don't classify small lesions near an "actual" lesion as a new one. 3. This year, we perform a dilation on the Predicted labels as well and perform the same analysis in the previous step. This is mainly done to avoid penalizing small false-positives that were produced by the team's model where it could be more sensitive. 4. For some of the challenges we also combine lesions by clinically derived rules to generate an accurate number of true lesions for each tissue class. 5. We perform connected component analysis on the Prediction label and compare it component by component to the GT label. 6. We calculate dice scores and HD95 scores for each lesion (or component) individually and we penalize all the False Positives and the False Negatives with a 0 score for dice and the maximum diagonal distance for HD95, we take the mean for the particular CaseID. For the MEN-RT challenge, we penalize only all the False Negatives. 7. All lesions that are below the challenge specific threshold, are not evaluated. This is done to avoid penalizing teams due to human-error in segmentation. Below are the specifics for each of the challenges - ### GLI - Tissue Evaluations - 1. WT (1;2;3) 2. TC (1;3) 3. NETC (1) 4. SNFH (2) 5. ET (3) 6. RC (4) Radiologist Defined Parameters - 1. dilation factor = 3 2. dilation factor = 5 (For SNFH; NETC; RC) 3. lesion volume threshold = 20 mm3 4. lesion volume threshold = 10 mm3 (For ET) Changes - 1. HD-95 penalty = 337 based on MNI atlas 2. Combination of Tissue based on the 4 rules - Combine NETC lesions that are connected by ET or RC Combine SNFH lesions that are connected by NETC or RC or ET Combine ET lesions that are connected by NETC or RC Combine RC lesions that are connected by ET or NETC ### MEN-RT - Tissue Evaluations - 1. GTV (1) Radiologist Defined Parameters - 1. dilation factor = 1 2. lesion volume threshold = 50 mm3 Changes - 1. HD-95 penalty is calculated on the fly for subject native space 2. Remove FP count in the lesion-wise calculations ### MET - Tissue Evaluations - 1. WT (1;2;3) 2. TC (1;3) 3. ET (3) Radiologist Defined Parameters - 1. dilation factor = 1 2. lesion volume threshold = 2 mm3 Changes - 1. Combination of Tissue based on the 1 rule - Combine ET lesions that are connected by NETC ### SSA - Tissue Evaluations - 1. WT (1;2;3) 2. TC (1;3) 3. ET (3) Radiologist Defined Parameters - 1. dilation factor = 3 2. lesion volume threshold = 50 mm3 Changes - 1. Combination of Tissues based on the 1 rule - Combine ET lesions that are connected by NETC ### PED - Tissue Evaluations - 1. WT (1;2;3;4) 2. TC (1;2;3) 3. ET (1) 4. NETC (2) 5. CC (3) 6. ED (4) Radiologist Defined Parameters - 1. dilation factor = 5 2. lesion volume threshold = 10 Please let us know if you have any questions. -- Rachit Saluja, Maria Correia de Verdier, Jeff Rudie