Dear MIDI-B challengers, We recently found an issue with the measurement metric on the leaderboard. The current algorithm does not check for missing files, which could result in a high score even when files are missing. We have fixed this issue in the latest version. However, the update might not be immediately reflected in the validation phase. In the upcoming test phase, the measurement metric will be based on the latest version. We apologize for the mistake and any inconvenience this may have caused. MIDI-B Challenge Organizing Committee

Created by Linmin Pei phmoer
Thank you for reporting this discrepancy. I believe you mean (text_retained) rather than (tag_retained). I do find that the answer key is inconsistent for this particular element, however, the text_retained errors are consistent with how TCIA handles the tag using the Clean Desc. Option in the confidentiality profiles. https://dicom.nema.org/medical/dicom/2014b/output/chtml/part15/chapter_E.html#table_E.1-1 Since we aren't seeing conflicting answer key entries (both text_retained and text_removed for the single element), and the entry, when present, is consistent with TCIA practices, corrections will not be made at this time. Good luck! Michael
Hi, If possible could you let me know if any change was made to test regarding the "Writer" occupation so I know what version to submit for the test tomorrow. Thank you!
I also found that the validation script requires the occupation to be removed only if the occupation is "Writer". Hopefully that can be fixed for the test phase. Thanks! Edit: requires the occupation to be kept (tag_retained) only if the occupation is "Writer"

! Measurement Metric Issue in the Leaderboard ! page is loading…