Hi,
I am seeking guidance on how to download the answer key data mentioned in the user manual of MIDI validation.
Thank you.
Created by Hongzhu Jiang jhzshan Ok understood - thanks for getting back to me quickly. Hi @admicr,
Unfortunately, we are not currently providing the Answerkey of the public training data, and have no plan to share it during the challenge. You have said earlier that the answer keys (I think sqlite DBs....) are not available - but could we have access to the key file db just for the public training data (ie the Rutherford paper) to run the validation script against? As mentioned by @sdvoretskii running things locally would likely help with internal testing,,,
Many thanks! The answer key is different from other conventional challenges. It contains the information that needs to be changed and its values. I'd like to recommend you review the [Evaluation Section](https://www.synapse.org/Synapse:syn53065760/wiki/627888), and the pseudo data in the [IDC](https://portal.imaging.datacommons.cancer.gov/explore/filters/?collection_id=MIDI&collection_id=pseudo_phi_dicom_data) and the [TCIA](https://www.cancerimagingarchive.net/collection/pseudo-phi-dicom-data/) (before and after de-identification). Is there at least any way to know the structure of the answer key? It would be useful to set up the validation script locally, for the own data evaluation. The answer key is used to evaluate the submission's performance. It is generated by the Challenge Organizer, and you can consider the answer key as a "ground truth". Does the answer key need to be provided by the contestant or is it automatically generated by the code of validation script? The Answer key is not available to the public during the MIDI-B challenge.
-Linmin