Dear orgs,
Can you prease suggest what we do wrong, to help us not to run out of the submissions. We checked, that output files has proper naming and dimension. That is why it was read and scored :)
We checked that it is 3D and [240, 240, 155], input type is `float64`. The only thing we observe to be changed - it is our confusion with class labes, we have saved it as [1,2,3] instead of [1,2,4]. **But it does not seem to us as leading to DICE == 0.**
**What else we can think of before sending the next package?**
``Your submission (id: 9716291) has been scored and below are the metric averages:
Dice_ET : 0.0
Dice_TC : 0.0
Dice_WT : 0.0
Hausdorff95_ET : 374.0
Hausdorff95_TC : 374.0
Hausdorff95_WT : 374.0
Sensitivity_ET : 0.0
Sensitivity_TC : 0.0
Sensitivity_WT : 0.0
Specificity_ET : 0.0
Specificity_TC : 0.0
Specificity_WT : 0.0
cases_evaluated : 0
```
regards, hellomri team
Created by kondratevakate Thank you, @kondratevakate !
This is to confirm that we will be submitting `docker.synapse.org/syn26242551/hellomri-model2:stable` for hellormi's final submission against the test data. Thank you so much for the possibility and the extra notification!
The docker is: **docker.synapse.org/syn26242551/hellomri-model2**
https://www.synapse.org/#!Synapse:syn26242551/docker/
tag **stable**: sha256:c54429cdc1f95b33e3655359b9f80aada8e574073f1b70be040ede7e41a86dca
if there is something that I've missed?
kind regards, Kate Hi @kondratevakate ,
I wanted to follow-up one more time on whether your team would like to submit a model for final evaluation against the test data. If so, please provide us with the image name and tag, and I will add your model to the final queue.
Thank you! @kondratevakate, @hellomri_2021, and team --
I brought up your request for a late submission with the Challenge Organizers and they are in agreement to accept your model for the final run against the test data. Please provide us with the image name and tag that you would like to be considered as your final submission, and I will add it to the queue for the final test data. Thanks, for your reply and extention. Unfortunately, we were late to read this comment and submit the docker to the challenge. And anyways, it was great challenge, thanks for the organization
Will we have a rebuttal period to submit and evaluate the docker? @hellomri_2021 ,
Apologies, I did not realize your team had made additional submissions since my last message. I have invalidated submissions 9716383 and 9716384 as well. To figure that out we need to rewrite affine, origin and shape.
**And guess what? Now we know, that:**
img = sitk.ReadImage('test_img.nii.gz')
i = sitk.GetArrayFromImage(img)
i.shape
(155, 240, 240)
img = nilearn.image.load_img('test_img.nii.gz')
img.get_data().shape
(240, 240, 155)
Now we know, that resemple with target affine in SiTK will generate redundant class in the array:
`np.unique(i)`
array([0, 1, 2, 3, 4])
then in original image, it was [0, 1, 2, 4].
We know that
`image_new = nilearn.image.resample_img(img, target_affine=target_affine, target_shape=target_shape)`
will give empty array.
This information though, shouldn't cost us that much, if you have provided any descriptive manual. Or point out, that output files should have exactly same affine as Seg. files. And how to save nii images exactly with this affine. Becouse in the previos submissions were not experiencing any problems with shape and origins.
And do not make us guess each and every step: shape, dtype, origin. In the very end we have NO scored submissins, started more than one week before.
And above you write that you will extend our quota and invalidate all our 0 submissions. Well it just appeared have only two queries at all, and onece we figured that all out - unexpectedly we have no submission quota, just realised that. What is the size/dimension of the segmentation/prediction file?
Thanks, we just check - the change on the origin to (0.0, -239.0, 0.0)) doesn't help. Hi @hellomri_2021,
It's not only the origin but also the dimension of the nifti file that should be checked before you submit for the evaluation. @kondratevakate , @hellomri_2021 --
I have invalidated your team's submissions that scored 0; you may submit again up until the deadline tonight at 23:59 EST. Ok, I see, that was nothing about the origin in manual. Is there anything we can do now, if the deadline is in several hours and we run out of quota of submissions? Hi @kondratevakate,
Please make sure your predictions must be of dimension 240x240x155 and origin at 0,-239,0.
It seems your predictions are not of this size and with origin at 0,0,0
Please try to visualize this in CaPTk (https://cbica.github.io/CaPTk/) Dear @kondratevakate and team,
I have passed along your prediction files from submission 9716291 to one of the challenge organizers and they will get back to soon on the details of the scoring issues!