Hi,
Do you plan to release the images? (Or at least the ones with cancer, or the ones had been called back for further evaluation. thats a much smaller scale.)
Sorry if it has been asked before, but the discussion website makes it hard to see old threads.
Thanks
Dezso
Created by Dezso Ribli riblidezso > I was wondering if the data providers plan to release the dataset. Do you know anything about that?
We have negociated the access to the data before the start of the challenge. At the best of my knowledge, the position of the data provider has not changed regarding a public release of their data.
> if the data can not released, do you plan to set up some evaluation server? Even if people can not benefit from the data as training data, it would also extremely useful as benchmark dataset.
I agree with you. This would be great. However, you understand that this would have a cost in terms or infrastructure and persons. There are also risk that the dataset leak over a long period of time if users are allowed to retrieve information. We will however release the infrastructure that we have developed for this challenge with the hope that this will help third parties to organize challenges with data that would otherwise not be visible.
Thanks,
Thomas
Thanks @tschaffter , I know you can not release the images at this point, but I was wondering if the data providers plan to release the dataset. Do you know anything about that?
I think it would be a great contribution to the field. You can see how that most top teams used DDSM data, which is 2 decades old and they are scanned films... New data would be very useful for the community.
Another important point is that altough the top teams released their source code, It is impossible for anyone to reproduce them, or make some changes, because they don't have data to train on. ( My solution is an exception because thats not trained on DREAM data, but thats not the winning solution. ) Maybe large players with their own large proprietary dataset can use the solutions, but I am not sure if the community would benefit from that.
Another question, if the data can not released, do you plan to set up some evaluation server? Even if people can not benefit from the data as training data, it would also extremely useful as benchmark dataset. I think many mammography prediction software will come in the future, and a fair benchmark would be very important too.
Thanks
Dezso > Do you plan to release the images?
No, our data provider does not allow us to release the images. That's why we had to develop the docker based infrastructure used during the challenge. Thanks to our approach, we were able to shed light on the existence of this dataset, and maybe at some point the dataset will be released by the data provider.
> Sorry if it has been asked before, but the discussion website makes it hard to see old threads.
I agree. I've given to Synapse a few suggestions to improve the discussion forum such as adding a search tool and other features existing in widely used issues trackers (e.g. GitHub).