I have made submission for Digital Mammography Prediction - Sub-Challenge 1 Express Lane using [this file](https://www.synapse.org/#!Synapse:syn7773987) but haven't received any feedback (email, status table .etc) so far (6+ hours by now). How do I know that my scoring works well? Will there be any submission status table for inference (express/regular) like the one we have for training? I think it will be very useful if we have such thing for every submission.

Created by Yohanes Gultom yohanesgultom
@brucehoff thanks for your assistance! Yes, it ran properly but hit an error. I have fixed it and submitted another one. I hope this one will be successful.
Update: The submission has now run. This looks like the correct log file (showing an error): ``` Using Theano backend. Using gpu device 0: Tesla K80 (CNMeM is enabled with initial size: 80.0% of memory, cuDNN 4008) Loading model Predicting image by image Traceback (most recent call last): File "sc1_infer.py", line 35, in data = preprocess_image(os.path.join(dcm_dir, dcm_filename), EXPECTED_DIM[1], MAX_VALUE, FILTER_THRESHOLD) TypeError: preprocess_image() takes exactly 2 arguments (4 given) Command exited with non-zero status 1 20.32user 3.10system 0:23.49elapsed 99%CPU (0avgtext+0avgdata 1472748maxresident)k 642232inputs+148088outputs (215major+889936minor)pagefaults 0swaps ```
Update: Submission 7798509 is now restarted and scoring is in process. You should receive an email notification as to the outcome. Please keep in mind that the numerical values of scores received from the 'express lane' are not really meaningful. The purpose of the express lane is simply to provide a quick check that your code is error free.
@yohanesgultom: Yes, you created submission 7798509. You should receive an email notification regarding its status. In the near future we hope to add a status table on our web site as well. Further, there was a small infrastructure problem that made your submission fail, now fixed. Please disregard the initial error message (if any).
I finally got an email notification saying that my inference/scoring submission failed ``` STDERR: File "sc1_infer.py", line 26, in STDERR: with open(crosswalk_file, 'rb') as tsvin: STDERR: IOError: [Errno 21] Is a directory: '/metadata/images_crosswalk.tsv' ``` Apparently I got this IO error when trying to read `images_crosswalk.tsv`. What did I do wrong?
@brucehoff @thomas.yu I just submitted a new one and got it completed. Thanks! I also immediately made another express-lane inference/scoring submission. But I haven't got any notification nor haven't seen the entry in the dashboard. How do I track my express-lane inference/scoring submission?
@yohanesgultom: Below is the list of 'express lane' submissions from you. It looks like all have been processed and the last five completed successfully. If you feel there is an outstanding problem would you please reply, including the Submission ID of the problematic submission? Thank you. ${leaderboard?path=%2Fevaluation%2Fsubmission%2Fquery%3Fquery%3Dselect%2B%2A%2Bfrom%2Bevaluation%5F7500018%2Bwhere%2BUSER%5FID%253D%253D%25223346962%2522&paging=true&queryTableResults=true&showIfLoggedInOnly=false&pageSize=100&showRowNumber=false&jsonResultsKeyName=rows&columnConfig0=none%2CSubmission ID%2CobjectId%3B%2CNONE&columnConfig1=none%2CStatus%2Cstatus%3B%2CNONE&columnConfig2=none%2CStatus Detail%2CSTATUS%5FDESCRIPTION%3B%2CNONE&columnConfig3=cancelcontrol%2CCancel%2CcancelControl%3B%2CNONE&columnConfig4=epochdate%2CLast Updated%2CmodifiedOn%3B%2CNONE&columnConfig5=synapseid%2CSubmitted Repository or File%2CentityId%3B%2CNONE&columnConfig6=none%2CFile Version%2CversionNumber%3B%2CNONE&columnConfig7=synapseid%2CLog Folder%2CSUBMISSION%5FFOLDER%3B%2CNONE&columnConfig8=none%2CSubmitting User or Team%2CSUBMITTER%3B%2CNONE&columnConfig9=synapseid%2CModel State File%2CMODEL%5FSTATE%5FENTITY%5FID%3B%2CNONE}
Dear Yohanes, Apologies for the delay in response. I see that you have made two submissions and they are both CLOSED due to infrastructure error. We hope to get this fixed as soon as possible. Thanks, Thomas
I think the express lane is having issue again. My submission hasn't been running since +7 hours ago. Maybe the server has been overwhelmed by the number of submissions? Anyway I wish the best luck for the team :)
@josecp thanks! I really missed that
If you notice carefully in the dictionary.xls file there is one less column for the crosswalk file during sub-challenge 1 testing; the examIndex column is not present. Hence row[5] doesn't exist.
@thomas.yu thanks, Thomas. I noticed that my submission has been executed. But it was failed with this error: ``` STDERR: File "sc1_infer.py", line 34, in STDERR: dcm_filename = row[5] STDERR: IndexError: list index out of range ``` Basically, it couldn't find column 5 in the crosswalk file which supposed to contain DCM filename. Is the crosswalk file for inference/scoring different with the one provided during training?
Dear Yohanes, Thank you for your patience. We are currently looking into this error. Best, Thomas

No feedback for Digital Mammography Prediction - Sub-Challenge 1 Express Lane page is loading…