Despite of having validation error (as described in [my previous post](https://www.synapse.org/#!Synapse:syn4224222/discussion/threadId=1266)), I got email notification providing scoring result (apparently for express lane but there is no way I can tell as I can't track the submission id) ``` Your submission "submission_scoring.txt" (ID: 7799685) to the Digital Mammography Prediction - Sub-Challenge 1 has been scored: Your submission was missing predictions for 0.000000% of the samples, the confidence values of these samples will be filled in with the median. Your scores are: auc: 0.551637057858 partial AUC: 0.0238748040368 Specificity At Sensitivity 0.8: 0.278982951205. ``` But I also realized there are 2 error notification emails before it (same submission ID). Could anyone help me to understand the meaning? Was my scoring submission successful or failed? If it's failed how can I fix it as the logs showed that the errors weren't coming from my code. 1st email ``` Sorry, but we were unable to process your submission to the Digital Mammography Prediction - Sub-Challenge 1. Please refer to the challenge instructions which can be found at https://www.synapse.org/#!Synapse:syn4224222/wiki/ and to the error message below: submission name: submission_scoring.txt submission ID: 7799685 Traceback (most recent call last): File "challenge.py", line 222, in score score, message = conf.score_submission(syn, evaluation, submission) File "/home/ubuntu/DM-Challenge-Scoring/challenge_config.py", line 227, in score_submission results = scoring_func(evaluation,outtxt,goldstandard) File "/home/ubuntu/DM-Challenge-Scoring/challenge_config.py", line 108, in score gold['cancer'] = gold['confidence'] File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1969, in __getitem__ return self._getitem_column(key) File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1976, in _getitem_column return self._get_item_cache(key) File "/usr/lib/python2.7/dist-packages/pandas/core/generic.py", line 1091, in _get_item_cache values = self._data.get(item) File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3211, in get loc = self.items.get_loc(item) File "/usr/lib/python2.7/dist-packages/pandas/core/index.py", line 1759, in get_loc return self._engine.get_loc(key) File "pandas/index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:3979) File "pandas/index.pyx", line 157, in pandas.index.IndexEngine.get_loc (pandas/index.c:3843) File "pandas/hashtable.pyx", line 668, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12265) File "pandas/hashtable.pyx", line 676, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12216) KeyError: 'confidence' ``` 2nd email ``` Sorry, but we were unable to process your submission to the Digital Mammography Prediction - Sub-Challenge 1. Please refer to the challenge instructions which can be found at https://www.synapse.org/#!Synapse:syn4224222/wiki/ and to the error message below: submission name: submission_scoring.txt submission ID: 7799685 Traceback (most recent call last): File "challenge.py", line 222, in score score, message = conf.score_submission(syn, evaluation, submission) File "/home/ubuntu/DM-Challenge-Scoring/challenge_config.py", line 228, in score_submission results = scoring_func(evaluation,outtxt,goldstandard) File "/home/ubuntu/DM-Challenge-Scoring/challenge_config.py", line 114, in score combined = combined[combined['cancer'] != "."] File "/usr/lib/python2.7/dist-packages/pandas/core/ops.py", line 726, in wrapper res = na_op(values, other) File "/usr/lib/python2.7/dist-packages/pandas/core/ops.py", line 682, in na_op raise TypeError("invalid type comparison") TypeError: invalid type comparison ``` Thank you