Hello
We have a strange problem that creates a core dumped when our inference program finishes.
However we are always able to correctly create the predictions.tsv under the output directory. In the
round 2 this was not an issue and we got scored.
Our submission ID 8474696 has finished and the last lines of the log say:
STDOUT: 2017-03-21T02:03:08.020982534Z Evaluation finished ok!
STDERR: 2017-03-21T02:03:08.041563696Z terminate called without an active exception
STDERR: 2017-03-21T02:03:11.277030192Z /sc2_infer.sh: line 21: 7 Aborted (core dumped) python /root/bin/evaluation_merge.py
This means that we were able to create the output but crashed afterwards when releasing the pool of threads that we use to speed up the algorithm
Can you check that the output is there and we get scored?
Thanks
Created by Alberto Albiol alalbiol 8495520 has been terminated. Thank you for freeing up resources for others. Hello again
Since we finally got submission 8474693 scored, please kill submission 8495520 this submission has a bug
and will crash at the end using precious CPU time for others
Thanks Although our system should have deleted the predictions file, by chance it did not do so. I captured the file, attached it to your submission, and it was scored.
> the system seems to detect these core dumped situations and makes the submission invalid automatically without checking if a valid predictions file was generated.
I think that is the behavior we want. It's best not to trust output when the code encountered an error. Thanks Tom
At this point I assume that you can't kill submission 8495520, it is fine it will die
anyway :-)
My only request is that if it is possible to score submission 8474693, as explained
in the beginning of this thread it completed the inference successfully (generated the predictions.tsv according
to the last few lines of the logs ) but generated a core dumped at the very end when it was releasing the resources.
In the second round this was Ok, but in this round the system seems to detect these
core dumped situations and makes the submission invalid automatically without checking if a valid
predictions file was generated.
Thanks again for your patience
Dear Alberto,
Ok. I made your submission: 8504081 valid again.
I have forwarded your request of cancelling submissions to other challenge organizers.
Best,
Tom Tom,
I can't kill submission 8495520 from the link that you have sent me. I can only cancel
submission which have not yet started. And 8495520 is running
Also I probably did not explain myself correctly submission 8504081 which you made invalid is Ok.
Can you make it valid again?
Thanks
Dear Alberto,
Apologies for the delay in response. Please go to this [page](https://www.synapse.org/#!Synapse:syn4224222/wiki/409764) to kill your submission: 8495520
I have made your 8504081 invalid so you an go ahead and submit another submission.
Best,
Tom Hello again and apoligies for all this mess
Since the deadline is approaching and we did not get any response we have sent
a new submision (id 8474696) with all the fixes that replaces de crashed submision (ID 8474696) (although we
created the predictions.tsv file)
We have the same problem with submision 8495520 which is currently running. So please KILL this submision because
the newer submision 8504081 has all the fixes.
Finally, submission 8474693, has also crashed but has created the predictions.tsv. If you kill the sub 8504081, we can
launch a new submision with all the fixes (before the deadline). If not please take the predicions.tsv file in the /output folder of submision 8474693
so we can get scored.
Again apologies for all this mess and would like to ask for your comprehension, there are so many little things to consider,
short time, and very difficult to debug
Thanks We finally found and solved the problem that caused the crash when exiting the program after writing
the prediction.tsv file
If you can confirm us that the submision will be evaluated with the output that we have generated is Ok to us. If
not please let us know before the deadline so we can resubmit with the patches
Thanks