Hello, we are having the an issue/error in the OS subchallenge where our main script runs properly (we can see print statements in the logs) and predictions.csv is written to /output/ on our local machine. We have verified the correct spelling too. We previously had no issues submitting our first try for the OS subchallenge or any other attempts for other subchallenges. We can't figure out why this isn't working, is there any additional info on the backend? Thanks.
Created by Andrea Castro andreabc @andreabc , @adamklie --
Thank you for waiting! After doing some digging, it looks like your models run successfully against the synthetic data (hence explaining why it works locally on your end and the email you receive about a valid submission), but they unfortunately do not run successfully against the real data. As a result, no predictions file is generated, which would explain the error you are receiving in the TOIL logs. @adamklie and @andreabc ,
Thank you. We will look into the issue now and try to respond soon! These are our most recent tries: 9710790, 9710787, 9710785. Thank you If it helps, this is part of the error we are seeing in the log file
```
STDERR: 2021-02-22T20:43:28.236076090Z Exception: No 'predictions.csv' file written to /output, please check inference docker
STDERR: 2021-02-22T20:43:28.236079962Z [job run_docker.cwl] Max memory used: 57MiB
STDERR: 2021-02-22T20:43:28.236083856Z INFO:cwltool:[job run_docker.cwl] Max memory used: 57MiB
STDERR: 2021-02-22T20:43:28.236087758Z [job run_docker.cwl] Job error:
STDERR: 2021-02-22T20:43:28.236091543Z ("Error collecting output for parameter 'predictions':\nAnti-PD1-DREAM-Infrastructure-master/run_docker.cwl:67:7: Did not find output file with glob pattern: '['predictions.csv']'", {})
```
@andreabc ,
Thank you for the update. Let me see what is happening on the backend of the submission system in that case. To help with debugging, can you let me know what your submission IDs are?
Thank you! Hi @vchung thanks for your help. I just tried running with a 6GB and 1CPU limit and we still get /output/predictions.csv written as expected Hi @vchung thanks for your response, I just tried setting a 6GB memory limit, the predictions file still writes without issue, I also limited the number of CPUs to 1 and the container still writes as expected Hi @andreabc ,
Apologies that your team is experiencing issues with the second sub-challenge! When you run your container locally, do you also include a 6G memory limit? This will emulate the same environment that we are using when we run your submission.
Let us know!
Drop files to upload
No predictions.csv file written for OS challenge page is loading…