Hi there, We are working on evaluating the validation dataset, our single image shape is [240, 240, 155], however we encountered Memory Error after we submitted validation files. Please find the detailed error messages attached below. Thanks for helping. NGresearch. STDERR: 2021-08-03T09:07:07.088120326Z [22] Failed to execute script Hausdorff95 STDERR: 2021-08-03T09:07:07.088122539Z Traceback (most recent call last): STDERR: 2021-08-03T09:07:07.088124663Z File "Hausdorff95.py", line 34, in STDERR: 2021-08-03T09:07:07.088127138Z File "site-packages/medpy/metric/binary.py", line 396, in hd95 STDERR: 2021-08-03T09:07:07.088129479Z File "site-packages/medpy/metric/binary.py", line 1224, in __surface_distances STDERR: 2021-08-03T09:07:07.088131803Z File "site-packages/scipy/ndimage/morphology.py", line 2202, in distance_transform_edt STDERR: 2021-08-03T09:07:07.088134086Z MemoryError: Unable to allocate 204. MiB for an array with shape (3, 155, 240, 240) and data type float64 STDERR: 2021-08-03T09:07:07.088136376Z terminate called after throwing an instance of 'std::out_of_range' STDERR: 2021-08-03T09:07:07.088138683Z what(): basic_string::erase: __pos (which is 18446744073709551615) > this->size() (which is 0) STDERR: 2021-08-03T09:07:07.088143169Z /usr/local/lib/python3.6/site-packages/pandas/compat/__init__.py:120: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError. STDERR: 2021-08-03T09:07:07.088145781Z warnings.warn(msg) STDERR: 2021-08-03T09:07:07.088147907Z Traceback (most recent call last): STDERR: 2021-08-03T09:07:07.088150013Z File "/usr/local/bin/score.py", line 141, in STDERR: 2021-08-03T09:07:07.088152304Z main() STDERR: 2021-08-03T09:07:07.088154404Z File "/usr/local/bin/score.py", line 115, in main STDERR: 2021-08-03T09:07:07.088156600Z results = score(preds, golds, args.captk_path) STDERR: 2021-08-03T09:07:07.088158774Z File "/usr/local/bin/score.py", line 102, in score STDERR: 2021-08-03T09:07:07.088160935Z run_captk(captk_path, pred, gold, tmp_output) STDERR: 2021-08-03T09:07:07.088163045Z File "/usr/local/bin/score.py", line 63, in run_captk STDERR: 2021-08-03T09:07:07.088165258Z subprocess.check_call(cmd) STDERR: 2021-08-03T09:07:07.088167375Z File "/usr/local/lib/python3.6/subprocess.py", line 291, in check_call STDERR: 2021-08-03T09:07:07.088169637Z raise CalledProcessError(retcode, cmd) STDERR: 2021-08-03T09:07:07.088171809Z subprocess.CalledProcessError: Command '['/work/CaPTk/bin/Utilities', '-i', 'RSNA_ASNR_MICCAI_BraTS2021_ValidationGT/BraTS2021_00001_seg.nii.gz', '-lsb', 'BraTS2021_00001.nii.gz', '-o', 'tmp.csv']' died with . STDERR: 2021-08-03T09:07:07.088174472Z [job score.cwl] Max memory used: 954MiB STDERR: 2021-08-03T09:07:07.088176984Z INFO:cwltool:[job score.cwl] Max memory used: 954MiB STDERR: 2021-08-03T09:07:07.088179152Z [job score.cwl] Job error: STDERR: 2021-08-03T09:07:07.088181295Z ("Error collecting output for parameter 'results':\nbrats-dream-challenge-infra-main/score.cwl:46:7: Did not find output file with glob pattern: '['results.json']'", {}) STDERR: 2021-08-03T09:07:07.088183869Z ERROR:cwltool:[job score.cwl] Job error: STDERR: 2021-08-03T09:07:07.088185991Z ("Error collecting output for parameter 'results':\nbrats-dream-challenge-infra-main/score.cwl:46:7: Did not find output file with glob pattern: '['results.json']'", {}) STDERR: 2021-08-03T09:07:07.088188506Z [job score.cwl] completed permanentFail STDERR: 2021-08-03T09:07:07.088190639Z WARNING:cwltool:[job score.cwl] completed permanentFail STDERR: 2021-08-03T09:07:07.088192779Z Traceback (most recent call last): STDERR: 2021-08-03T09:07:07.088194881Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-03T09:07:07.088197171Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-03T09:07:07.088199399Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-03T09:07:07.088201675Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-03T09:07:07.088206294Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-03T09:07:07.088208600Z return self.run(fileStore) STDERR: 2021-08-03T09:07:07.088210772Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-03T09:07:07.088213047Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-03T09:07:07.088215180Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-03T09:07:07.088217341Z ERROR:toil.worker:Exiting the worker because of a failed job on host 3485a45736c9 STDERR: 2021-08-03T09:07:07.088219679Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/cde2a488-3ff3-4215-8c81-9fcbe3327a7f/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_cde2a488-3ff3-4215-8c81-9fcbe3327a7f_brats-dream-challenge-infra-main_score.cwl/instance-4c7kxvzh with ID kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_cde2a488-3ff3-4215-8c81-9fcbe3327a7f_brats-dream-challenge-infra-main_score.cwl/instance-4c7kxvzh to 0

Created by NGresearch
Hi @NGresearch, The issue is sorted out now. I could verify that your submission is evaluated now. Ujjwal.
I get the same error. I upload the files as a single .zip archive. Every *.nii.gz file has a shape (240, 240, 155) and data type uint8.
Yes. I did it the same way. Place all the *nii.gz in a single zip and submitted. Did you have any error (invalid submission) while submitting?
Could I ask about your folder structure? I contained *.nii.gz files inside of a single zip and submitted that zip for evaluation. Is it correct?
I am also having this error. Same error is being output when I tried submitting one of the given seg.nii.gz file.
Same error here, files were saved as int16 but it seems they are loaded as float64?

Submission Evaluation INVALID due to MemoryError ----Failed to execute script Hausdorff95 page is loading…