Hi,
I submitted a Keras - Theano model (https://www.synapse.org/#!Synapse:syn7494638/files/) into both the express and normal lanes. The express lane checked out ok, but I got this in the normal lane.
STDERR: Using Theano backend.
STDERR: Using gpu device 0: Tesla K80 (CNMeM is enabled with initial size: 70.0% of memory, cuDNN 5103)
STDERR: Command terminated by signal 9
STDERR: 42.23user 304.96system 15:38.05elapsed 37%CPU (0avgtext+0avgdata 209316180maxresident)k
STDERR: 288859749inputs+119744outputs (216major+27697220minor)pagefaults 0swaps
Am I correct in assuming that signal 9 means that I'm out of allocated RAM?
It happens when I mount my preprocessed .h5 file, but I know several people who use a similar preprocessing pipeline that isn't getting the signal 9 error.
Created by Muhammad Ihsan Mas dyecoupler @dyecoupler I don't have an immediate answer but will ask around. I've downgraded my Dockers' Theano from 0.9 dev 4 to 0.8.2, the submission passed the express lane for both training (submission ID 7831008) and scoring (ID: 7831146).
However when submitted to the regular training lane, the same error as the post above resulted. The latest submission ID is 7831240. What might be causing this?