My submission 9725576, the docker worked on my machine and fastlane.
Would like to know if there are any difference in the TASK lane for submission.
Thankou in advanced!
Created by RPS PS maximus @maximus ,
Correct. [Here](https://www.synapse.org/#!Synapse:syn26133770/discussion/threadId=9522) is discussion post related to the `project` column in the input files.
Additionally, a similar question was also asked during [Office Hours](syn35034737) at the 21:05 time mark:
> **On the training data, we have projects "A" to "J". Is the validation data from the same projects or totally new ones or partially new projects?**
> _Totally new projects. [...] If you have a model that does do some grouping by project, they will be assigned a letter not "A" to "J"._
I see.
The test ( unseen) data, has columns "projects" names beyond letter "J", like "K", "V", etc? My models learned those letters.
If they are different, it will probably results in a error.
I suppose this can be the case. @maximus ,
Looks like submission ID 9725629 is encountering an error similar to [your submissions to Task 1](https://www.synapse.org/#!Synapse:syn26133770/discussion/threadId=9601), that is:
```
Traceback (most recent call last):
File "/usr/local/bin/run_model2.py", line 210, in
main()
File "/usr/local/bin/run_model2.py", line 193, in main
results = modelx(args.input_dir)
File "/usr/local/bin/run_model2.py", line 159, in modelx
predicts = model.predict(df)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node 'model/string_lookup_2/Assert/Assert' defined at (most recent call last):
File "/usr/local/bin/run_model2.py", line 210, in
main()
File "/usr/local/bin/run_model2.py", line 193, in main
results = modelx(args.input_dir)
File "/usr/local/bin/run_model2.py", line 159, in modelx
predicts = model.predict(df)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 2033, in predict
tmp_batch_outputs = self.predict_function(iterator)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1845, in predict_function
return step_function(self, iterator)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1834, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1823, in run_step
outputs = model.predict_step(data)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1791, in predict_step
return self(x, training=False)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 490, in __call__
return super().__call__(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/functional.py", line 458, in call
return self._run_internal_graph(
File "/usr/local/lib/python3.8/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/layers/preprocessing/index_lookup.py", line 628, in call
lookups = self._lookup_dense(inputs)
File "/usr/local/lib/python3.8/dist-packages/keras/layers/preprocessing/index_lookup.py", line 676, in _lookup_dense
assertion = tf.Assert(tf.equal(tf.size(oov_indices), 0), [msg])
Node: 'model/string_lookup_2/Assert/Assert'
Detected at node 'model/string_lookup_2/Assert/Assert' defined at (most recent call last):
File "/usr/local/bin/run_model2.py", line 210, in
main()
File "/usr/local/bin/run_model2.py", line 193, in main
results = modelx(args.input_dir)
File "/usr/local/bin/run_model2.py", line 159, in modelx
predicts = model.predict(df)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 2033, in predict
tmp_batch_outputs = self.predict_function(iterator)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1845, in predict_function
return step_function(self, iterator)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1834, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1823, in run_step
outputs = model.predict_step(data)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1791, in predict_step
return self(x, training=False)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 490, in __call__
return super().__call__(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/functional.py", line 458, in call
return self._run_internal_graph(
File "/usr/local/lib/python3.8/dist-packages/keras/engine/functional.py", line 596, in _run_internal_graph
outputs = node.layer(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/engine/base_layer.py", line 1014, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 92, in error_handler
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/keras/layers/preprocessing/index_lookup.py", line 628, in call
lookups = self._lookup_dense(inputs)
File "/usr/local/lib/python3.8/dist-packages/keras/layers/preprocessing/index_lookup.py", line 676, in _lookup_dense
assertion = tf.Assert(tf.equal(tf.size(oov_indices), 0), [msg])
Node: 'model/string_lookup_2/Assert/Assert'
2 root error(s) found.
(0) INVALID_ARGUMENT: assertion failed: [When `num_oov_indices=0` all inputs should be in vocabulary, found OOV values [\"K\" \"V\" \"K\" ... \"K\" \"K\" \"K\"], consider setting `num_oov_indices=1`.]
[[{{node model/string_lookup_2/Assert/Assert}}]]
[[model/string_lookup/Where/_3744]]
(1) INVALID_ARGUMENT: assertion failed: [When `num_oov_indices=0` all inputs should be in vocabulary, found OOV values [\"K\" \"V\" \"K\" ... \"K\" \"K\" \"K\"], consider setting `num_oov_indices=1`.]
[[{{node model/string_lookup_2/Assert/Assert}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_predict_function_21262]
```
Hope this helps! I see.
Indicated the wrong file to you to verify.
but this is correct
9725629( task 2 - invalid)
9725630 ( fast lane ok)
Could you please, help me to find this? Thankyou!
Hi @maximus ,
For submission ID 9725576, the following error was received:
```python
python: can't open file '/usr/local/bin/run_model.py': [Errno 2] No such file or directory
```
"Peeking" at your Dockerfile, it looks like the script has been renamed to `run_model2.py`?
```bash
$ docker history docker.synapse.org/syn35955922/model@sha256:8ab11b16...
IMAGE CREATED CREATED BY SIZE COMMENT
9acec8e4f6b9 14 hours ago ENTRYPOINT ["python" "/usr/local/bin/run_mod? 0B buildkit.dockerfile.v0
14 hours ago COPY model.h5 /usr/local/bin/. # buildkit 5.98MB buildkit.dockerfile.v0
14 hours ago COPY run_model2.py /usr/local/bin/. # buildk? 7.63kB buildkit.dockerfile.v0
...
```
I checked your submissions between the Fast Lane queue and the Task 2 queue, and I think a different version may have been submitted:
**Queue** | **Submission Digest**
--|--
Fast Lane | `docker.synapse.org/syn35955922/model@sha256:bca8f87f...`
Task 2 | `docker.synapse.org/syn35955922/model@sha256:8ab11b16...`
Hope this helps!