This thread is the designated place to ask questions about running the **pcawg-delly-sv-caller** workflow.
Created by James Eddy jaeddy Sure. Thanks @bogdang! Sorry, did not have time to debug the checker tool as well. You can use cwltool for running the checker tool for now. Hi @bogdang,
Not a problem.
I have tested with your docker image and this works fine. Thank you for the link. Do you have an image for the checker tool as well?
Regards,
Vipin Hi @vipin ,
I am out of office this week, so I hope to submit a report some time next week.
In the meantime, you can try to replace the docker image in the pcawg-delly-sv-caller.cwl tool with this one: images.sbgenomics.com/bogdang/pcawg-delly:1
Setting this docker image instead of the original one (quay.io/pancancer/pcawg_delly_workflow:feature_gosu_and_icgc_portal) should help in running it with rabix and on the CGC.
Hope this will help,
Bogdan Hi @Junjun,
I was trying to run `pcawg-delly-sv-caller` workflow using `rabix` as the runner. But I am encountering the following error message:
```shell
~/tools/rabix-cli-dev/rabix-cli-1.0.1/rabix --tmpdir-prefix /home/wfrunner/tmp/ --tmp-outdir-prefix /home/wfrunner/tmp/ pcawg-delly-sv-caller.cwl pcawg-delly-sv-caller.cwl.json
[2017-12-11 15:45:43.149] [INFO] Job root has started
[2017-12-11 15:46:59.071] [INFO] Pulling docker image quay.io/pancancer/pcawg_delly_workflow:feature_gosu_and_icgc_portal
[2017-12-11 15:46:59.934] [INFO] Running command line: /start.sh perl /usr/bin/run_seqware_workflow.pl --run-id run_id --normal-bam /home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_rabix/./input/HCC1143_BL.bam --tumor-bam /home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_rabix/./input/HCC1143.bam --reference-gz /home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_rabix/./input/genome.fa.gz --reference-gc /home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_rabix/./input/hs37d5_1000GP.gc
[2017-12-11 15:47:04.750] [ERROR] Job 0db18949-15e5-4a66-8d77-2dca1718bf30 failed with exit code 1.
[2017-12-11 15:47:04.751] [INFO] Job 0db18949-15e5-4a66-8d77-2dca1718bf30 failed with exit code 1.
[2017-12-11 15:47:04.796] [WARN] Root job 0db18949-15e5-4a66-8d77-2dca1718bf30 failed.
[2017-12-11 15:47:04.887] [INFO] Failed to execute a Job
```
```YAML
docker_version: 1.12.6
runner_version: rabix 1.0.1
```
Please let me know if you want to get more details from my side to look at the issue.
@bogdang: I am seeing from the above message that you were able to execute this workflow but I am not seeing the documentation, Can you please provide me if you have a reference.
Thank you,
Vipin Hi @Junjun,
I am trying to run your workflow with `cwltoil` and it fails at the result validation step. I can provide you the error message which I am getting:
```shell
cwltoil pcawg-delly-sv-caller_checker.cwl pcawg-delly-sv-caller_checker.cwl.json
wfexecutor 2017-12-11 15:03:23,066 MainThread INFO toil.lib.bioio: Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
wfexecutor 2017-12-11 15:03:23,067 MainThread INFO toil.lib.bioio: Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
wfexecutor 2017-12-11 15:03:23,070 MainThread INFO toil.jobStores.abstractJobStore: The workflow ID is: '57fb47ac-6006-4b79-8068-2f3039d9e104'
Resolved 'pcawg-delly-sv-caller_checker.cwl' to 'file:///home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_toil/pcawg-delly-sv-caller_checker.cwl'
wfexecutor 2017-12-11 15:03:23,072 MainThread INFO cwltool: Resolved 'pcawg-delly-sv-caller_checker.cwl' to 'file:///home/wfrunner/dream_challenge/2017/workflow_execution/workflows/pcawg-delly-sv-caller_toil/pcawg-delly-sv-caller_checker.cwl'
Traceback (most recent call last):
File "/home/wfrunner/tools/venv/bin/cwltoil", line 11, in
sys.exit(main())
File "/home/wfrunner/tools/venv/lib/python2.7/site-packages/toil/cwl/cwltoil.py", line 873, in main
strict=useStrict)
File "/home/wfrunner/tools/venv/lib/python2.7/site-packages/cwltool/load_tool.py", line 285, in load_tool
strict=strict, fetcher_constructor=fetcher_constructor)
File "/home/wfrunner/tools/venv/lib/python2.7/site-packages/cwltool/load_tool.py", line 216, in validate_document
schema.validate_doc(avsc_names, processobj, document_loader, strict)
File "/home/wfrunner/tools/venv/lib/python2.7/site-packages/schema_salad/schema.py", line 338, in validate_doc
strip_dup_lineno(bullets(anyerrors, "* ")))
schema_salad.validate.ValidationException: pcawg-delly-sv-caller_checker.cwl:1:1: Object `pcawg-delly-sv-caller_checker.cwl` is not valid
because
tried `CommandLineTool` but
pcawg-delly-sv-caller_checker.cwl:9:1: the `inputs` field is not valid because
pcawg-delly-sv-caller_checker.cwl:15:3: item is invalid because
pcawg-delly-sv-caller_checker.cwl:16:5: the `type` field is not valid because
- tried CommandInputRecordSchema but
pcawg-delly-sv-caller_checker.cwl:17:7: * the `type` field is not valid
because
the value 'array' is not a valid
Record_symbol, expected 'record'
pcawg-delly-sv-caller_checker.cwl:18:7: * invalid field `items`,
expected one of: 'fields', 'type', 'label'
pcawg-delly-sv-caller_checker.cwl:19:7: * invalid field `default`,
expected one of: 'fields', 'type', 'label'
pcawg-delly-sv-caller_checker.cwl:16:5: - tried CommandInputEnumSchema but
* missing required field `symbols`
pcawg-delly-sv-caller_checker.cwl:17:7: * the `type` field is not valid
because
the value 'array' is not a valid
Enum_symbol, expected 'enum'
pcawg-delly-sv-caller_checker.cwl:18:7: * invalid field `items`,
expected one of: 'symbols', 'type', 'label',
'inputBinding'
pcawg-delly-sv-caller_checker.cwl:19:7: * invalid field `default`,
expected one of: 'symbols', 'type', 'label',
'inputBinding'
pcawg-delly-sv-caller_checker.cwl:16:5: - tried CommandInputArraySchema but
pcawg-delly-sv-caller_checker.cwl:19:7: invalid field `default`,
expected one of: 'items', 'type', 'label',
'inputBinding'
```
Here is information about my environment:
```YAML
Environment: RHEL
cwltoil --version: 3.11.0
```
Please let me know if you need any additional information from my side to test.
Thanks!
Vipin Hi @Junjun
One more update, the workflow completed successfully on [CGC](https://cgc.sbgenomics.com/) as well with this update to the file in docker. Hi @Junjun
I tested the `pcawg-delly-sv-caller` with `rabix` (version 1.0.1) and encountered a minor issue related to differences in environments when running docker with cwltool and rabix. These differences are related to environment variables (HOME and TMPDIR) and directories /var/spool/cwl (which does not exist in rabix execution) and /tmp. These directories are cwltool specific and not defined by the CWL specification.
To work around this difference I added a couple of lines to `/start.sh` file in the docker.
```
#!/usr/bin/env bash
set -o errexit
set -o pipefail
set -x
mkdir -p /var/spool/cwl
export HOME=/var/spool/cwl
export TMPDIR=/tmp
gosu root chmod a+wrx /tmp
gosu root chmod a+wrx /var/spool/cwl
env
gosu seqware bash -c "$*"
#allow cwltool to pick up the results created by seqware
gosu root chmod -R a+wrx /var/spool/cwl
```
Maybe not all commands really need to be added for the workflow to run with rabix (mkdir -p /var/spool/cwl is definitely required as chmod would fail since it did not exist)
Does this seem like something that can be added to the `/start.sh` file in the docker image, as I think it would help make the workflow a bit more "executor agnostic"?
Drop files to upload
pcawg-delly-sv-caller workflow help page is loading…