Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ATAC example miss /bamCoverage/*.filtered.seq_depth_norm.bw files #910

Open
HamletShaoE opened this issue Jul 27, 2023 · 5 comments
Open

ATAC example miss /bamCoverage/*.filtered.seq_depth_norm.bw files #910

HamletShaoE opened this issue Jul 27, 2023 · 5 comments
Labels
ATAC-seq test_data

Comments

@HamletShaoE
Copy link

HamletShaoE commented Jul 27, 2023 •
edited

I have installed snakepipes and downloaded ATACseq example for test run. However, the example does not run as I expected.
I first run following the suggestion from command.sh and get this:

(snakePipes) shaoyi@login02 ~/DATA/snakePipe/testData/ATAC $ATAC-seq -d . --sampleSheet sampleSheet.tsv --DAG dm6_ensembl_release94
Sample sheet found and header is ok!

---- This analysis has been done using snakePipes version 2.7.3 ----
Sample sheet found and header is ok!
SystemExit in line 33 of /GPFS/zhangli_lab_permanent/shaoyi/software/mambaforge/envs/snakePipes/lib/python3.11/site-packages/snakePipes/workflows/ATAC-seq/internals.snakefile:
1
File "/GPFS/zhangli_lab_permanent/shaoyi/software/mambaforge/envs/snakePipes/lib/python3.11/site-packages/snakePipes/workflows/ATAC-seq/Snakefile", line 26, in
File "/GPFS/zhangli_lab_permanent/shaoyi/software/mambaforge/envs/snakePipes/lib/python3.11/site-packages/snakePipes/workflows/ATAC-seq/internals.snakefile", line 33, in
File "", line 26, in call
ERROR: Required file "/GPFS/zhangli_lab_permanent/shaoyi/snakePipe/testData/ATAC/bamCoverage/SRR7013050.filtered.seq_depth_norm.bw" for sample "SRR7013050" specified in configuration file is NOT available.
Error: snakemake returned an error code of 1, so processing is incomplete!

I suppose it is because bamCoverage/SRR7013050.filtered.seq_depth_norm.bw is missing.
I notice files in filtered_bam folder are links to Bowtie2. So I run with flag --fromBAM. This runs for a while and slurm goes well, yet end with error in the step of link_bam_bai_external and its' error message is

(snakePipes) shaoyi@login01 ~/DATA/snakePipe/testData/ATAC $ATAC-seq -d . --sampleSheet sampleSheet.tsv --DAG --fromBAM filtered_bam dm6_ensembl_release94
...
[Thu Jul 27 11:25:09 2023]
Error in rule link_bam_bai_external:
jobid: 10
input: EXTERNAL_BAM/SRR7013046.bam, EXTERNAL_BAM/SRR7013046.bam.bai
output: filtered_bam/SRR7013046.filtered.bam, filtered_bam/SRR7013046.filtered.bam.bai
shell:

            ln -s ../EXTERNAL_BAM/SRR7013046.bam filtered_bam/SRR7013046.filtered.bam;
            ln -s ../EXTERNAL_BAM/SRR7013046.bam.bai filtered_bam/SRR7013046.filtered.bam.bai

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
cluster_jobid: Submitted batch job 5652530

Error executing rule link_bam_bai_external on cluster (jobid: 10, external: Submitted batch job 5652530, jobscript: /GPFS/zhangli_lab_permanent/shaoyi/snakePipe/testData/ATAC/.snakemake/tmp.onwy0y99/snakejob.link_bam_bai_external.10.sh). For error details see the cluster log and the log files of the involved rule(s).
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-07-27T110334.970475.snakemake.log

!!! ERROR in the ATAC-seq open chromatin workflow! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Error: snakemake returned an error code of 1, so processing is incomplete!

I notice that EXTERNAL_BAM are links to files in filtered_bam, and this command is trying to link files in filtered_bam back to EXTERNAL_BAM.

I would appreciate if *.filtered.seq_depth_norm.bw files are provided. Or any other suggestions that would help me go through the test.
@NixBio
Copy link

NixBio commented Jul 27, 2023 via email

@katsikora
Copy link
Contributor

katsikora commented Aug 1, 2023 •
edited

Hi,

did your DNA-mapping run prior to running the ATAC-seq workflow in it produce any errors?

Second: when you run ATAC-seq with --fromBAM, did you specify a new output folder, other than that hosting the bam files?

Best wishes,

Katarzyna

@HamletShaoE
Copy link
Author

Hi,

did your DNA-mapping run prior to running the ATAC-seq workflow in it produce any errors?

Second: when you run ATAC-seq with --fromBAM, did you specify a new output folder, other than that hosting the bam files?

Best wishes,

Katarzyna

Hi, thank you for replying.
I download the test data listed in the [snakePipes page](https://snakepipes.readthedocs.io/en/latest/content/setting_up.html) for setting up. The data I downloaded named "ATACseq.tar.gz", whose md5 is 7c822de19958d69bcebe22b1d48885a3. In this file, only bam files contained in Bowtie2 folder and some bamPEFragments contained in deepTools_qc folder. filtered_bam folder contains the soft link to these bam files. In the first run, I just run according to the command.sh asked. There is no fastq files to be worked for DNA-mapping run in this test data.

Also, I tried with -d for the ./(where I extract test data) and --fromBAM for the filtered_bam(where the extracted data with soft links of bam files) from the second run, I did specified the output folder with '-d' which is the upper level directory of the bam file folder.

-d, --working-dir

working directory is output directory and must contain DNA-mapping pipeline output files

Do you mean I should change the -d to another folder? OK, I'll test that.

84 of 84 steps (100%) done
Complete log: .snakemake/log/2023-08-02T105846.019871.snakemake.log

---- This analysis has been done using snakePipes version 2.7.3 ----
Sample sheet found and header is ok!
Building DAG of jobs...

It seems this works fine. thank you.

@katsikora
Copy link
Contributor

Oh I see. I'll have a look if either the test data or the command.sh script on zenodo should be updated to reflect the latest workflow version. Thanks for clarifying that!

@arrowifjn
Copy link

ATAC-seq -d . --sampleSheet sampleSheet.tsv --local --DAG dm6 --fromBAM Bowtie2 --bamExt .bam

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ATAC-seq test_data
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@HamletShaoE @katsikora @NixBio @arrowifjn