Blast Database in unmapped_blast rule

[ezhang113]

Describe the bug
Unable to run unmapped_blast rule of the snakemake file due to issues with blast database.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'eclip-qc' in Emily_working branch
2. Click on 'Snakefile'
3. Scroll down to 'rule unmapped_blastx'
4. See error

Or alternatively in command line:
Nonworking code:
export BLASTDB=/projects/ps-yeolab3/bay001/annotations/nr
blastx -db nr -query LARP6.CTRL_IN1.umi.r1.fq_unmapped_downsampled.fasta -out LARP6.CTRL_IN1.umi.r1.fq_unmappedblast_downsampled_blastx.tsv -outfmt 6 -max_target_seqs 5 -max_hsps 1

Working code: blastx -db pdbaa -query LARP6.CTRL_IN1.umi.r1.fq_unmapped_downsampled.fasta -out LARP6.CTRL_IN1.umi.r1.fq_unmappedblast_downsampled_blastx.tsv -outfmt 6 -max_target_seqs 5 -max_hsps 1

Expected behavior
If the rule was to run as intended, a tsv file should be produced (and subsequent piechart).

Additional context
Seems to be an issue with the database. Have tried running with a ncbi database (pdbaa), which worked out okay.

[byee4]

@ezhang113 could you add the branch you're working under and the example LARP6 downsampled fasta file to that branch?

[ezhang113]

Here is the link to my branch and file: https://github.com/YeoLab/eclip-qc/blob/Emily_working/Snakefile

[byee4]

Can you try using diamond? It's another aligner that is supposed to be similar in performance to BLAST, but run much faster:

module load diamond;
diamond blastx -d /projects/ps-yeolab3/bay001/annotations/nr/nr -q test.fa -o matches.tsv

[ezhang113]

Updates as of 2/28: some issues with adding diamond to the pipeline, same command works on commandline

I attempted the diamond prepdb command as some users suggested in the thread you sent me but it seems that I still get the same error and I think it has to do with how diamond is being used with the BLAST db, but I'm not sure why it won't run here when it runs on the commandline alright.

[byee4]

@ezhang113 can you include the snakemake command? You also need to update the envs so I can see how you're deploying diamond (which version you're using etc) as I don't see any updated spec here: https://github.com/YeoLab/eclip-qc/tree/Emily_working/envs

[ezhang113]

diamond prepdb -d /projects/ps-yeolab3/bay001/annotations/nr/nr
diamond blastx -d /projects/ps-yeolab3/bay001/annotations/nr/nr -q {input} -o {output} -k 5

Sorry I forgot to update the envs for diamond. I updated it just now.

[byee4]

try installing diamond 2.0.4 instead? There might be a bug that requires dropping down a version, since the index appears to work with 2.0.4

[byee4]

actually try this fix instead: bbuchfink/diamond#503

[ezhang113]

I found something similar: https://github.com/bbuchfink/diamond_docs/blob/master/Documentation.MD

that suggests this command to prep fasta databases to run with diamond before the diamond command
$ diamond makedb --in nr.faa -d nr

so I changed my command to:
"""
diamond makedb --in /projects/ps-yeolab3/bay001/annotations/nr/nr -d nr
diamond blastx -d nr -q {input} -o {output} -k 5
"""

but now I get an error: Command must be given as string after the shell keyword. (Snakefile, line 74)
which is so strange because the command is as a string so I'm thinking it's probably a different error.

[byee4]

That is the command I used already to build the index. Not sure why it isn’t working but let me know if dropping down to 2.0.4 solves the issue SECURE: MESSAGE FROM ezhang113 ON 2/28/23, 1:22 PM I found something similar: https://github.com/bbuchfink/diamond_docs/blob/master/Documentation.MD<https://urldefense.com/v3/__https:/github.com/bbuchfink/diamond_docs/blob/master/Documentation.MD__;!!LLK065n_VXAQ!nE10uQPoqmMAPEfRalhwxB5CCs04-ZUGGW36AAd8lFTmjQ-KNzi7Hqu-8K6HZabtH7RAz8fxdP0C8oaNFPlV5XYGVwFg$> that suggests this command to prep fasta databases to run with diamond before the diamond command $ diamond makedb --in nr.faa -d nr but now I get an error: Command must be given as string after the shell keyword. (Snakefile, line 74) which is so strange because the command is as a string so I'm thinking it's probably a different error. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/YeoLab/eclip-qc/issues/7*issuecomment-1448933330__;Iw!!LLK065n_VXAQ!nE10uQPoqmMAPEfRalhwxB5CCs04-ZUGGW36AAd8lFTmjQ-KNzi7Hqu-8K6HZabtH7RAz8fxdP0C8oaNFPlV5XA--mNn$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AB7TJP6VZTD5LDAILOVKWWTWZZT7JANCNFSM6AAAAAAUXFFCW4__;!!LLK065n_VXAQ!nE10uQPoqmMAPEfRalhwxB5CCs04-ZUGGW36AAd8lFTmjQ-KNzi7Hqu-8K6HZabtH7RAz8fxdP0C8oaNFPlV5XE7fEME$>. You are receiving this because you commented.Message ID: ***@***.***>

[ezhang113]

hmm I tried changing the version and I get a new issue:
TypeError in line 71 of /oasis/tscc/scratch/eczhang/snakemake/Snakefile:
Workflow.conda() missing 1 required positional argument: 'conda_env'
File "/oasis/tscc/scratch/eczhang/snakemake/Snakefile", line 71, in

I'm now thinking it's an issue with the envs/diamond.yaml file I made

[ezhang113]

@byee4
https://github.com/YeoLab/eclip-qc/tree/testing-3/11 is the working branch with Snakefile_Wrapper_Test as the wrapper test file and Snakefile the original pipeline code with the diamond command.

[byee4]

@ezhang113 Fix your pie chart script but the blastx rule works: https://github.com/YeoLab/eclip-qc/tree/testing-3/11-brian