conda environment error

[erbon7]

Hi, I have an error while trying to run the hic pipeline on the test data with the conda environment.

Looks like a python module is missing.

Thanks for your help

$ ./nextflow  run nf-core/hic -profile test,conda
N E X T F L O W  ~  version 21.04.3
Launching `nf-core/hic` [distracted_swanson] - revision: ac74763a91 [master]


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/hic v1.3.0
------------------------------------------------------

WARN: The `into` operator should be used to connect two or more target channels -- consider to replace it with `.set { fasta_for_index }`
Core Nextflow options
  revision                     : master
  runName                      : distracted_swanson
  container                    : nfcore/hic:1.3.0
  launchDir                    : /env/export/v_bigtmp/eb/test
  workDir                      : /env/export/v_bigtmp/eb/test/work
  projectDir                   : /home/bonnet/.nextflow/assets/nf-core/hic
  userName                     : bonnet
  profile                      : test,conda
  configFiles                  : /home/bonnet/.nextflow/assets/nf-core/hic/nextflow.config

Input/output options
  input_paths                  : [[SRR4292758_00, [https://github.com/nf-core/test-datasets/raw/hic/data/SRR4292758_00_R1.fastq.gz, https://github.com/nf-core/test-datasets/raw/hic/data/SRR4292758_00_R2.fastq.gz]]]

Reference genome options
  fasta                        : https://github.com/nf-core/test-datasets/raw/hic/reference/W303_SGD_2015_JRIU00000000.fsa

Digestion Hi-C
  restriction_site             : A^AGCTT
  ligation_site                : AAGCTAGCTT

DNAse Hi-C
  min_cis_dist                 : 0

Alignments
  bwt2_opts_end2end            : --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder
  bwt2_opts_trimmed            : --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder

Valid Pairs Detection
  max_insert_size              : 600
  min_insert_size              : 100
  max_restriction_fragment_size: 100000
  min_restriction_fragment_size: 100

Contact maps
  bin_size                     : 1000
  ice_filter_high_count_perc   : 0

Downstream Analysis
  res_dist_decay               : 1000
  tads_caller                  : insulation,hicexplorer
  res_tads                     : 1000
  res_compartments             : 1000

Generic options
  max_multiqc_email_size       : 25 MB

Max job request options
  max_cpus                     : 2
  max_memory                   : 4 GB
  max_time                     : 1h

Institutional config options
  config_profile_name          : Hi-C test data from Schalbetter et al. (2017)
  config_profile_description   : Minimal test dataset to check pipeline function

------------------------------------------------------
 Only displaying parameters that differ from defaults.
------------------------------------------------------
executor >  local (29)
[c1/b39a04] process > get_software_versions                                                                                      [100%] 1 of 1 ✔
[9b/c38a4f] process > makeBowtie2Index (W303_SGD_2015_JRIU00000000)                                                              [100%] 1 of 1 ✔
[d5/16c011] process > makeChromSize (W303_SGD_2015_JRIU00000000.fsa)                                                             [100%] 1 of 1 ✔
[a6/f68e54] process > getRestrictionFragments (W303_SGD_2015_JRIU00000000.fsa A^AGCTT)                                           [100%] 1 of 1 ✔
[33/2a727e] process > bowtie2_end_to_end (SRR4292758_00_R1)                                                                      [100%] 2 of 2 ✔
[ea/c7188f] process > trim_reads (SRR4292758_00_R2)                                                                              [100%] 2 of 2 ✔
[3e/3ca322] process > bowtie2_on_trimmed_reads (SRR4292758_00_R2)                                                                [100%] 2 of 2 ✔
[ce/d74ebe] process > bowtie2_merge_mapping_steps (SRR4292758_00_R2 = SRR4292758_00_R2.bam + SRR4292758_00_R2_unmap_trimmed.bam) [100%] 2 of 2 ✔
[bd/fd6221] process > combine_mates (SRR4292758_00 = SRR4292758_00_R1 + SRR4292758_00_R2)                                        [100%] 1 of 1 ✔
[c9/c1e0bc] process > get_valid_interaction (SRR4292758_00)                                                                      [100%] 1 of 1 ✔
[f5/2f1982] process > remove_duplicates (SRR4292758_00)                                                                          [100%] 1 of 1 ✔
[3f/680f09] process > merge_stats (mRSstat)                                                                                      [100%] 4 of 4 ✔
[-        ] process > build_contact_maps                                                                                         -
[-        ] process > run_ice                                                                                                    -
[1a/f1bc79] process > convert_to_pairs (SRR4292758_00)                                                                           [100%] 1 of 1 ✔
[74/2a2b0e] process > cooler_raw (SRR4292758_00 - 1000)                                                                          [100%] 1 of 1 ✔
[7e/a1b205] process > cooler_balance (SRR4292758_00 - 1000)                                                                      [100%] 1 of 1 ✔
[98/37cb6f] process > cooler_zoomify (SRR4292758_00)                                                                             [100%] 1 of 1 ✔
[76/f1f2bd] process > dist_decay (SRR4292758_00)                                                                                 [100%] 1 of 1 ✔
[33/859499] process > compartment_calling (SRR4292758_00 - 1000)                                                                 [100%] 1 of 1, failed: 1 ✘
[80/ec5fcd] process > tads_hicexplorer (SRR4292758_00 - 1000)                                                                    [100%] 1 of 1 ✔
[d5/3959f7] process > tads_insulation (SRR4292758_00 - 1000)                                                                     [100%] 1 of 1, failed: 1 ✘
[a3/21a9d8] process > multiqc                                                                                                    [100%] 1 of 1 ✔
[e5/9db365] process > output_documentation                                                                                       [100%] 1 of 1 ✔
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/hic] Pipeline completed with errors-
Error executing process > 'tads_insulation (SRR4292758_00 - 1000)'

Caused by:
  Process `tads_insulation (SRR4292758_00 - 1000)` terminated with an error exit status (1)

Command executed:

  cooltools diamond-insulation --window-pixels SRR4292758_00_1000_norm.cool 15 25 50 > SRR4292758_00_insulation.tsv

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/env/export/v_bigtmp/eb/test/work/conda/nf-core-hic-1.3.0-6b228b9d59279339a55b4a54b5df58de/bin/cooltools", line 5, in <module>
      from cooltools.cli import cli
    File "/env/export/v_bigtmp/eb/test/work/conda/nf-core-hic-1.3.0-6b228b9d59279339a55b4a54b5df58de/lib/python3.7/site-packages/cooltools/__init__.py", line 18, in <module>
      from .lib import numutils, download_data, print_available_datasets, get_data_dir, download_file, get_md5sum
    File "/env/export/v_bigtmp/eb/test/work/conda/nf-core-hic-1.3.0-6b228b9d59279339a55b4a54b5df58de/lib/python3.7/site-packages/cooltools/lib/numutils.py", line 8, in <module>
      import numba
  ModuleNotFoundError: No module named 'numba'

Work dir:
  /env/export/v_bigtmp/eb/test/work/d5/3959f75f7d599155c053cca573ae37

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

[nservant]

Hi @erbon7
Thanks for reporting the issue. Indeed I can reproduce it, but I'm really surprise about it.
I'll try to investigate and come back to you
N

[nservant]

@erbon7
It seems that installing cooltools from bioconda instead of pip solves the issue.
I'll need to make additional test, to update the environment and make a new release ... which could take some times.

In the meantime, you can for instance, manually update your conda env as follow ;

conda activate work/conda/nf-core-hic-1.3.0-6b228b9d59279339a55b4a54b5df58de/
conda install -c bioconda cooltools
conda deactivate

[nservant]

Note that once your conda env is created (and corrected), you can set up a small config file for the conda cache dir to avoid regenerating the conda each time ...

$ cat conda.conf
conda {
  cacheDir = my_conda_dir
}
$ nextflow  run nf-core/hic -profile test,conda -c conda.conf

[erbon7]

Indeed, with the patch the pipeline runs successfully.
Thanks Nicolas for your fast response and for the tip.

[zcmit]

@erbon7 It seems that installing cooltools from bioconda instead of pip solves the issue. I'll need to make additional test, to update the environment and make a new release ... which could take some times.

In the meantime, you can for instance, manually update your conda env as follow ;

conda activate work/conda/nf-core-hic-1.3.0-6b228b9d59279339a55b4a54b5df58de/
conda install -c bioconda cooltools
conda deactivate

Hi, I got a similar error, but to install 'cooltools from 'bioconda' doesn't fix it.
Any thoughts about it will be really appreciated.

Core Nextflow options
revision : 1.3.0
runName : thirsty_lamarck
container : nfcore/hic:1.3.0
launchDir : /media/b/Bigdata3/Chi/Ales_Cvekl/HiC_fastq
workDir : /media/b/Bigdata3/Chi/Ales_Cvekl/HiC_fastq/work
projectDir : /home/b/.nextflow/assets/nf-core/hic
userName : b
profile : test,conda
configFiles : /home/b/.nextflow/assets/nf-core/hic/nextflow.config, /media/b/Bigdata3/Chi/Ales_Cvekl/HiC_fastq/conda_hic.conf

Error executing process > 'tads_insulation (SRR4292758_00 - 1000)'

Caused by:
Process tads_insulation (SRR4292758_00 - 1000) terminated with an error exit status (2)

Command executed:

cooltools diamond-insulation --window-pixels SRR4292758_00_1000_norm.cool 15 25 50 > SRR4292758_00_insulation.tsv

Command exit status:
2

Command output:
(empty)

Command error:
Usage: cooltools [OPTIONS] COMMAND [ARGS]...
Try 'cooltools -h' for help.

Error: No such command 'diamond-insulation'.

Work dir:
/media/b/Bigdata3/Chi/Ales_Cvekl/HiC_fastq/work/4c/66d24e86e4b823228ce8865ec6305e

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh