Error executing process > 'NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)'

[anijudy]

Description of the bug

Hi everyone!

I was running the atacseq pipeline v 2.0 using Slurm as the executor. However, towards the end, it raised an error as following:

Error executing process > 'NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)'

Caused by:
  Process `NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)` terminated with an error exit status (1)

The complete message is copied in the Command used and terminal output session. The script, config file, log files, and the sample sheet can be found in the zip folder. Can someone help me understand what's going on and how this problem can be fixed?

Thank you so much!

Best,
Yifan

Command used and terminal output

# Command Used
#!/bin/bash
#SBATCH --job-name=ATACseq                     # Job nname
#SBATCH --cpus-per-task=4                       # Number of cpus for each task -c for short
#SBATCH --mem=16GB                              # Job memory request
#SBATCH --error=nextflow_job_%j.err             # Standard output and error log
#SBATCH --output=nextflow_job_%j.out

### activate nextflow conda env before running batch job
### see enviroment_nextflow.yml for conda env setup

date
pwd



nextflow run nf-core/atacseq \
	 -resume \
	 -profile singularity \
	 -r 2.0 \
	 --input design.csv \
	 --genome mm10 \
	 --read_length 150 \
	 --outdir results 

date


# Output from the Execution Report

Error executing process > 'NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)'

Caused by:
  Process `NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)` terminated with an error exit status (1)

Command executed [/Fingerlin/home/zhaoy/.nextflow/assets/nf-core/atacseq/./workflows/../modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoftwareversions.py]:

  #!/usr/bin/env python
  
  
  """Provide functions to merge multiple versions.yml files."""
  
  
  import yaml
  import platform
  from textwrap import dedent
  
  
  def _make_versions_html(versions):
      """Generate a tabular HTML output of all versions for MultiQC."""
      html = [
          dedent(
              """\
              
              
              """
          )
      ]
      for process, tmp_versions in sorted(versions.items()):
          html.append("")
          for i, (tool, version) in enumerate(sorted(tmp_versions.items())):
              html.append(
                  dedent(
                      f"""\
                      
                      """
                  )
              )
          html.append("")
      html.append("
 Process Name 	 Software 	 Version  
{process if (i == 0) else ''}	{tool}	{version}
")
      return "\n".join(html)
  
  
  def main():
      """Load all version files and generate merged output."""
      versions_this_module = {}
      versions_this_module["NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS"] = {
          "python": platform.python_version(),
          "yaml": yaml.__version__,
      }
  
      with open("collated_versions.yml") as f:
          versions_by_process = yaml.load(f, Loader=yaml.BaseLoader) | versions_this_module
  
      # aggregate versions by the module name (derived from fully-qualified process name)
      versions_by_module = {}
      for process, process_versions in versions_by_process.items():
          module = process.split(":")[-1]
          try:
              if versions_by_module[module] != process_versions:
                  raise AssertionError(
                      "We assume that software versions are the same between all modules. "
                      "If you see this error-message it means you discovered an edge-case "
                      "and should open an issue in nf-core/tools. "
                  )
          except KeyError:
              versions_by_module[module] = process_versions
  
      versions_by_module["Workflow"] = {
          "Nextflow": "23.04.1",
          "nf-core/atacseq": "2.0",
      }
  
      versions_mqc = {
          "id": "software_versions",
          "section_name": "nf-core/atacseq Software Versions",
          "section_href": "https://github.com/nf-core/atacseq",
          "plot_type": "html",
          "description": "are collected at run time from the software output.",
          "data": _make_versions_html(versions_by_module),
      }
  
      with open("software_versions.yml", "w") as f:
          yaml.dump(versions_by_module, f, default_flow_style=False)
      with open("software_versions_mqc.yml", "w") as f:
          yaml.dump(versions_mqc, f, default_flow_style=False)
  
      with open("versions.yml", "w") as f:
          yaml.dump(versions_this_module, f, default_flow_style=False)
  
  
  if __name__ == "__main__":
      main()

Command exit status:
  1

Command output:
  (empty)

Command error:
  INFO:    Converting SIF file to temporary sandbox...
  Traceback (most recent call last):
    File ".command.sh", line 101, in 
      main()
    File ".command.sh", line 61, in main
      versions_by_process = yaml.load(f, Loader=yaml.BaseLoader) | versions_this_module
    File "/usr/local/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
      return loader.get_single_data()
    File "/usr/local/lib/python3.10/site-packages/yaml/constructor.py", line 49, in get_single_data
      node = self.get_single_node()
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 36, in get_single_node
      document = self.compose_document()
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 55, in compose_document
      node = self.compose_node(None, None)
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 84, in compose_node
      node = self.compose_mapping_node(anchor)
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 133, in compose_mapping_node
      item_value = self.compose_node(node, item_key)
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 84, in compose_node
      node = self.compose_mapping_node(anchor)
    File "/usr/local/lib/python3.10/site-packages/yaml/composer.py", line 127, in compose_mapping_node
      while not self.check_event(MappingEndEvent):
    File "/usr/local/lib/python3.10/site-packages/yaml/parser.py", line 98, in check_event
      self.current_event = self.state()
    File "/usr/local/lib/python3.10/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
      if self.check_token(KeyToken):
    File "/usr/local/lib/python3.10/site-packages/yaml/scanner.py", line 116, in check_token
      self.fetch_more_tokens()
    File "/usr/local/lib/python3.10/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens
      return self.fetch_value()
    File "/usr/local/lib/python3.10/site-packages/yaml/scanner.py", line 577, in fetch_value
      raise ScannerError(None, None,
  yaml.scanner.ScannerError: mapping values are not allowed here
    in "collated_versions.yml", line 91, column 13
  INFO:    Cleaning up image...

Work dir:
  /proj/CGEHcore/Alam_Alternaria_ILC2_mouse_ATACSeq/preprocessing_repooling2/work/29/e31924c2822ef8f83d5b1a5e500fb7

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

Relevant files

Script_Config_Log.zip

System information

• nextflow version 23.04.1.5866
• Hardware: HPC
• Executor: Slurm
• Container engine: Singularity
• OS: Linux
• Version of nf-core/atacseq: 2.0

[anijudy]

Hi everyone,

I pulled the latest version of atacseq pipeline using nextflow pull nf-core/atacseq -r 2.1.2 and ran the pipeline with the option -r 2.1.2. It finished without any problem. So I guess the new version fixed the problem somehow.

Thanks,
Yifan