Workflow execution completed unsuccessfully! Error executing process > 'calling_pipeline:medakaNetwork (2)'

[Lenfera]

Operating System

Ubuntu 22.04

Other Linux

No response

Workflow Version

v0.4.0

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

v5.1.2

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

Hello Epi2me labs team,

We tried to run the workflow 'bacterial genome' within an oracle VM with Ubuntu 22.04.3 LTS 64-bit in the Epi2me labs desktop application.
To check if it works, we tried 'Use Demo Data'

We had this report :

Workflow execution completed unsuccessfully!
The exit status of the task that caused the workflow execution to fail was: 134.

The full error message was:

Error executing process > 'calling_pipeline:medakaNetwork (2)'

Caused by:
Process calling_pipeline:medakaNetwork (2) terminated with an error exit status (134)

Command executed:

medaka --version
echo r1041_e82_400bps_sup_v4.2.0

echo r1041_e82_400bps_sup_v4.2.0

medaka consensus align.bam "test1.consensus_probs.hdf" --threads 2 --regions "NC_000962.3:999000-1999000

" --model r1041_e82_400bps_sup_v4.2.0

Command exit status:
134

Command output:
medaka 1.9.1
r1041_e82_400bps_sup_v4.2.0
r1041_e82_400bps_sup_v4.2.0

Command error:
Cannot import pyabpoa, some features may not be available.
medaka 1.9.1
r1041_e82_400bps_sup_v4.2.0
r1041_e82_400bps_sup_v4.2.0
The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
.command.sh: line 8: 90 Aborted (core dumped) medaka consensus align.bam "test1.consensus_probs.hdf" --threads 2 --regions "NC_000962.3:999000-1999000
" --model r1041_e82_400bps_sup_v4.2.0

Work dir:
/home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/work/72/fb42aff10d6ba33fdc644968b2c3ff

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

I tried this : tensorflow/tensorflow#24548
(mehdirezaie commented on Mar 27, 2020)

But we still have the same error.
What could we do to solve this ?

Relevant log output

N E X T F L O W  ~  version 23.04.2
Launching `/home/sprzto/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes/main.nf` [wizardly_rosalind] DSL2 - revision: 3d3482183d
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-bacterial-genomes v0.4.0
--------------------------------------------------------------------------------
Core Nextflow options
  runName                 : wizardly_rosalind
  containerEngine         : docker
  launchDir               : /home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK
  workDir                 : /home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/work
  projectDir              : /home/sprzto/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes
  userName                : sprzto
  profile                 : standard
  configFiles             : /home/sprzto/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes/nextflow.config, /home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/demo.config
Input Options
  fastq                   : /home/sprzto/epi2melabs/demo/epi2me-labs/wf-bacterial-genomes/v0.4.0/wf-bacterial-genomes-demo/isolates_fastq
  reference_based_assembly: true
  reference               : /home/sprzto/epi2melabs/demo/epi2me-labs/wf-bacterial-genomes/v0.4.0/wf-bacterial-genomes-demo/ref/ref.fasta.gz
Sample Options
  sample_sheet            : /home/sprzto/epi2melabs/demo/epi2me-labs/wf-bacterial-genomes/v0.4.0/wf-bacterial-genomes-demo/isolates_sample_sheet.csv
Output Options
  out_dir                 : /home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/output
Isolate options
  isolates                : true
Advanced Options
  threads                 : 3
!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-bacterial-genomes for your analysis please cite:
* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x
--------------------------------------------------------------------------------
This is epi2me-labs/wf-bacterial-genomes v0.4.0.
--------------------------------------------------------------------------------
Checking fastq input.
Reference based assembly selected.
[43/4d29ed] Submitted process > calling_pipeline:getParams
[07/745d32] Submitted process > calling_pipeline:lookup_medaka_consensus_model (1)
[81/f490ce] Submitted process > calling_pipeline:prokkaVersion
[f7/f99deb] Submitted process > calling_pipeline:lookup_medaka_variant_model (1)
[f0/d397d2] Submitted process > validate_sample_sheet
[7e/199664] Submitted process > calling_pipeline:medakaVersion
[4d/6d385d] Submitted process > calling_pipeline:mlstVersion
[5b/d1a558] Submitted process > calling_pipeline:getVersions
[b5/a842fb] Submitted process > fastcat (1)
[57/e3b0f0] Submitted process > fastcat (2)
[60/d2ab0f] Submitted process > calling_pipeline:collectFastqIngressResultsInDir (1)
[4b/2f1d69] Submitted process > calling_pipeline:alignReads (1)
[cb/50852a] Submitted process > calling_pipeline:collectFastqIngressResultsInDir (2)
[16/ac5817] Submitted process > calling_pipeline:alignReads (2)
[02/076eb3] Submitted process > calling_pipeline:readStats (1)
[e2/641ec3] Submitted process > calling_pipeline:splitRegions (1)
[12/82ae58] Submitted process > calling_pipeline:coverStats (1)
[ec/986fa7] Submitted process > calling_pipeline:splitRegions (2)
[df/f0eb5c] Submitted process > calling_pipeline:readStats (2)
[85/5daa57] Submitted process > calling_pipeline:coverStats (2)
[72/fb42af] Submitted process > calling_pipeline:medakaNetwork (2)
[33/178ff6] Submitted process > calling_pipeline:medakaNetwork (3)
[8f/6feb25] Submitted process > calling_pipeline:medakaVariantHdf (4)
ERROR ~ Error executing process > 'calling_pipeline:medakaNetwork (2)'
Caused by:
  Process `calling_pipeline:medakaNetwork (2)` terminated with an error exit status (134)
Command executed:
  medaka --version
      echo r1041_e82_400bps_sup_v4.2.0
  
      echo r1041_e82_400bps_sup_v4.2.0
  
      medaka consensus align.bam "test1.consensus_probs.hdf"         --threads 2 --regions "NC_000962.3:999000-1999000
  " --model r1041_e82_400bps_sup_v4.2.0
Command exit status:
  134
Command output:
  medaka 1.9.1
  r1041_e82_400bps_sup_v4.2.0
  r1041_e82_400bps_sup_v4.2.0
Command error:
  Cannot import pyabpoa, some features may not be available.
  medaka 1.9.1
  r1041_e82_400bps_sup_v4.2.0
  r1041_e82_400bps_sup_v4.2.0
  The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
  .command.sh: line 8:    90 Aborted                 (core dumped) medaka consensus align.bam "test1.consensus_probs.hdf" --threads 2 --regions "NC_000962.3:999000-1999000
  " --model r1041_e82_400bps_sup_v4.2.0
Work dir:
  /home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/work/72/fb42aff10d6ba33fdc644968b2c3ff
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
 -- Check '/home/sprzto/epi2melabs/instances/wf-bacterial-genomes_01HBGHJP7JQSFKRC9P6QD7WTJK/nextflow.log' file for details
WARN: Killing running tasks (2)

Application activity log entry

No response

[Lenfera]

Do you need further information?

[SamStudio8]

@Lenfera Can you provide some information about the host machine?

 The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.

It looks like it does not support the AVX instructions required for TensorFlow to work correctly.

[Lenfera]

It is a Oracle VM Virtual Box.
Hardware Model: Innotek GmbH VirtualBox
Memory: 7,7 Gib
Processor: Intel® Core™ i9-10920X CPU @ 3.50GHz x 6
Graphics: llvmpipe (LLVM 15.0.7, 128 bits)
Disk capacity: 214,7 GB
OS Name: Ubuntu 22.04.3 LTS
OS type: 64-bit
GNOME Version: 42.9
Windowing System: Wayland
Virtualization: Oracle

And I tried to install directly tensorflow following this "method" :

tensorflow/tensorflow#24548
(mehdirezaie commented on Mar 27, 2020)

But it didn't work, I still have the same issue.

[cjw85]

Hi @Lenfera,

My hunch here is that this is likely due to you running in a virtual machine. We've seen issues before with computational code such as medaka when running in VMs because often they do not support certain types of processor instruction.

Is it possible for you to try running the code outside of Oracle Virtual Box. On Windows we test and deploy our workflows under Windows Subsytem for Linux.

[cjw85]

Closing through lack of response.