another type of error

[ospfsg]

Hi

After editing the reference genome file, as you suggested, it worked for individual sample files but there is still a problem with PoolSeq data . It crashed and the terminal closed by himself. The problem seems to be during the snpcalling

This is my config part for freebayes

  # ----------------------------------------------------------------------
  #     freebayes
  # ----------------------------------------------------------------------

  # Used only if settings:calling-tool == freebayes
  # See https://github.com/freebayes/freebayes
  freebayes:

    # Extra parameters for freebayes.
    extra: "-p 40 --use-best-n-alleles 4 --pooled-discrete"

    # Settings for parallelization
    threads: 120
    compress-threads: 2
    chunksize: 100000

I attached the log file

2024-04-09T151509.566084.snakemake.log

and the two log files that seems to be relevant

log: logs/freebayes/JAVFWQ010000033.1.log

JAVFWQ010000033.1.log

log: logs/freebayes/chromosome_1.log

chromosome_1.log

thank you again
osp

[lczech]

Hi @ospfsg,

interesting. What exactly do you mean by

It crashed and the terminal closed by himself.

Without any error message, the terminal just closes? That does not seem like anything that would happen from running snakemake or grenepipe, so there must be something more weird going on. Could you somehow maybe give me some steps for how I could reproduce this behavior, or describe what is happening in more detail? Or even better, use a screen recording tool to get a video clip of what is happening?

From your log files, there is nothing that explains the behavior that you describe. I am seeing

Invalid tag name: "technology.-"

in there. Is that name something that shows up anywhere in your data, as a sample name or chromosome name, or somewhere else? Apart from that, there are no clues that would help right now.

I am traveling and won't be able to investigate this issue for another week. I'll then try to reproduce with the freebayes settings that you posted above. Let's see if I can recreate the issue as well. If not, and in the meantime: How big is your data set? Could you maybe prepare a minimal subset of your data that causes the issue, and that I can use to investigate? It seems like a rather involved and complicated issue, if the terminal just closes. I have never seen that, and tools such as freebayes, snakemake (and grenepipe) should usually not do that - hence, I have not really a good idea what is going on there.

So long, let's see what we can figure out
Lucas

[ospfsg]

Hi

I just run 1/4 of the poolseq dataset and everything went smoothly in less than 24 hours, generating 65Million SNPs!

So maybe is my workstation (128 threads and 512GB RAM) that doesn't cope with the full dataset. I am going to try with increase amount to check how far I can go.

osp

[lczech]

Hi @ospfsg,

how are things going? Did the pipeline work now with a reduced data set? From what you described, I can only guess what has happened - but if it works with smaller data, and fails with larger, then it is rather likely that this is some issue with memory or your workstation, instead of something related to grenepipe itself.

Let me know what you think, and if we can close this issue then.

Cheers and so long
Lucas