Unresolved problem with concoct_coverage_table.py - write() argument must be str, not bytes

[Marlinski95]

Hi,
I am currently experiencing issues with the concoct_coverage_table.py file. I know that there is already entries for similar issues but none of them resolved my problem. I ran the split command (which worked perfectly fine) and have all my .bam and .bam.bai files in the same directory:

[mjensen2 BC-1]$ ls

BC-1_contigs_10K.bed
BC-1_contigs_10K.fa
BC-1_Mapped_BC-1_BGI_Sorted.bam
BC-1_Mapped_BC-1_GSL_Sorted.bam
BC-1_Mapped_BC-4_BGI_Sorted.bam
BC-1_Mapped_BC-4_GSL_Sorted.bam
BC-1_final.contigs.fa
BC-1_Mapped_BC-1_BGI_Sorted.bam.bai
BC-1_Mapped_BC-1_GSL_Sorted.bam.bai
BC-1_Mapped_BC-4_BGI_Sorted.bam.bai
BC-1_Mapped_BC-4_GSL_Sorted.bam.bai

When I run concoct_coverage_table.py BC-1_contigs_10K.bed ./*_Sorted.bam > BC-1_coverage_table.tsv I get the following error message + a completely empty .tsv file

Errors in BED line 'k127_6325776 0 356 k127_6325776'
Errors in BED line 'k127_6325777 0 391 k127_6325777'
Errors in BED line 'k127_6325778 0 989 k127_6325778'
Errors in BED line 'k127_6325779 0 597 k127_6325779'
Errors in BED line 'k127_6325780 0 404 k127_6325780'
Errors in BED line 'k127_6325781 0 636 k127_6325781'
Errors in BED line 'k127_6325782 0 405 k127_6325782'
Errors in BED line 'k127_6325783 0 592 k127_6325783'
Errors in BED line 'k127_6325784 0 347 k127_6325784'
Errors in BED line 'k127_6325785 0 2192 k127_6325785'
Errors in BED line 'k127_6325786 0 443 k127_6325786'
Errors in BED line 'k127_6325787 0 422 k127_6325787'

Traceback (most recent call last):
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 77, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 28, in generate_input_table
sys.stderr.write(out)
TypeError: write() argument must be str, not bytes

I re-installed CONCOCT in a conda environment and tried to rerun it but it gives me the same error. I am using python Python 3.9.2 and concoct 1.1.0 in the environment I created. In other posts I also saw that some headers in the coverage file caused issues but since mine is completely empty that wasn't something I could do.

I also tried to run each file individually and that gave me the following error:

ERROR: fail to open index BAM file 'BC-1_Mapped_BC-1_BGI.bam'
Traceback (most recent call last):
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 77, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 28, in generate_input_table
sys.stderr.write(out)
TypeError: write() argument must be str, not bytes

I don't quite understand why this is a problem as the .bam and .bam.bai files work completely fine with other binning tools I tried. I would appreciate your help with this!

Cheers,
Marlene

[AriBad]

Hi,

I have the same problem. I use CONCOCT with cwl. When I call concoct in a biocontainers there is no problem but when I call it with a conda environment (with bioconda) from cwl I have exactly the same error.

Is there a solution ?

Thanks,
Ariane

[AriBad]

Hi !

I solve the problem if you are still interested. You must precise to install samtools version 1.7=1 and it work.

good afternoon,
Ariane

[Marlinski95]

Wonderful, thank you!

…

On Tue, Oct 26, 2021 at 9:46 AM Ariane ***@***.***> wrote: Hi ! I solve the problem if you are still interested. You must precise to install samtools version 1.7=1 and it work. good afternoon, Ariane — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#307 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMWYRTIRB7QYGKLMEAGW2OLUI25NXANCNFSM5EWIA3YQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[saras224]

Hello! @Marlinski95
Did it work? Installing samtools 1.7=1?

[hannah-doris]

I am having this same issue. I was wondering if I could have some clarification on the samtools 1.7=1. Does this mean I need to use a specific samtools version? I have been using 1.10.

[hannah-doris]

Hey just to follow up on this again. How was this solved. I have noticed if I use Bowtie2 to map instead of BWA or Minimap I am able to successfully run CONCOCT. But when I use BWA or Minimap I have this same problem. I would prefer to use BWA or Minimap as it is quicker and I have less memory error issues. Any advice would be greatly appreciated.

[Bennitokscience]

Hello! I've been searching for this problem all over the forum, but I haven't found a solution. I have the same issue where the program doesn't recognize the index or the .bam file. I read a comment that suggested that indexing with samtools version 1.0 would solve it, but I'm still getting the same error. Has anyone managed to solve it?

(concoct_env) [cjerez@soroban bowtie2]$ concoct_coverage_table.py ~/matias/raw_data/concoct/C17A_bin/C17A_10K.bed /home/bioren/cjerez/matias/raw_data/bowtie2/C17A-sorted.bam > C17A/C17A_coverage_table.tsv
[E::hts_open] fail to open file '/home/bioren/cjerez/matias/raw_data/bowtie2/C17A-sorted.bam'
Traceback (most recent call last):
File "/home/bioren/cjerez/miniconda3/envs/concoct_env/bin/concoct_coverage_table.py", line 91, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/home/bioren/cjerez/miniconda3/envs/concoct_env/bin/concoct_coverage_table.py", line 41, in generate_input_table
sys.stderr.write(out)
TypeError: write() argument must be str, not bytes

[hannah-doris]

Hey @Bennitokscience

I have not found a solution yet and I have reached out to many people. I have heard from many people that I have to use version 1.17 but that hasn't solved my problem yet. I have found that when I use Bowtie2 for mapping I am able to run CONCOCT afterwards without any errors. Yet if I run Minimap or BWA for mapping I get the error:

Errors in BED line 'k127_6325786 0 443 k127_6325786'
Errors in BED line 'k127_6325787 0 422 k127_6325787'

Traceback (most recent call last):
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 77, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/tools/miniconda3/envs/CONCOCT/bin/concoct_coverage_table.py", line 28, in generate_input_table
sys.stderr.write(out)
TypeError: write() argument must be str, not bytes

Unfortunately it looks like you do use bowtie2 for mapping so not sure.

[Bennitokscience]

Hi @hannah-doris I found the solution to my problem, at least. It turns out that when I was running samtools, I only had a .bam file, which was the sorted.bam. However, before performing the "sort" operation, I needed to execute the samtools view -bS command to convert my .sam file to .bam format. This file should be kept because the Python script requires the .bam files. That's why if you only have the sorted.bam file, the script conflicts and doesn't recognize it. Therefore, in your folder, you should have your reads.bam (from samtools view), your sorted.bam (from samtools sort), and your sorted.bam.bai (from samtools index). I hope this helps you. Best regards.
P.S. I performed the mapping using bowtie2.