Earl Grey

[TobyBaril]

First Release of Earl Grey: A fully automated combined library-based and de novo TE annotation pipeline, integrating a BLAST, Extract, Extend process and generating informative summary figures for any input genome.

---

This discussion was created from the release Earl Grey.

[SJ-Smit]

Hi Toby. Cant wait to try this out! I attended your BGA23 workshop, which was great btw! I see on that gitpod yml that repeatmasker and repeatmodeler were installed via mamba in the conda env. I am assuming that this is acceptable as an alternative to your github instructions via the tar route?

[TobyBaril]

Hi @SJ-Smit, following on from this, Earl Grey is now available as a completely configured package in Bioconda! This of course contains RepeatModeler and RepeatMasker, so should make things significantly easier.

[TobyBaril]

Hi! Thanks for checking out the workshop! So yes, on the gitpod we configured these in mamba. It is possible for you to do this, but it can be a bit painful to get them to configure properly, and also to update and reconfigure RepeatMasker with the correct DFAM libraries, which you will also need to do. If you get it working and tested, it should work as long as it is in the same environment when you start running Earl Grey.

[hkania]

Hi Dr. Baril,

I am using EarlGrey for de novo TE identification in lemur genomes after attending BGA23. I downloaded the bioconda package following the instructions on GitHub (version 3.1). I am running into some unexpected outputs and roadblocks, and I am wondering if you have any advice.

1. When I run the program, the final TE proportions are less repeat content than expected (~10% total, where it should be ~45%).
2. I noticed that the generated [species]_strainer/ directories have sequences in them, but the [species]_Curated_Library/ directories are empty. Should they be empty?
3. Can you provide advice on how much memory to request? The genomes are around 2.7Gb.
4. Lastly, should I update to 3.2, and if so, is there a quick conda command?

If it is useful, I can share the script that I use to submit the jobs on my cluster. The log files make it seem as if the jobs are completed without major issues.

I sincerely appreciate any advice you can provide!

Cheers,
Hannah

[TobyBaril]

Hi Hannah,
I’m still looking into/thinking about this, but currently at a conference until Sunday. It looks like python is using a different biopython version to the one supplied in conda (this is a new attribute in the latest biopython). This can sometimes happen depending on how conda and system libraries have been installed, which sometimes causes issues on servers. Do you have anyone who manages your server? They might be able to debug this in the time that I have a deeper look, as this is likely specific to your server setup.

[hkania]

Hi Toby,

Thank you for your help so far!

I did a little bit of digging, and the Biopython version that was installed into the earlgrey environment is 1.78. I am attempting to update to version 1.81, which I believe should be the compatible package version. I will keep you posted. Enjoy the conference!

Cheers,
Hannah

[TobyBaril]

Hi Hannah,

Any luck with updating BioPython? I just checked my environments on the clusters here and we have 1.81. If this doesn't work for you, I can give you the changes to make so it plays nicely with older BioPython!

[hkania]

Hi Toby,

The update to 1.81 seems to have worked! The RepeatMasker output table is up to 37%.

For some reason when I run it as a batch script on a shared cluster, it says sh: /dev/tty: No such device or address during the TEstrainer run. This doesn't seem to be affecting the outputs, so I believe it is good to go.

Thank you so much for all of your help!

Cheers,
Hannah

[TobyBaril]

That's great!

Yes we get the same running on our clusters here. It doesn't affect the running of the pipeline at all, and is due to slurm running jobs without controlling terminals (ie we can't get a nice loading bar!).

Glad that worked for you!

[robaina62]

Hi Toby,
I tried earlgrey installation via bioconda package and found the following:
(Bioconda) C:\Users\gaude>conda create -n earlgrey -c conda-forge -c bioconda earlgrey
Retrieving notices: ...working... done
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

• earlgrey

Current channels:

• https://conda.anaconda.org/conda-forge/win-64
• https://conda.anaconda.org/conda-forge/noarch
• https://conda.anaconda.org/bioconda/win-64
• https://conda.anaconda.org/bioconda/noarch
• https://conda.anaconda.org/sean/win-64
• https://conda.anaconda.org/sean/noarch
• https://repo.anaconda.com/pkgs/main/win-64
• https://repo.anaconda.com/pkgs/main/noarch
• https://repo.anaconda.com/pkgs/r/win-64
• https://repo.anaconda.com/pkgs/r/noarch
• https://repo.anaconda.com/pkgs/msys2/win-64
• https://repo.anaconda.com/pkgs/msys2/noarch
  any suggestion?

[TobyBaril]

Hi, from the output this looks like you are trying to install on windows. Earl Grey is currently only available for Linux and MacOS architectures. If you are using windows, I would recommend configuring the windows subsystem for Linux, and then installing Earl grey via the terminal for this.

[robaina62]

Many thanks, Toby, I will proceed so as I misunderstood the information somewhere pointing that earlgrey was also compatible with windows arch. and could be run with anaconda/bioconda. After careful review of the information at your workspace I realised about.
Sorry to bother you with this simply technical issue.
my best
Rafael

[robaina62]

Many thanks, Toby, I will proceed so as I misunderstood the information somewhere pointing that earlgrey was also compatible with windows arch. and could be run with anaconda/bioconda. After careful review of the information at your workspace I realised about. I am know building a linux terminal to run earlgrey on it, let´s see what`s going on. Sorry to bother you with this rather simply technical issue. my best Rafael [cid:9951cc39-f053-4be4-8a64-b7a213c53f71] orcid 0000-0003-4265-5809 Rafael R Robaina (researchgate.net)<https://www.researchgate.net/profile/Rafael-Robaina>

…

________________________________ De: Tobias Baril ***@***.***> Enviado: miércoles, 21 de febrero de 2024 7:12 Para: TobyBaril/EarlGrey ***@***.***> Cc: Rafael Juan Robaina Romero ***@***.***>; Comment ***@***.***> Asunto: Re: [TobyBaril/EarlGrey] Earl Grey (Discussion #2) Hi, from the output this looks like you are trying to install on windows. Earl Grey is currently only available for Linux and MacOS architectures. If you are using windows, I would recommend configuring the windows subsystem for Linux, and then installing Earl grey via the terminal for this. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/TobyBaril/EarlGrey/discussions/2*discussioncomment-8539562__;Iw!!D9dNQwwGXtA!VuZmckNN1RHf7jvQO77xalIcFEpPdUUe7J9bk-GyqVCa5ifufG9OhV6BLvKL1Im07_1uazETm0hwZAfXOW9VVbFerC5cDg$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/BEMMKA4O46ZYSXGTMOEEFC3YUWM5FAVCNFSM5HSZLPA2U5DIOJSWCZC7NNSXTOKENFZWG5LTONUW63SDN5WW2ZLOOQ5TQNJTHE2TMMQ__;!!D9dNQwwGXtA!VuZmckNN1RHf7jvQO77xalIcFEpPdUUe7J9bk-GyqVCa5ifufG9OhV6BLvKL1Im07_1uazETm0hwZAfXOW9VVbHL6wborQ$>. You are receiving this because you commented.

[TobyBaril]

Absolutely no problem! Let me know if you need any further assistance with the configuration once you have a linux environment running!

Best wishes,

Toby