Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install augustus from source without root privileges #358

Open
dariober opened this issue Aug 20, 2022 · 14 comments
Open

Install augustus from source without root privileges #358

dariober opened this issue Aug 20, 2022 · 14 comments

Comments

@dariober
Copy link

dariober commented Aug 20, 2022

Since installing augustus from source without root previleges wasn't easy for me, I thought posting here the commands I used for installation.

  • If not already done, install conda, bioconda, and, optional but recommended, mamba (these are useful anyway).

  • Create and activate a dedicated environment for augustus (or activate an existing one of your choice):

conda create --yes --name augustus
conda activate augustus
  • Write the list of dependencies to file requirements.txt and install them:
echo "zlib 
boost-cpp 
gsl 
htslib
lp_solve 
biopython 
perl 
perl-app-cpanminus 
perl-module-build 
perl-yaml 
perl-dbi 
perl-scalar-list-utils 
perl-file-which 
perl-parallel-forkmanager 
sqlite 
suitesparse 
cdbtools 
diamond 
ucsc-fatotwobit 
ucsc-twobitinfo 
bamtools 
tar" > requirements.txt

mamba install --yes --file requirements.txt  # or `conda install`
  • Get the latest source code of augustus:
git clone https://github.com/Gaius-Augustus/Augustus.git
cd Augustus
  • Edit common.mk with the correct settings:
git checkout common.mk # Reset to original before editing it
> common.tmp
echo "
INCLUDE_PATH_ZLIB        := -I${CONDA_PREFIX}/include
INCLUDE_PATH_BOOST       := -I${CONDA_PREFIX}/include
INCLUDE_PATH_LPSOLVE     := -I${CONDA_PREFIX}/include/lpsolve
INCLUDE_PATH_SUITESPARSE := -I${CONDA_PREFIX}/include/suitesparse/
INCLUDE_PATH_GSL         := -I${CONDA_PREFIX}/include
INCLUDE_PATH_SQLITE      := -I${CONDA_PREFIX}/include
INCLUDE_PATH_BAMTOOLS    := -I${CONDA_PREFIX}/include/bamtools
INCLUDE_PATH_HTSLIB      := -I${CONDA_PREFIX}/include/htslib
INCLUDE_PATH_SEQLIB      := -I ${CONDA_PREFIX}/include/SeqLib -I${CONDA_PREFIX}/include/htslib -I${CONDA_PREFIX}/include/jsoncpp
LIBRARY_PATH_LPSOLVE     := -L${CONDA_PREFIX}/lib/lp_solve/ -Wl,-rpath,${CONDA_PREFIX}/lib/lp_solve/

LIBRARY_PATH_ZLIB        := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_BOOST       := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_SUITESPARSE := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_GSL         := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_SQLITE      := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_BAMTOOLS    := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_HTSLIB      := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
LIBRARY_PATH_SEQLIB      := -L${CONDA_PREFIX}/lib -Wl,-rpath,${CONDA_PREFIX}/lib
" >> common.tmp
cat common.mk >> common.tmp
mv common.tmp common.mk
  • Compile augustus and the auxiliary programs:
make MYSQL=false augustus auxprogs
echo $? # Check 0 exit code
  • Copy the executables, scripts, and configs to the root directory of the conda environment:
mv bin/* ${CONDA_PREFIX}/bin/
cp -r scripts/* ${CONDA_PREFIX}/bin/
mkdir -p ${CONDA_PREFIX}/config
cp -r config/* ${CONDA_PREFIX}/config/

# Some basic tests:
augustus --version
autoAug.pl --help
  • Before using augustus set the relevant variables:
export AUGUSTUS_CONFIG_PATH=${CONDA_PREFIX}/config/
export AUGUSTUS_SCRIPTS_PATH=${CONDA_PREFIX}/bin/
export AUGUSTUS_BIN_PATH=${CONDA_PREFIX}/bin/

Tested (not thoroughly) on Ubuntu 18 and CentOS 7 with augustus commit d1fd097. Note that augustus is also on bioconda but the version there may be outdated. Any feedback most welcome.

@jwasmuth
Copy link

jwasmuth commented Sep 7, 2022

Thank you! This worked for me and saved me hours of pain. If our paths ever cross, I owe you a beer.

@AnneJRomero
Copy link

Hi, thanks for this, super useful! :)

I followed all your instructions but I got an error on:
$ augustus --version
augustus: error while loading shared libraries: libboost_iostreams.so.1.71.0: cannot open shared object file: No such file or directory

I do not have libboost_iostreams.so.1.71.0, I have these instead:
$ ls libboost_iostreams.*
libboost_iostreams.a libboost_iostreams.so.1.73
libboost_iostreams.so libboost_iostreams.so.1.73.0
libboost_iostreams.so.1

any advice on how to fix this?

@KatharinaHoff
Copy link
Member

KatharinaHoff commented Sep 8, 2022 via email

@AnneJRomero
Copy link

You could try a softlink with the required file name. However, it might produce a segmentation fault. But it’s worth a try. AnneJRomero @.> schrieb am Do. 8. Sept. 2022 um 10:48:

Hi, thanks for this, super useful! :) I followed all your instructions but I got an error on: $ augustus --version augustus: error while loading shared libraries: libboost_iostreams.so.1.71.0: cannot open shared object file: No such file or directory I do not have libboost_iostreams.so.1.71.0, I have these instead: $ ls libboost_iostreams.
libboost_iostreams.a libboost_iostreams.so.1.73 libboost_iostreams.so libboost_iostreams.so.1.73.0 libboost_iostreams.so.1 any advice on how to fix this? — Reply to this email directly, view it on GitHub <#358 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JG4NDSH4YJC3VAP4NTV5GR47ANCNFSM57DTKYAA . You are receiving this because you are subscribed to this thread.Message ID: @.
*>

Hi, I tried that and I'm still getting the same error message.

@KatharinaHoff
Copy link
Member

KatharinaHoff commented Sep 8, 2022 via email

@AnneJRomero
Copy link

Hi, I'm retrying again.

For the step, 'Compile augustus and the auxiliary programs':

make MYSQL=false augustus auxprogs
echo $? # Check 0 exit code

I get the error:

mkdir -p bin
cd src && make
make[1]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/src' echo "-Wall -Wno-sign-compare -pedantic -O3 -std=c++11 -DZIPINPUT -DCOMPGENEPRED -DTESTING -DM_SQLITE" > cxxflags make[1]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/src'
mkdir -p bin
cd auxprogs && make
make[1]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs' cd aln2wig; make; make[2]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/aln2wig'
make[2]: aln2wig' is up to date. make[2]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/aln2wig'
cd bam2hints; make;
make[2]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/bam2hints' g++ -Wall -O3 -L/scratch/ar14g12/.conda/envs/augustus/lib -Wl,-rpath,/scratch/ar14g12/.conda/envs/augustus/lib -L/scratch/ar14g12/.conda/envs/augustus/lib -Wl,-rpath,/scratch/ar14g12/.conda/envs/augustus/lib -I/scratch/ar14g12/.conda/envs/augustus/include/bamtools -I/scratch/ar14g12/.conda/envs/augustus/include -o bam2hints bam2hints.o -lbamtools -lz bam2hints.o: In function main':
bam2hints.cc:(.text.startup+0x73e): undefined reference to BamTools::BamReader::Open(std::string const&)' collect2: error: ld returned 1 exit status make[2]: *** [bam2hints] Error 1 make[2]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/bam2hints'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs'
make: *** [auxprogs] Error 2
2

What am I doing wrong?

@dariober
Copy link
Author

dariober commented Sep 8, 2022

@AnneJRomero Can you start from a fresh conda environment, show the commands you execute with their output?

If it helps, on my setup libboost_iostreams.so is a symlink to libboost_iostreams.so.1.80.0:

conda activate augustus
ls -l ${CONDA_PREFIX}/lib/libboost_iostreams.so

lrwxrwxrwx 1 db291g db291g 28 Aug 20 18:42 /export/projects/III-data/wcmp_bioinformatics/db291g/miniconda3/envs/augustus/lib/libboost_iostreams.so -> libboost_iostreams.so.1.80.0

and this is my conda environment:

conda list
# packages in environment at /export/projects/III-data/wcmp_bioinformatics/db291g/miniconda3/envs/augustus:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bamtools                  2.5.2                hd03093a_0    bioconda
biopython                 1.79            py310h5764c6d_2    conda-forge
boost-cpp                 1.80.0               h75c5d50_0    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.6.15            ha878542_0    conda-forge
cdbtools                  0.99                 hd03093a_7    bioconda
diamond                   0.9.19               h2e03b76_5    bioconda
gmp                       6.2.1                h58526e2_0    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
htslib                    1.15.1               h9753748_0    bioconda
icu                       70.1                 h27087fc_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.3               h3790be6_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcurl                   7.83.1               h7bff187_0    conda-forge
libdeflate                1.10                 h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libnghttp2                1.47.0               hdcd2b5c_1    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_1    conda-forge
libpng                    1.6.37               h753d276_4    conda-forge
libsqlite                 3.39.2               h753d276_1    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.12               h166bdaf_2    conda-forge
lp_solve                  5.5.2.5           h14c3975_1001    conda-forge
metis                     5.1.0             h58526e2_1006    conda-forge
mpfr                      4.1.0                h9202a9a_1    conda-forge
mysql-connector-c         6.1.11            h6eb9d5d_1007    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
numpy                     1.23.2          py310h53a5b5f_0    conda-forge
openssl                   1.1.1q               h166bdaf_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
perl-apache-test          1.43            pl5321hd8ed1ab_0    conda-forge
perl-app-cpanminus        1.7046          pl5321hd8ed1ab_0    conda-forge
perl-base                 2.23            pl5321hdfd78af_2    bioconda
perl-carp                 1.50            pl5321hd8ed1ab_0    conda-forge
perl-class-load           0.25            pl5321hdfd78af_1    bioconda
perl-class-load-xs        0.10            pl5321h9f5acd7_6    bioconda
perl-class-method-modifiers 2.13            pl5321hdfd78af_0    bioconda
perl-constant             1.33            pl5321hd8ed1ab_0    conda-forge
perl-cpan-meta            2.150010        pl5321hdfd78af_1    bioconda
perl-cpan-meta-requirements 2.140           pl5321hdfd78af_1    bioconda
perl-cpan-meta-yaml       0.018           pl5321hdfd78af_1    bioconda
perl-data-dumper          2.183           pl5321h166bdaf_0    conda-forge
perl-data-optlist         0.112           pl5321hdfd78af_0    bioconda
perl-dbi                  1.643           pl5321hec16e2b_1    bioconda
perl-devel-globaldestruction 0.14            pl5321hdfd78af_1    bioconda
perl-devel-overloadinfo   0.007           pl5321hdfd78af_0    bioconda
perl-devel-stacktrace     2.04            pl5321hdfd78af_1    bioconda
perl-dist-checkconflicts  0.11            pl5321hdfd78af_3    bioconda
perl-encode               3.19            pl5321hec16e2b_1    bioconda
perl-eval-closure         0.14            pl5321h9f5acd7_6    bioconda
perl-exporter             5.74            pl5321hd8ed1ab_0    conda-forge
perl-extutils-cbuilder    0.280230        pl5321hdfd78af_2    bioconda
perl-extutils-makemaker   7.64            pl5321hd8ed1ab_0    conda-forge
perl-extutils-manifest    1.73            pl5321hdfd78af_0    bioconda
perl-extutils-parsexs     3.44            pl5321hdfd78af_0    bioconda
perl-file-path            2.18            pl5321hd8ed1ab_0    conda-forge
perl-file-temp            0.2304          pl5321hd8ed1ab_0    conda-forge
perl-file-which           1.24            pl5321hd8ed1ab_0    conda-forge
perl-getopt-long          2.52            pl5321hdfd78af_0    bioconda
perl-ipc-cmd              1.04            pl5321hdfd78af_0    bioconda
perl-json-pp              4.11            pl5321hd8ed1ab_0    conda-forge
perl-locale-maketext-simple 0.21            pl5321hdfd78af_3    bioconda
perl-module-build         0.4231          pl5321hdfd78af_0    bioconda
perl-module-corelist      5.20220620      pl5321hdfd78af_0    bioconda
perl-module-implementation 0.09            pl5321hdfd78af_3    bioconda
perl-module-load          0.34            pl5321hdfd78af_0    bioconda
perl-module-load-conditional 0.68            pl5321hdfd78af_3    bioconda
perl-module-metadata      1.000037        pl5321hdfd78af_0    bioconda
perl-module-runtime       0.016           pl5321hdfd78af_2    bioconda
perl-module-runtime-conflicts 0.003           pl5321hdfd78af_1    bioconda
perl-moo                  2.005004        pl5321hdfd78af_0    bioconda
perl-moose                2.2201          pl5321hec16e2b_2    bioconda
perl-mro-compat           0.15            pl5321hdfd78af_0    bioconda
perl-package-deprecationmanager 0.17            pl5321hdfd78af_1    bioconda
perl-package-stash        0.40            pl5321h87f3376_1    bioconda
perl-package-stash-xs     0.29            pl5321h87f3376_1    bioconda
perl-parallel-forkmanager 2.02            pl5321hdfd78af_1    bioconda
perl-params-check         0.38            pl5321hdfd78af_2    bioconda
perl-params-util          1.102           pl5321h9f5acd7_1    bioconda
perl-parent               0.238           pl5321hd8ed1ab_0    conda-forge
perl-pathtools            3.75            pl5321h166bdaf_0    conda-forge
perl-perl-ostype          1.010           pl5321hd8ed1ab_0    conda-forge
perl-role-tiny            2.002004        pl5321hdfd78af_0    bioconda
perl-scalar-list-utils    1.63            pl5321h166bdaf_0    conda-forge
perl-storable             3.15            pl5321h166bdaf_0    conda-forge
perl-sub-exporter         0.988           pl5321hdfd78af_0    bioconda
perl-sub-exporter-progressive 0.001013        pl5321hdfd78af_1    bioconda
perl-sub-identify         0.14            pl5321hec16e2b_2    bioconda
perl-sub-install          0.928           pl5321hdfd78af_3    bioconda
perl-sub-name             0.21            pl5321hec16e2b_3    bioconda
perl-sub-quote            2.006006        pl5321hdfd78af_0    bioconda
perl-text-abbrev          1.02            pl5321hdfd78af_1    bioconda
perl-text-parsewords      3.31            pl5321hdfd78af_0    bioconda
perl-try-tiny             0.31            pl5321ha770c72_0    conda-forge
perl-version              0.9929          pl5321h166bdaf_0    conda-forge
perl-xsloader             0.24            pl5321hd8ed1ab_0    conda-forge
perl-yaml                 1.30            pl5321hdfd78af_0    bioconda
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
python                    3.10.5          h582c2e5_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
setuptools                65.1.0          py310hff52083_0    conda-forge
sqlite                    3.39.2               h4ff8645_1    conda-forge
suitesparse               5.10.1               h9e50725_1    conda-forge
tar                       1.34                 ha1f6473_0    conda-forge
tbb                       2021.5.0             h924138e_1    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2022c                h191b570_0    conda-forge
ucsc-fatotwobit           377                  ha8a8165_5    bioconda
ucsc-twobitinfo           377                  ha8a8165_3    bioconda
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.12               h166bdaf_2    conda-forge
zstd                      1.5.2                h6239696_4    conda-forge

@AnneJRomero
Copy link

Hi,

I copied your list of packages, installed ones that I didn't have and updated the packages that were outdated.

I'm not getting the error message anymore. :
$ augustus --version
AUGUSTUS (3.4.0) is a gene prediction tool.
Sources and documentation at https://github.com/Gaius-Augustus/Augustus

Thank you! :)

I tried testing the installation with:
$ augustus --species=human --UTR=on examples/example.fa
Segmentation fault (core dumped)

Why am I getting 'Segmentation fault (core dumped)' error?

@Ingway
Copy link

Ingway commented Nov 4, 2022

I had the same problem as AnneJRomero. For some reason I couldn't install the same package versions as listed by dario using conda.
For me, the only way to make a compiled augustus and auxprogs to work was to follow the install.md instructions (install every dependencies manually and point them in common.mk). Mysql++-dev without root it seens nearly impossible to do it, so use MYSQL=false argument as pointed by Dario. It surprised me that such a popular predictor is this hard to install... I really appreciate Dario's initiative to make it easier.

@KatharinaHoff
Copy link
Member

We have several docker containers that have Augustus with all dependencies. For example, it's in the GALBA container (https://github.com/Gaius-Augustus/GALBA). These containers can be executed either with Docker, or Singularity (the latter for non-root scenarios).

@cfz1998
Copy link

cfz1998 commented Feb 18, 2023

I had the same problem as AnneJRomero. ( ´•̥̥̥ω•̥̥̥` )

@jkl826
Copy link

jkl826 commented Mar 14, 2024

g++ -DZIPINPUT -DCOMPGENEPRED -DTESTING -DM_MYSQL -DM_SQLITE -Wall -Wno-sign-compare -pedantic -O3 -std=c++11 -I../include -I/usr/include/lpsolve -I/usr/include/mysql -c -o genbank.o genbank.cc
In file included from ../include/genbank.hh:12:0,
from genbank.cc:10:
../include/types.hh:16:43: fatal error: boost/archive/text_oarchive.hpp: No such file or directory
#include <boost/archive/text_oarchive.hpp>
^
compilation terminated.
make[1]: *** [genbank.o] Error 1
make[1]: Leaving directory `/home/u041175/workspace/jiakeli/rnaseq/software/new/Augustus-3.5.0/src'
make: *** [augustus] Error 2
I need help

@KatharinaHoff
Copy link
Member

KatharinaHoff commented Mar 14, 2024 via email

@QianChwnLyn
Copy link

Hi, I'm retrying again.

For the step, 'Compile augustus and the auxiliary programs':

make MYSQL=false augustus auxprogs echo $? # Check 0 exit code

I get the error:

mkdir -p bin cd src && make make[1]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/src' echo "-Wall -Wno-sign-compare -pedantic -O3 -std=c++11 -DZIPINPUT -DCOMPGENEPRED -DTESTING -DM_SQLITE" > cxxflags make[1]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/src' mkdir -p bin cd auxprogs && make make[1]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs' cd aln2wig; make; make[2]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/aln2wig' make[2]: aln2wig' is up to date. make[2]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/aln2wig' cd bam2hints; make; make[2]: Entering directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/bam2hints' g++ -Wall -O3 -L/scratch/ar14g12/.conda/envs/augustus/lib -Wl,-rpath,/scratch/ar14g12/.conda/envs/augustus/lib -L/scratch/ar14g12/.conda/envs/augustus/lib -Wl,-rpath,/scratch/ar14g12/.conda/envs/augustus/lib -I/scratch/ar14g12/.conda/envs/augustus/include/bamtools -I/scratch/ar14g12/.conda/envs/augustus/include -o bam2hints bam2hints.o -lbamtools -lz bam2hints.o: In function main': bam2hints.cc:(.text.startup+0x73e): undefined reference to BamTools::BamReader::Open(std::string const&)' collect2: error: ld returned 1 exit status make[2]: *** [bam2hints] Error 1 make[2]: Leaving directory /mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs/bam2hints' make[1]: *** [all] Error 2 make[1]: Leaving directory `/mainfs/scratch/ar14g12/PIPS/annotation/genes/Augustus/auxprogs' make: *** [auxprogs] Error 2 2

What am I doing wrong?

Maybe you have solved this problem? I met the same one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants