HDF5Array is not available for package glmGamPoi

[nlubock]

Hey there!

I'm trying to install this package in a rocker/tidyverse container, but I'm running into this issue:

ERROR: dependency ‘HDF5Array’ is not available for package ‘glmGamPoi’
* removing ‘/usr/local/lib/R/site-library/glmGamPoi’

The downloaded source packages are in
        ‘/tmp/RtmpzPbMXu/downloaded_packages’
Old packages: 'curl', 'dbplyr', 'evaluate', 'gert', 'lubridate', 'tinytex',
  'vroom', 'withr', 'foreign', 'KernSmooth', 'lattice', 'Matrix', 'mgcv',
  'nlme', 'spatial', 'survival'
Update all/some/none? [a/s/n]: n
Warning messages:
1: In install.packages(...) :
  installation of package ‘Rhdf5lib’ had non-zero exit status
2: In install.packages(...) :
  installation of package ‘rhdf5filters’ had non-zero exit status
3: In install.packages(...) :
  installation of package ‘rhdf5’ had non-zero exit status
4: In install.packages(...) :
  installation of package ‘HDF5Array’ had non-zero exit status
5: In install.packages(...) :
  installation of package ‘glmGamPoi’ had non-zero exit status
> install.package('ERROR: dependency ‘HDF5Array’ is not available for package ‘glmGamPoi’z
> install.packages('HDF5Array')
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
Warning message:
package ‘HDF5Array’ is not available for this version of R

A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages

I also tried installing HDF5Array before installing glmGamPoi and ran into:

* installing *source* package ‘Rhdf5lib’ ...
** using staged installation
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether the compiler supports GNU C++... yes
checking whether g++ -std=gnu++17 accepts -g... yes
checking for g++ -std=gnu++17 option to enable C++11 features... none needed
COMPILER = gcc
checking whether C compiler accepts -w... yes
checking whether to use -Wl,-rpath setting... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for zlib.h... yes
checking for szlib.h... no
checking for libaec.h... no
checking for curl/curl.h... yes
checking for openssl/evp.h... yes
checking for openssl/hmac.h... yes
checking for openssl/sha.h... yes
checking for curl_global_init in -lcurl... yes
checking for EVP_sha256 in -lcrypto... yes
S3_VFD=--enable-ros3-vfd
configure: creating ./config.status
config.status: creating src/Makevars
** libs
using C compiler: ‘gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
gunzip -dc hdf5small_cxx_hl_1.10.7.tar.gz | tar xf -;
cd hdf5/libaec-1.0.4/build; \
../configure --with-pic --enable-shared=no --prefix=/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src/hdf5/libaec-1.0.4/build/szip --libdir=/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src/hdf5/libaec-1.0.4/build/szip/lib \
CC='gcc' CXX='g++ -std=gnu++17' \
CPPFLAGS='-I/usr/local/include' \
CFLAGS='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -w -fpic' \
AR='ar' RANLIB='ranlib'; \
make; \
make install
/bin/bash: line 1: cd: hdf5/libaec-1.0.4/build: Operation not supported
configure: WARNING: unrecognized options: --with-pic, --enable-shared
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether the compiler supports GNU C++... yes
checking whether g++ -std=gnu++17 accepts -g... yes
checking for g++ -std=gnu++17 option to enable C++11 features... none needed
COMPILER = gcc
checking whether C compiler accepts -w... yes
checking whether to use -Wl,-rpath setting... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for zlib.h... yes
checking for szlib.h... no
checking for libaec.h... no
checking for curl/curl.h... yes
checking for openssl/evp.h... yes
checking for openssl/hmac.h... yes
checking for openssl/sha.h... yes
checking for curl_global_init in -lcurl... yes
checking for EVP_sha256 in -lcrypto... yes
S3_VFD=--enable-ros3-vfd
configure: creating ./config.status
config.status: creating src/Makevars
configure: WARNING: unrecognized options: --with-pic, --enable-shared
make[1]: Entering directory '/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src'
make[1]: *** No targets specified and no makefile found.  Stop.
make[1]: Leaving directory '/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src'
make[1]: Entering directory '/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src'
make[1]: *** No rule to make target 'install'.  Stop.
make[1]: Leaving directory '/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src'
make: *** [Makevars:65: build-szip] Error 2
ERROR: compilation failed for package ‘Rhdf5lib’
* removing ‘/usr/local/lib/R/site-library/Rhdf5lib’
rm: cannot remove '/tmp/Rtmp90LmfB/R.INSTALL277b6080340d/Rhdf5lib/src/hdf5': Directory not empty
ERROR: dependency ‘Rhdf5lib’ is not available for package ‘rhdf5filters’
* removing ‘/usr/local/lib/R/site-library/rhdf5filters’
ERROR: dependencies ‘Rhdf5lib’, ‘rhdf5filters’ are not available for package ‘rhdf5’
* removing ‘/usr/local/lib/R/site-library/rhdf5’
ERROR: dependencies ‘rhdf5’, ‘rhdf5filters’, ‘Rhdf5lib’ are not available for package ‘HDF5Array’
* removing ‘/usr/local/lib/R/site-library/HDF5Array’

Here's the relevant sessionInfo()

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] BiocManager_1.30.22 compiler_4.3.1      tools_4.3.1

Any help would be appreciated! I can install no problem on an Apple Silicon machine with the following sessionInfo()

sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.4.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] BiocManager_1.30.22 compiler_4.3.1      fastmap_1.1.1       cli_3.6.1           htmltools_0.5.6     tools_4.3.1        
 [7] rstudioapi_0.15.0   yaml_2.3.7          rmarkdown_2.24      knitr_1.44          digest_0.6.33       xfun_0.40          
[13] rlang_1.1.1         evaluate_0.21  

[const-ae]

Hi nlubock,

thanks for the detailed report. The cause of the error seems to be that Rhdf5lib cannot be installed.

Maybe the maintainers of Rhdf5lib and HDF5Array (@grimbough, @hpages) have an idea what causes the issue.

Best,
Constantin

[hpages]

Seems to be related to lack of AEC library on your container @nlubock. Note that you should be able to fix that with sudo apt install libaec-dev.

Even better: I highly recommend that you switch to the Bioconductor's Docker image. Then you will be able to install pre-compiled binaries of any Bioconductor or CRAN package (using BiocManager::install(), like usual). Installation will be smooth and fast! In particular it will avoid the hassle of having to install missing system-level components in order to compile things.

That being said, it could be argued that lack of AEC library should not break Rhdf5lib's configure script. Any thought on that @grimbough? Thanks.

Best,
H.

[grimbough]

Rhdf5lib comes bundled with libaec. It should use a system version if it's available, but otherwise the bundled copy will be compiled and linked against.

This error seems to occur at the first step of that compilation effort:

/bin/bash: line 1: cd: hdf5/libaec-1.0.4/build: Operation not supported

I'm not sure exactly what operation isn't permitted. Maybe something odd is happening with user permissions relating the using Docker, but I have successfully installed Rhdf5lib inside a Bioconductor Docker container many times without issue.

@nlubock are you carrying out the installation inside the Dockerfile? If so, could you share it so I can test locally?

[nlubock]

OK so I was able to build the following Dockerfile

FROM rocker/tidyverse

RUN install2.r BiocManager
RUN  R -e 'BiocManager::install("glmGamPoi")'

The trouble comes from trying to do this from within the container. Specifically when Podman is orchestrating things... The following produces the error:

> podman run --rm -ti docker.io/rocker/tidyverse R
> install.packages('BiocManager')
> BiocManager::install('glmGamPoi')

If I run the same commands with Docker, everything works fine. I think you may be on to something @grimbough with the permissions issues...

Thanks everyone for their help!