Memory not mapped error when working on DECIPHER and phangorn packages

[alptaciroglu]

Hello,

I have 1000 sequences which I align them with DECIPHER packages AlignSeqs() function.
Then I try to convert it phyDat object using phyDat() function and run a ML based method to create a phylogenetic tree. Code is attached below. I think the error is with convertion of DECIPHER::AlignSeqs() output which is AAStringSet to phyDat object.

 fastaDecipherAligned <- DECIPHER::AlignSeqs(fastaRefined62)
 save(fastaDecipherAligned, file = 'fastaDecipherAligned.RData')
 fastaphyDat <- phyDat(as(fastaDecipherAligned, 'matrix'), type = 'AA')  ## Guessing this is not working
 rm(fastaDecipherAligned)
 names(fastaphyDat) <- homologySummary62[1:length(fastaphyDat),]$SequenceFullName

dm <- dist.ml(fastaphyDat, model="JTT")
tree <- NJ(dm)
## cat('Testing all models ...\n')
## (mt <- modelTest(fastaphyDat, model="all", multicore=multiCoreBool))
## env <- attr(mt, "env")
## fitStart <- eval(get(mt$Model[which.min(mt$BIC)], env), env)
## fitNJ <- pml(tree, fastaphyDat, model=fitStart$model, k=4, inv=.2)
fitNJ <- pml(tree, fastaphyDat, model = 'JTT', k=4, inv=.2)
fit <- optim.pml(fitNJ, rearrangement = "stochastic",
                     optInv=TRUE, optGamma=TRUE)
fit
## Code truncated and also commented out modelTest as it also returned memory not mapped error

The code is run on the server and the following is the complete output.

nohup: ignoring input
Loading required package: methods
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, parApply, parCapply, parLapply,
    parLapplyLB, parRapply, parSapply, parSapplyLB

The following objects are masked from ‘package:stats’:

    IQR, mad, sd, var, xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, cbind, colMeans, colnames,
    colSums, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
    grepl, intersect, is.unsorted, lapply, lengths, Map, mapply, match,
    mget, order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
    rbind, Reduce, rowMeans, rownames, rowSums, sapply, setdiff, sort,
    table, tapply, union, unique, unsplit, which, which.max, which.min

Loading required package: S4Vectors
Loading required package: stats4

Attaching package: ‘S4Vectors’

The following object is masked from ‘package:base’:

    expand.grid

Loading required package: IRanges
Loading required package: XVector

Attaching package: ‘Biostrings’

The following object is masked from ‘package:base’:

    strsplit


Attaching package: ‘seqinr’

The following object is masked from ‘package:Biostrings’:

    translate


Attaching package: ‘ape’

The following objects are masked from ‘package:seqinr’:

    as.alignment, consensus

The following object is masked from ‘package:Biostrings’:

    complement


Attaching package: ‘rphast’

The following object is masked from ‘package:ape’:

    complement

The following object is masked from ‘package:msa’:

    msa

The following object is masked from ‘package:Biostrings’:

    complement

Loading required package: RSQLite
Loading required package: ggplot2
Loading required package: treeio

Attaching package: ‘treeio’

The following object is masked from ‘package:phangorn’:

    getRoot

The following objects are masked from ‘package:ape’:

    drop.tip, Nnode, Ntip

The following object is masked from ‘package:Biostrings’:

    mask

ggtree v1.10.5  For help: https://guangchuangyu.github.io/ggtree

If you use ggtree in published research, please cite:
Guangchuang Yu, David Smith, Huachen Zhu, Yi Guan, Tommy Tsan-Yuk Lam. ggtree: an R package for visualization and annotation of phylogenetic trees with their covariates and other associated data. Methods in Ecology and Evolution 2017, 8(1):28-36, doi:10.1111/2041-210X.12628

Attaching package: ‘ggtree’

The following object is masked from ‘package:ape’:

    rotate

The following object is masked from ‘package:Biostrings’:

    collapse

The following object is masked from ‘package:IRanges’:

    collapse

The following object is masked from ‘package:S4Vectors’:

    expand

negative edges length changed to 0!

 *** caught segfault ***
address 0x2616e4c0, cause 'memory not mapped'

Traceback:
 1: .Call("PML0", dlist = data, el, as.double(g), nr, nc, k, eig,     as.double(bf), node, edge, nTips, nco, contrast, N = as.integer(length(edge)))
 2: pml.fit(tree, data, bf, shape = shape, k = k, Q = Q, levels = attr(data,     "levels"), inv = inv, rate = rate, g = g, w = w, eig = eig,     INV = INV, ll.0 = ll.0, llMix = llMix, wMix = wMix, site = TRUE)
 3: pml(tree, fastaphyDat, model = "JTT", k = 4, inv = 0.2)
 4: phylogeneticTreeCreate(homologySummary = homologyResults, uniprot.db = "uniprot_sprot.fasta",     numberOfSeq = 1000, multiCoreBool = TRUE, useExistingAlignment = TRUE,     alignmentType = "DECIPHER")
An irrecoverable exception occurred. R is aborting now ...
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            [1] "mtArt+G"

 *** caught segfault ***
address 0x1005c5ac0, cause 'memory not mapped'

Traceback:
 1: pml.fit(tree, data, bf, shape = shape, k = k, Q = Q, levels = attr(data,     "levels"), inv = inv, rate = rate, g = g, w = w, eig = eig,     INV = INV, ll.0 = ll.0, llMix = llMix, wMix = wMix, site = TRUE)
 2: update.pml(fittmp, k = k)
 3: update(fittmp, k = k)
 4: FUN(X[[i]], ...)
 5: lapply(X = S, FUN = FUN, ...)
 6: doTryCatch(return(expr), name, parentenv, handler)
 7: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 8: tryCatchList(expr, classes, parentenv, handlers)
 9: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        msg <- conditionMessage(e)        sm <- strsplit(msg, "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && identical(getOption("show.error.messages"),         TRUE)) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
10: try(lapply(X = S, FUN = FUN, ...), silent = TRUE)
11: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE))
12: FUN(X[[i]], ...)
13: lapply(seq_len(cores), inner.do)
14: mclapply(model, fitPar, fit, G, I, k, FREQ, mc.cores = mc.cores)
15: modelTest(fastaphyDat, model = "all", multicore = multiCoreBool)
16: phylogeneticTreeCreate(homologySummary = homologyResults, uniprot.db = "uniprot_sprot.fasta",     numberOfSeq = 1000, multiCoreBool = TRUE, useExistingAlignment = TRUE,     alignmentType = "DECIPHER")
An irrecoverable exception occurred. R is aborting now ...
[1] "HIVw+G"

 *** caught segfault ***
address 0x1005c5ac0, cause 'memory not mapped'

Traceback:
 1: pml.fit(tree, data, bf, shape = shape, k = k, Q = Q, levels = attr(data,     "levels"), inv = inv, rate = rate, g = g, w = w, eig = eig,     INV = INV, ll.0 = ll.0, llMix = llMix, wMix = wMix, site = TRUE)
 2: update.pml(fittmp, k = k)
 3: update(fittmp, k = k)
 4: FUN(X[[i]], ...)
 5: lapply(X = S, FUN = FUN, ...)
 6: doTryCatch(return(expr), name, parentenv, handler)
 7: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 8: tryCatchList(expr, classes, parentenv, handlers)
 9: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        msg <- conditionMessage(e)        sm <- strsplit(msg, "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && identical(getOption("show.error.messages"),         TRUE)) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
10: try(lapply(X = S, FUN = FUN, ...), silent = TRUE)
11: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE))
12: FUN(X[[i]], ...)
13: lapply(seq_len(cores), inner.do)
14: mclapply(model, fitPar, fit, G, I, k, FREQ, mc.cores = mc.cores)
15: modelTest(fastaphyDat, model = "all", multicore = multiCoreBool)
16: phylogeneticTreeCreate(homologySummary = homologyResults, uniprot.db = "uniprot_sprot.fasta",     numberOfSeq = 1000, multiCoreBool = TRUE, useExistingAlignment = TRUE,     alignmentType = "DECIPHER")
An irrecoverable exception occurred. R is aborting now ...
Error in x[[jj]][iseq] <- vjj : replacement has length zero
Calls: phylogeneticTreeCreate -> modelTest -> [<- -> [<-.data.frame
Execution halted

sessionInfo

> sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
 [1] ggtree_1.10.5       treeio_1.2.2        ggplot2_3.2.0
 [4] phangorn_2.5.5      ape_5.3             DECIPHER_2.6.0
 [7] RSQLite_2.1.1       msa_1.10.0          Biostrings_2.46.0
[10] XVector_0.18.0      IRanges_2.12.0      S4Vectors_0.16.0
[13] BiocGenerics_0.24.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1       pillar_1.4.1     compiler_3.4.4   tools_3.4.4
 [5] zlibbioc_1.24.0  digest_0.6.19    bit_1.1-14       jsonlite_1.6
 [9] memoise_1.1.0    tibble_2.1.3     nlme_3.1-131     gtable_0.3.0
[13] lattice_0.20-35  pkgconfig_2.0.2  rlang_0.3.4      Matrix_1.2-12
[17] fastmatch_1.1-0  igraph_1.2.4.1   DBI_1.0.0        rvcheck_0.1.3
[21] withr_2.1.2      dplyr_0.8.1      tidyselect_0.2.5 bit64_0.9-7
[25] grid_3.4.4       glue_1.3.1       R6_2.4.0         tidyr_0.8.3
[29] purrr_0.3.2      blob_1.1.1       magrittr_1.5     scales_1.0.0
[33] assertthat_0.2.1 colorspace_1.4-1 quadprog_1.5-7   lazyeval_0.2.2
[37] munsell_0.5.0    crayon_1.3.4

[KlausVigo]

Hi @alptaciroglu,
I can't really make a call what is going wrong without the data. Can you send me the data or provide me with a link where I can download them.
Regards,
Klaus

[alptaciroglu]

Hello @KlausVigo,

Thank you so much for your email. I am adding a link for the data for download in .RData format. First code chunk in OP is how I tried to proceed with. (Out of topic note: I could use other packages but somehow only decipher package seemed to align my sequences)

https://we.tl/t-E4FNrOnHez

I also tried the alter the code like following

 fastaDecipherAligned <- DECIPHER::AlignSeqs(fastaRefined62)
 save(fastaDecipherAligned, file = 'fastaDecipherAligned.RData')
 fastaDecipherAligned <- as.AAbin(fastaDecipherAligned)
 fastaphyDat <- as.phyDat(fastaDecipherAligned, type = 'AA')
 rm(fastaDecipherAligned)

dm <- dist.ml(fastaphyDat, model="JTT")
tree <- NJ(dm)
## cat('Testing all models ...\n')
## (mt <- modelTest(fastaphyDat, model="all", multicore=multiCoreBool))
## env <- attr(mt, "env")
## fitStart <- eval(get(mt$Model[which.min(mt$BIC)], env), env)
## fitNJ <- pml(tree, fastaphyDat, model=fitStart$model, k=4, inv=.2)
fitNJ <- pml(tree, fastaphyDat, model = 'JTT', k=4, inv=.2)
fit <- optim.pml(fitNJ, rearrangement = "stochastic",
                     optInv=TRUE, optGamma=TRUE)
fit

You would only need to load('fastaRefined62.RData') to test.