dependencies from non-cran sources?

[cboettig]

Can you specify dependencies from other standard R repositories as well? (Or more ideally, just extract the repositories URLs from the AdditionalRepositories list of the DESCRIPTION file)

[richfitz]

sure thing - point me at a package?

[richfitz]

(To be clear, you can't yet but it'd just be a case of expanding this function and the template -- I just implemented a set sufficient for my needs)

[cboettig]

how about like this: https://github.com/ropensci/RNeXML/blob/master/DESCRIPTION#L20-21

[richfitz]

Do you know R's logic for looking for packages in the different repos? (e.g., the logic that would be used by install.packages?). Or would you be OK with writing that small bit of yaml? If any of the packages have system dependencies you'll need to write the yaml anyway to list the packages for apt-get.

[cboettig]

I would just add that list to the repos argument of install.packages(), and they should work just like dependencies from CRAN (e.g. we'd still need to install system dependencies, like we do on CRAN, but not specify individual packages and urls manually).

[richfitz]

Sorry, I'm being dense here. Does the actual call to install.packages look like

install.packages(c("ape", "rrdf"), repos="omegahat_url")

or two calls

install.packages("ape") # cran packages
install.packages("rrdf", repos="omegahat_url") # other packages

Basically: where is the logic that says "this package comes from this repo" - is that done automagically in install.packages or manually as in the human-readable info in your DESCRIPTION?

[cboettig]

I think the ideal call concatenates the full package list and a full repo list, including a cran repo, like so:

install.packages(c("ape", "rrdf"), repos=c("http://cran.rstudio.com", "http://www.omegahat.org/R"))

The others would work most of the time, but could fail if the non-cran repo has cran-based dependencies. I believe install.packages just checks all repos listed for all packages listed, and grabs the latest version available. no idea what it does for ties.

[richfitz]

OK, great. I'll look for that field at set up the call like above provided I can get the arguments passed through to install2.r

[cboettig]

@richfitz oh right, looks like I had intended to get that into install2.r but then never got around to it. Added to my fork, see eddelbuettel/littler#12

[richfitz]

I was just going to go via /etc/littler.r but I can leave a note to move over to your fork once it's in r-base?

[eddelbuettel]

Have you guys looked at drat ?

I now think of its drat:::add() aka addRepo() as the best solution for this.

[richfitz]

OK, this is largely done now.

If you have rrdf installed it should pick up all system dependencies (but this needs more tweaking - @gaborcsardi has pointed me in the right direction to do this without relying on installed packages via crandb).

Otherwise make a file in the package root .dockertest.yml containing

system_ignore_packages:
  - animation
system:
  - default-jre
  - default-jdk

which will organise installing rJava (rrdf depends on this, rJava requires system deps, etc, etc).

Then dockertest::build() builds an image. I can get your package to compile but it crashes (segfault) part way through the tests. I've not had a chance to delve further as to the cause sorry.

Generated Dockerfile that I get

FROM r-base

RUN apt-get update && apt-get -y install   \
    curl  \
    default-jdk  \
    default-jre  \
    git  \
    libcurl4-openssl-dev  \
    libgmp-dev  \
    libxml2-dev  \
    libxslt1-dev  \
    pandoc  \
    pandoc-citeproc  \
    ssh

# More recent copy of install2.r so that we can set repos more easily
RUN rm /usr/local/bin/install2.r && \
  wget --no-check-certificate \
  https://raw.githubusercontent.com/cboettig/littler/master/examples/install2.r\
  -O /usr/local/bin/install2.r \
  && chmod +x /usr/local/bin/install2.r

RUN install2.r --error   \
    --repos=http://cran.rstudio.com  \
    --repos=http://server.carlboettiger.info:5555/R  \
    --repos=http://www.omegahat.org/R   \
    ape  \
    devtools  \
    geiger  \
    httr  \
    knitcitations  \
    knitr  \
    phylobase  \
    phytools  \
    plyr  \
    reshape2  \
    rfigshare  \
    rmarkdown  \
    rrdf  \
    Sxslt  \
    taxize  \
    testthat  \
    uuid  \
    XML


COPY scripts /usr/local/bin/

WORKDIR /root
CMD bash

@eddelbuettel - I've not looked at drat much yet, though it did look interesting. I think the dependency management problem has reached the point where more sophisticated tools are needed and if drat helps there then, that will be excellent.

[richfitz]

The crash is in comparative analyses:

> nexml_validate("geospiza.xml")

 *** caught segfault ***
address 0xfbad2a7c, cause 'memory not mapped'

Traceback:
 1: .Call("R_post_form", curl, .opts, .params, TRUE, matchPostStyle(style),     PACKAGE = "RCurl")
 2: RCurl::.postForm(handle$handle, curl_opts, body$body, body$style)
 3: perform(handle, writer, method, opts, body)
 4: make_request("post", hu$handle, hu$url, config, body_config(body,     encode))
 5: POST("http://www.nexml.org/nexml/phylows/validator", body = list(file = upload_file(file)))
 6: nexml_validate("geospiza.xml")

So presumably I've somehow generated a broken RCurl?

[richfitz]

And solved, ish. Adding the Omegahat repo to the search list installed RCurl from there, and that crashes. It's a different version to the CRAN version. I get a 403 on the validate command, but I've been getting them all over the show today because our network is currently flaking out as well as being slow.