[BUGZILLA #16921] Rscript does not load the methods package by default

[MichaelChirico]

Created attachment 2093 [details]
Patch to include the methods package in Rscript defaults

This is a proposal to change the default behavior of Rscript to load the methods package by default. This has been a frequent issue when code that runs in an interactive session or by calling R directly fails to run when using Rscript and it is not intuitively obvious to users why the code fails.

In addition, the performance impact is now largely attenuated due to lazy loading of package functions in recent R versions.

for i in {1..5}; do time R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e 'invisible()';done

R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e   0.08s user 0.05s system 95% cpu 0.137 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e   0.08s user 0.05s system 95% cpu 0.132 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e   0.08s user 0.05s system 95% cpu 0.132 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e   0.08s user 0.04s system 95% cpu 0.128 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats' Rscript -e   0.09s user 0.05s system 94% cpu 0.139 total

for i in {1..5}; do time R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript -e 'invisible()';done

R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript   0.14s user 0.05s system 96% cpu 0.195 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript   0.14s user 0.05s system 95% cpu 0.193 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript   0.13s user 0.05s system 96% cpu 0.187 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript   0.13s user 0.05s system 96% cpu 0.191 total
R_DEFAULT_PACKAGES='datasets,utils,grDevices,graphics,stats,methods' Rscript   0.13s user 0.05s system 96% cpu 0.191 total

The attached patch changes the default to 'datasets,utils,grDevices,graphics,stats,methods' and updates the documentation of ?Rscript and in R-admin. If there are other places where changes are necessary I am happy to provide updated patches.

---

METADATA

• Bug author - Jim Hester
• Creation time - 2016-05-23 16:21:15 UTC
• Bugzilla link
• Status - CLOSED FIXED
• Alias - None
• Component - Startup
• Version - R 3.3.*
• Hardware - All All
• Importance - P5 enhancement
• Assignee - R-core
• URL -
• Modification time - 2018-01-07 04:25 UTC

[MichaelChirico]

That is (was?) a conscious design decision, at least when Rscript was released.

[ Disclaimer: I am not unbiased here as littler came out a little earlier, always loaded the methods package, and was (and still is) still quicker to start. ]

---

METADATA

• Comment author - Dirk Eddelbuettel
• Timestamp - 2016-05-23 20:27:55 UTC

[MichaelChirico]

Thanks for raising this issue. Rscript was designed for running short snippets of R code in the context of programs like R CMD check. It sounds like you've encountered use cases where more complex programs are invoked via the command line. Would you please provide some examples of such cases? Thanks in advance.

---

METADATA

• Comment author - Michael Lawrence
• Timestamp - 2016-05-24 17:44:03 UTC

[MichaelChirico]

Easy: Any job you may want to run from, say, cron that involves S4.

Works interactively in R (as methods is loaded), bombs immediately via Rscript. I never understood how that is supposed to make sense, and I don't even program with S4.

---

METADATA

• Comment author - Dirk Eddelbuettel
• Timestamp - 2016-05-24 17:50:48 UTC

[MichaelChirico]

Yes, that is pretty obvious, but a lot of times scripts use S4 through packages that they are already attaching. Only things like as() might cause a problem. Personally, I think consistency should win here, but others have a different opinion.

---

METADATA

• Comment author - Michael Lawrence
• Timestamp - 2016-05-24 18:19:20 UTC

[MichaelChirico]

I have run into this a number of times personally, two recent ones were a user seeing different coverage results locally than on Travis (r-lib/covr#180) and coverage of the testthat package being lower than expected (r-lib/testthat#475).

Some more examples of people running into similar issues using a simple github search (https://github.com/search?utf8=%E2%9C%93&q=Rscript+methods+is%3Aissue)

• methods not loaded by Rscript tudo-r/BatchJobs#27
• Multicore with batchtools mllg/batchtools#15
• read_csv fails on Linux with large files when invoked via Rscript without the methods library tidyverse/readr#347
• Add support for loading methods package andrie/version.compare#2
• Need methods package on the search path tidyverse/dplyr#1760
• Add R_DEFAULT_PACKAGES to Renviron csgillespie/efficientR#21
• Rscript: trying to execute load actions without 'methods' package Bioconductor/BiocParallel#44
• calling with Rscript gives an error COMBINE-lab/wasabi#1
• glmnet version 2.0-2 fails to properly load the methods package dhimmel/elevcan#1

I am not familiar enough with S4 dispatch to know the exact cases when this poses a problem, but anecdotally it seems to cause problems anytime you call a package using S4 from Rscript as Dirk mentioned.

My feeling is more time has been wasted by users trying to debug the inconsistency than has been gained by the <60ms difference in startup time (from my benchmarks). This may have been a more reasonable trade-off when package loading was slower, but the slowdown with modern R seems slight to me.

---

METADATA

• Comment author - Jim Hester
• Timestamp - 2016-05-24 18:49:30 UTC

[MichaelChirico]

I also explained maybe half a dozen times on StackOverflow.

The un-intuitive nature of this is, as Jim argued, indeed less than helpful. I would welcome a reversal which makes Rscript closer to standard R.

---

METADATA

• Comment author - Dirk Eddelbuettel
• Timestamp - 2016-05-24 19:23:37 UTC

[MichaelChirico]

Thanks for these examples. While some of them have helpfully exposed bugs in the methods package or other packages, many of them do argue for consistency.

---

METADATA

• Comment author - Michael Lawrence
• Timestamp - 2016-05-24 19:29:48 UTC

[MichaelChirico]

I agree that interactive and non-interactive modes ought to use the same defaultPackages.

I suppose removing methods from the list of default packages in interactive mode would be too disruptive, so that implies the lesser evil is to add it for non-interactive mode.

On the other hand, the fact that some bugs would have gone undetected, had this already been the default, is something to consider.

---

METADATA

• Comment author - Benjamin Tyner
• Timestamp - 2016-05-25 16:55:22 UTC

[MichaelChirico]

This can now be closed; as of revision 74084, "Rscript now defaults to using the same default packages as R."

---

METADATA

• Comment author - Benjamin Tyner
• Timestamp - 2018-01-07 01:53:46 UTC

[MichaelChirico]

Yes, many thanks to Luke for making that change.

---

METADATA

• Comment author - Dirk Eddelbuettel
• Timestamp - 2018-01-07 02:21:57 UTC

[MichaelChirico]

Kurt's suggestion to add the R_SCRIPT_DEFAULT_PACKAGES environment variable made this work.

---

METADATA

• Comment author - Luke Tierney
• Timestamp - 2018-01-07 04:25:31 UTC