Use unix::rlimit_as() instead.
Limit R memory on POSIX systems (Linux, OS X, ...).
devtools::install_github("krlmlr/ulimit")
Execute the following to limit the memory available to R to 2000 MiB:
ulimit::memory_limit(2000)See also the package documentation and the small vignette.
RAppArmor has the rlimit_as() function and much, much more.