Enabling parallel computation on OSX - R

[dimagor]

I would appreciate any help I can get with setting up xgboost for R on OSX to support parallel computation. I've tried anything & everything I can find without any success. I have installed gcc as recommended without multilib and I've played with various incarantions of Makevars (latest was: http://stackoverflow.com/a/38514224).

Environment info

Operating System: OSX 10.11.6

Compiler: gcc brewed without multilib

Package used (python/R/jvm/C++): R

xgboost version used:

If you are using R package, please provide

The R sessionInfo()

R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.6 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.3.1

The command to install xgboost if you are not installing from source

> install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")
trying URL 'http://dmlc.ml/drat/src/contrib/xgboost_0.4-4.tar.gz'
Content type 'application/octet-stream' length 384404 bytes (375 KB)
==================================================
downloaded 375 KB

* installing *source* package ‘xgboost’ ...
** libs
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
In file included from xgboost_R.cc:3:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be comp... <truncated>
#pragma message("Warning: OpenMP is not available, "                    \
        ^
1 warning generated.
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
In file included from xgboost_custom.cc:5:
In file included from ./include/xgboost/logging.h:13:
In file included from ./include/xgboost/./base.h:10:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be comp... <truncated>
#pragma message("Warning: OpenMP is not available, "                    \
        ^
1 warning generated.
/usr/local/Cellar/gcc/6.1.0_1/bin/gcc-6 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG... <truncated>
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
In file included from amalgamation/xgboost-all0.cc:13:
In file included from amalgamation/../src/metric/metric.cc:6:
In file included from ./include/xgboost/metric.h:14:
In file included from ./include/xgboost/./data.h:15:
In file included from ./include/xgboost/./base.h:10:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be comp... <truncated>
#pragma message("Warning: OpenMP is not available, "                    \
        ^
In file included from amalgamation/xgboost-all0.cc:26:
amalgamation/../src/gbm/gbtree.cc:607:3: warning: 'CommitModel' overrides a member function but is n... <truncated>
  CommitModel(std::vector<std::unique_ptr<RegTree> >&& new_trees,
  ^
amalgamation/../src/gbm/gbtree.cc:355:3: note: overridden virtual function is here
  CommitModel(std::vector<std::unique_ptr<RegTree> >&& new_trees,
  ^
In file included from amalgamation/xgboost-all0.cc:27:
amalgamation/../src/gbm/gblinear.cc:102:16: warning: 'DoBoost' overrides a member function but is no... <truncated>
  virtual void DoBoost(DMatrix *p_fmat,
               ^
./include/xgboost/gbm.h:76:16: note: overridden virtual function is here
  virtual void DoBoost(DMatrix* p_fmat,
               ^
In file included from amalgamation/xgboost-all0.cc:30:
In file included from amalgamation/../src/data/data.cc:10:
amalgamation/../src/data/./simple_dmatrix.h:47:10: warning: 'GetColSize' overrides a member function... <truncated>
  size_t GetColSize(size_t cidx) const {
         ^
./include/xgboost/data.h:269:18: note: overridden virtual function is here
  virtual size_t GetColSize(size_t cidx) const = 0;
                 ^
In file included from amalgamation/xgboost-all0.cc:43:
amalgamation/../src/tree/updater_colmaker.cc:852:18: warning: 'SyncBestSolution' overrides a member ... <truncated>
    virtual void SyncBestSolution(const std::vector<int> &qexpand) {
                 ^
amalgamation/../src/tree/updater_colmaker.cc:881:11: note: in instantiation of member class 'xgboost::tree::DistColMaker<xgboost::tree::GradStats>::Builder' ... <truncated>
  Builder builder;
          ^
amalgamation/../src/tree/updater_colmaker.cc:893:16: note: in instantiation of template class 'xgboo... <truncated>
    return new DistColMaker<GradStats>();
               ^
amalgamation/../src/tree/updater_colmaker.cc:653:18: note: overridden virtual function is here
    virtual void SyncBestSolution(const std::vector<int> &qexpand) {
                 ^
In file included from amalgamation/xgboost-all0.cc:45:
amalgamation/../src/tree/updater_refresh.cc:28:8: warning: 'Update' overrides a member function but ... <truncated>
  void Update(const std::vector<bst_gpair> &gpair,
       ^
amalgamation/../src/tree/updater_refresh.cc:156:16: note: in instantiation of template class 'xgboos... <truncated>
    return new TreeRefresher<GradStats>();
               ^
./include/xgboost/tree_updater.h:45:16: note: overridden virtual function is here
  virtual void Update(const std::vector<bst_gpair>& gpair,
               ^
6 warnings generated.
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
In file included from amalgamation/dmlc-minimum0.cc:10:
In file included from amalgamation/../dmlc-core/src/data.cc:12:
In file included from amalgamation/../dmlc-core/src/data/disk_row_iter.h:19:
In file included from amalgamation/../dmlc-core/src/data/./libsvm_parser.h:13:
In file included from amalgamation/../dmlc-core/src/data/./text_parser.h:11:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be comp... <truncated>o get benefit of multi-threading. [-W#pragma-messages]
#pragma message("Warning: OpenMP is not available, "                    \
        ^
1 warning generated.
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc... <truncated>
clang++ -std=c++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_mo... <truncated>
ld: library not found for -lgomp
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [xgboost.so] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
Warning in install.packages :
  installation of package ‘xgboost’ had non-zero exit status

The downloaded source packages are in
    ‘/private/var/folders/sx/gzt5483s2wv6fdl_46p7gyz57nzz8m/T/RtmpYrZ6j7/downloaded_packages’

Steps to reproduce

What have you tried?

1. http://stackoverflow.com/a/38514224
2. Using source

[dimagor]

This did the trick (not exactly sure why it didn't work the first time I tried it... but the magic of restarting is the ultimate panacea):

#1136