RcppArrayFire - in Windows

[JKcme]

Hi! I've been trying to install in Windows - and I realize this works under Linux at the moment, but I thought I might be able to build from source since I have RTools installed.

But I get this error:

• installing source package 'RcppArrayFire' ...

  ---

  WARNING: this package has a configure script
  It probably needs manual configuration

  ---

** libs

*** arch - i386
c:/Rtools/mingw_32/bin/g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-3.4.3/include" -DNDEBUG -I"C:/Users/Documents/R/win-library/3.4/Rcpp/include" -O2 -Wall -mtune=generic -c RcppArrayFire.cpp -o RcppArrayFire.o
RcppArrayFire.cpp:23:27: fatal error: RcppArrayFire.h: No such file or directory
#include <RcppArrayFire.h>
^
compilation terminated.

Any ideas how I might proceed here? I'm not familiar with building from source, but I do use Rcpp. My ArrayFire install will run examples in Visual Studio with no problem. Thanks!

[rstub]

You could try to manually configure the package. The configure script tries to locate the ArrayFire installation. It then takes src/Makevars.in and R/flags.R.in as input and replaces @AF_INCLUDE@ and @AF_LIBS with appropriate values to produce src/Makevars and R/flags.R. You could try doing this by hand:

1. Copy src/Makevars.in and R/flags.R.in to src/Makevars and R/flags.R, respectively.
2. Replace @AF_INCLUDE@ in src/Makevars and R/flags.R with -I<arrayfire_root>\include
3. Replace @AF_LIBS@ in src/Makevars and R/flags.R with -L<arrayfire_root>\lib -laf

After that compilation might work. I am very interested in your results.

[rstub]

According to http://arrayfire.org/docs/using_on_windows.htm the ArrayFire installer sets an environment variable AF_PATH. Could you try to use the following as src/Makevars.win:

## enviorment variables for building R package
CXX_STD = CXX11
PKG_CXXFLAGS = -I../inst/include -I$(AF_PATH)/include
AF_LIBS = -L$(AF_PATH)/lib -laf
PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(AF_LIBS)

[JKcme]

Hey Ralf! Thanks so much - I'll try to get to this today or over the weekend. I'll let you know how this goes.

[JKcme]

Hey Ralf - I tried the adjustments to src/Makevars.win by simply opening src/Makevars.in and copying your code as above and saving as Makevars.win in RStudio. (I put all of your project code into a new project called "ArrayFireWin". I've built Rcpp packages before, but not from source....so I'm of limited diagnostic help here since I've got little experience in this particular area...also, I did not make the adjustments to src/Makevars.in and R/flags.R. Should this be done as well? Thanks!)

PS: I can load ArrayFire in Visual Studio and run examples, so I believe my system Path is correct.

I got this error when I run Rcmd.exe:

==> Rcmd.exe build ArrayFireWin

* checking for file 'ArrayFireWin/DESCRIPTION' ... OK
* preparing 'RcppArrayFire':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'RcppArrayFire' ...

**********************************************
WARNING: this package has a configure script
It probably needs manual configuration
**********************************************

** libs
c:/Rtools/mingw_64/bin/g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-34~1.3/include" -DNDEBUG -``I"C:/Users/eXXXXX/Documents/R/win-library/3.4/Rcpp/include" -I../inst/include -IC:\Program Files\ArrayFire\v3/include -O2 -Wall -mtune=generic -c RcppArrayFire.cpp -o RcppArrayFire.o
g++.exe: error: FilesArrayFirev3/include: No such file or directory
make: *** [RcppArrayFire.o] Error 1

[JKcme]

Also, I did the above and then copied R/flags.R.in to R/flags.R. I then changed @AF_INCLUDE@ to -I$(AF_PATH)/include and changed @AF_LIBS@ to -L$(AF_PATH)/lib -laf. I also tried changing @AF_LIBS@ to simply AF_LIBS and still got the same error.

[rstub]

Thanks for your tests! The Problem are the spaces in $AF_PATH, which make cannot handle. For now we have to go back to explicitly setting the paths in src/Makevars:

## enviorment variables for building R package
CXX_STD = CXX11
PKG_CXXFLAGS = -I../inst/include -I"C:/PROGRA~1/ArrayFire/v3/include"
AF_LIBS = -L"C:/PROGRA~1/ArrayFire/v3/lib" -laf
PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(AF_LIBS)

You can also make equivalent changes in R/flag.R, but that is of lower priority for now.

[rstub]

My ArrayFire install will run examples in Visual Studio with no problem.

This hints at an issue: The ArrayFire libraries are compiled with Visual Studio, but the Rtools toolchain uses gcc. The C++ ABIs are different for these two compilers. So I fear one either has to compile ArrayFire from source using gcc, or RcppArrayFire would have to use to C API instead of the C++ API, since the C ABIs should be compatible.

[JKcme]

Well, I think you're on to something here...I might have to build ArrayFire from source using gcc. I made the changes you suggested above and got the following (a partial readout....this is probably more problematic than we thought....):

** libs c:/Rtools/mingw_64/bin/g++ -std=gnu++11 -I"C:/PROGRA1/R/R-341.3/include" -DNDEBUG -`

I"C:/Users/e19665/Documents/R/win-library/3.4/Rcpp/include" -I../inst/include -

I"C:/PROGRA~1/ArrayFire/v3/include" -O2 -Wall -mtune=generic -c RcppArrayFire.cpp -o RcppArrayFire.o
In file included from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/combaseapi.h:155:0,
from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/objbase.h:14,
from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/ole2.h:17,
from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/wtypes.h:12,
from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/winscard.h:10,
from C:/Rtools/mingw_64/x86_64-w64-mingw32/include/windows.h:97,
from C:/PROGRA~1/ArrayFire/v3/include/af/timing.h:16,
from C:/PROGRA~1/ArrayFire/v3/include/arrayfire.h:320,
from ../inst/include/RcppArrayFireForward.h:29,
from ../inst/include/RcppArrayFire.h:29,
from RcppArrayFire.cpp:23:
C:/Rtools/mingw_64/x86_64-w64-mingw32/include/objidlbase.h:864:18: error: macro "Realloc" requires 3 arguments, but only 2 given SIZE_T cb) = 0;

C:/Rtools/mingw_64/x86_64-w64-mingw32/include/objidlbase.h:864:22: error: 'Realloc' declared as a 'virtual' field SIZE_T cb) = 0;
^
In file included from C:/PROGRA~1/R/R-34~1.3/include/R.h:91:0,
from C:/Users/eXXXX/Documents/R/win-library/3.4/Rcpp/include/Rcpp/r/headers.h:52,
from C:/Users/eXXXX/Documents/R/win-library/3.4/Rcpp/include/RcppCommon.h:38,
from ../inst/include/RcppArrayFireForward.h:26,
from ../inst/include/RcppArrayFire.h:29,
from RcppArrayFire.cpp:23:
C:/PROGRA~1/R/R-34~1.3/include/R_ext/RS.h:74:37: error: expected identifier before '(' token #define Free(p) (R_chk_free( (void *)(p) ), (p) = NULL)

[rstub]

I have raised the question of building ArrayFire with mingw in their slack community.

[JKcme]

Good idea - let's see where that goes. I may have a couple of people I can approach as well.

[rstub]

Response on slack was that it should work given that ArrayFire does not use any Visual C++ specialties on Windows. An alternative solution might be to build the R package using Visual C++, as it is done for LightGBM.

[JKcme]

Hmmm....not sure what to do now....lol....

[rstub]

I am not sure either. Good news is that I now have a Windows machine available and I intend to use it also for porting RcppArrayFire. I was already able to reproduce the errors you are getting.

[JKcme]

Hey Ralf, that's great - at least you've reproduced the errors! Let me know when you've made some progress and I'll definitely help test. - John

[JKcme]

Hey Ralf, by the way, I've switched to Windows10 - but this probably won't make much difference.

[JKcme]

Ralf - I've also successfully run some ArrayFire examples under Visual Studio using OpenCL - so I know that works too. I tried using CUDA, but it looks like the directory structure is wrong. I'm probably just going to stick with OpenCL when using with RcppArrayFire.

[rstub]

I have made some progress with building ArrayFire using a mingw tool chain, but this is slow going ...

[rstub]

And I can now rule out using Visual C++, since Rcpp is not compatible with that. It seems packages like lightGBM use R's C-API only.

[JKcme]

Yeah, I think the mingw tool chain is the best bet. Sorry I can't be of much help on this....

[Despertaferro]

Hello, thank you for the effort of creating RcppArrayFire. Is there any possibility there could be a Windows version? I see there were some ideas about it, but it is a few months since no changes. Best.

[rstub]

Thanks for your interest, @Despertaferro. The problem is simply that one has to compile ArrayFire for Windows using the mingw tool chain. There is no other way this can work. BTW, this also rules out CUDA support on Windows, since CUDA on windows requires VC++. I have tried to compile ArrayFire on Windows using RTools, but never quite succeeded. I currently do not have a Windows machine available. I am therefore flagging this issue as "help wanted".

[mschubert]

Drive-by comment for Realloc, Free. Maybe this helps someone. Either undefine the allocs:

#include <R.h>
#undef Realloc
#define R_Realloc(p,n,t) (t *) R_chk_realloc( (void *)(p), (size_t)((n) * sizeof(t)) )
#include <windows.h>

or define STRICT_R_HEADERS for Rcpp (original discussion here):

#define STRICT_R_HEADERS
#include <Rcpp.h>

or, even simpler, set -DSTRICT_R_HEADERS in eg CXXFLAGS in ~/.R/Makevars.

[rstub]

@mschubert Thanks for the comment. STRICT_R_HEADERS is already used. Maybe I should try again to build this for Windows. Especially since RTools4 is now available.