Does not build on MacOS Sierra

[kopertop]

I'm having trouble building node-parquet on my MacOS system:

Darwin Apollo.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64

Everything appears to build fine until the node-gyp rebuild step:

> node-gyp rebuild

  CXX(target) Release/obj.target/parquet/src/parquet_binding.o
In file included from ../src/parquet_binding.cc:3:
In file included from ../src/parquet_reader.h:8:
In file included from ../deps/parquet-cpp/src/parquet/api/reader.h:22:
../deps/parquet-cpp/src/parquet/column_reader.h:22:10: fatal error: 'cstdint' file not found
#include <cstdint>
         ^
1 error generated.
make: *** [Release/obj.target/parquet/src/parquet_binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (~/.nvm/versions/node/v6.11.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:285:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Darwin 16.7.0
gyp ERR! command "~/.nvm/versions/node/v6.11.0/bin/node" "~/.nvm/versions/node/v6.11.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd node-parquet
gyp ERR! node -v v6.11.0
gyp ERR! node-gyp -v v3.6.0
gyp ERR! not ok 
npm ERR! code ELIFECYCLE
npm ERR! errno 1

I have g++, cmake, boost, and thrift all installed. I've even tried upgrading to newer versions of cmake and boost building from source, re-installing packages, and everything else I could think of.

StackOverflow seems to think this "cstdint" package is included in a "tr1" folder, and proposes a solution: https://stackoverflow.com/questions/10116724/clang-os-x-lion-cannot-find-cstdint

however, the proposed solution doesn't work for me either.

Any help getting this to build would be greatly appreciated.

[xhochy]

Can you switch your build to use clang and not gcc?

I would suspect this comes from either an outdated/not supported compiler or a missing flag on the compiler call. Would it be possible to get to see the compiler call with all the flags?

[kopertop]

I tried with the following but got the same result. Here looks like a little bit more detail:

% CC=clang node-gyp rebuild
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@6.11.0 | darwin | x64
gyp info spawn ~/.virtualenv/bin/python2
gyp info spawn args [ '~/.nvm/versions/node/v6.11.0/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '~/node-parquet/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '~/.nvm/versions/node/v6.11.0/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '~/.node-gyp/6.11.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=~/.node-gyp/6.11.0',
gyp info spawn args   '-Dnode_gyp_dir=~.nvm/versions/node/v6.11.0/lib/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=~/.node-gyp/6.11.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=~/node-parquet',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CXX(target) Release/obj.target/parquet/src/parquet_binding.o
In file included from ../src/parquet_binding.cc:3:
In file included from ../src/parquet_reader.h:8:
In file included from ../deps/parquet-cpp/src/parquet/api/reader.h:22:
../deps/parquet-cpp/src/parquet/column_reader.h:22:10: fatal error: 'cstdint' file not found
#include <cstdint>
         ^
1 error generated.
make: *** [Release/obj.target/parquet/src/parquet_binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (~/.nvm/versions/node/v6.11.0/lib/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Darwin 16.7.0
gyp ERR! command "~/.nvm/versions/node/v6.11.0/bin/node" "~/.nvm/versions/node/v6.11.0/bin/node-gyp" "rebuild"
gyp ERR! cwd ~/node-parquet
gyp ERR! node -v v6.11.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 

[mvertes]

Following the update to parquet-cpp-1.2.0, the location and linking of some parquet-cpp dependencies seem to have changed. I can build now and run on linux, but not yet on MacOSX, where it fails now in the building of parquet-cpp, before node-gyp linking. I had to rework the build recipe, both on parquet-cpp and node side.

following is the build recipe (ok on linux, fails on mac):

BUILD_DIR=$PWD/build_deps/parquet-cpp
ARROW_EP=$BUILD_DIR/arrow_ep-prefix/src/arrow_ep-build
BROTLI_EP=$ARROW_EP/brotli_ep-prefix/src/brotli_ep-build

export SNAPPY_STATIC_LIB=$ARROW_EP/snappy_ep/src/snappy_ep-install/lib/libsnappy.a
export BROTLI_STATIC_LIB_ENC=$BROTLI_EP/libbrotlienc.a
export BROTLI_STATIC_LIB_DEC=$BROTLI_EP/libbrotlidec.a
export BROTLI_STATIC_LIB_COMMON=$BROTLI_EP/libbrotlicommon.a
export ZLIB_STATIC_LIB=$ARROW_EP/zlib_ep/src/zlib_ep-install/lib/libz.a

mkdir -p $BUILD_DIR
cd $BUILD_DIR
cmake -DPARQUET_BUILD_TESTS=OFF \
  -DPARQUET_MINIMAL_DEPENDENCY=ON \
  -DPARQUET_ARROW_LINKAGE="static" \
  -DPARQUET_BUILD_SHARED=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  ../../deps/parquet-cpp
make

It fails with:

[...]

-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Found cpplint executable at /Users/marc/mvertes/node-parquet/build_deps/parquet-cpp/arrow_ep-prefix/src/arrow_ep/cpp/build-support/cpplint.py
CMake Error at cmake_modules/BuildUtils.cmake:179 (set_target_properties):
  set_target_properties Can not find target to add properties to:
  arrow_shared
Call Stack (most recent call first):
  CMakeLists.txt:766 (ADD_ARROW_LIB)


-- Configuring incomplete, errors occurred!
See also "/Users/marc/mvertes/node-parquet/build_deps/parquet-cpp/arrow_ep-prefix/src/arrow_ep-build/CMakeFiles/CMakeOutput.log".
See also "/Users/marc/mvertes/node-parquet/build_deps/parquet-cpp/arrow_ep-prefix/src/arrow_ep-build/CMakeFiles/CMakeError.log".
make[2]: *** [arrow_ep-prefix/src/arrow_ep-stamp/arrow_ep-configure] Error 1
make[1]: *** [CMakeFiles/arrow_ep.dir/all] Error 2
make: *** [all] Error 2

Seems that on mac, building ARROW_STATIC fails. I will report to parquet-cpp and see if I can fix it.

[mvertes]

The cause of failure on MacOSX is -DPARQUET_BUILD_SHARED=OFF in build_parquet-cpp.sh . Removing it allows to compile and pass tests on Linux and MacOSX now.

[mvertes]

Can you confirm it works after the merge of #34 ?
Thanks