Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rebar3/erlang issues #370

Open
lhendre opened this issue May 23, 2022 · 6 comments
Open

Rebar3/erlang issues #370

lhendre opened this issue May 23, 2022 · 6 comments

Comments

@lhendre
Copy link

lhendre commented May 23, 2022

Trying to compile/build the project but i'm getting the error below. It saying I need to recompile with Erlang 25, but that is the current version I am on. Looking into if it is another dependency issue currently.

 ./rebar3 as prod tar
===> Verifying dependencies...
+ mkdir -p _build/cmake
+ cd _build/cmake
+ type cmake3
+ CMAKE=cmake
+ cmake ../../c_src
-- cmake args -DCMAKE_POSITION_INDEPENDENT_CODE=ON;-DCMAKE_CXX_COMPILER=/Library/Developer/CommandLineTools/usr/bin/c++;-DCMAKE_AR=/Library/Developer/CommandLineTools/usr/bin/ar;-DCMAKE_BUILD_TYPE=Release;-DFAIL_ON_WARNINGS=OFF;-DPORTABLE=ON;-DUSE_RTTI=1;-DWITH_TOOLS=OFF;-DWITH_TESTS=OFF;-DWITH_GFLAGS=OFF;-DWITH_JEMALLOC=OFF;-DWITH_TBB=;-DWITH_SNAPPY=;-DWITH_LZ4=;-DWITH_ZLIB=;-DWITH_BZ2=;-DWITH_ZSTD=
-- Found RocksDB library: /Users/lucashendren/workspace/arweave/ar2/arweave/_build/default/lib/rocksdb/_build/cmake/rocksdb-prefix/src/rocksdb-build/librocksdb.a
-- Found RocksDB includes: /Users/lucashendren/workspace/arweave/ar2/arweave/_build/default/lib/rocksdb/c_src/../deps/rocksdb/include
-- Using OTP lib: /usr/local/Cellar/erlang/25.0/lib/erlang/lib - found
-- Using erl_interface version: erl_interface-5.3
-- Using erts version: erts-13.0
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/lucashendren/workspace/arweave/ar2/arweave/_build/default/lib/rocksdb/_build/cmake
+ echo done.
done.
make: `/Users/lucashendren/workspace/arweave/ar2/arweave/_build/default/lib/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so' is up to date.
=ERROR REPORT==== 22-May-2022::22:23:40.929654 ===
beam/beam_load.c(148): Error loading module enc:
  please re-compile this module with an Erlang/OTP 25 compiler


escript: exception error: undefined function enc:main/1
  in function  escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1 
  in call from init:do_boot/3 
===> Hook for compile failed!
@ldmberman
Copy link
Member

Hi @lhendre, try to remove _build and recompile.

@lhendre
Copy link
Author

lhendre commented May 24, 2022

@ldmberman removed the _build directory and still having issues, will keep exploring

@ThomasBlock
Copy link

Hi @ldmberman . Thank you for the other reply. Following that i have to compile arweave for the first time. I have linux experience, but no erlang experience. i see the same error as described here. what could be wrong? here my commandline with a fresh Ubuntu20.04 VM:

echo "deb https://packages.erlang-solutions.com/ubuntu $(lsb_release -cs) contrib" | sudo tee /etc/apt/sources.list.d/erlang.list
curl -fsSL https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/erlang.gpg
apt update && apt upgrade -y
apt install build-essential erlang git libsqlite3-dev libgmp-dev curl cmake
erl
Erlang/OTP 25 [erts-13.0.3] [source] [64-bit] [smp:24:10] [ds:24:10:10] [async-threads:1] [jit:ns]

git clone --recursive https://github.com/ArweaveTeam/arweave.git
cd arweave
./rebar3 as prod tar
===> Verifying dependencies...
===> Fetching b64fast (from {git,"https://github.com/ArweaveTeam/b64fast.git",
                   {ref,"a0ef55ec66ecf705848716c195bf45665f78818a"}})
===> Fetching pc (from {git,"https://github.com/blt/port_compiler.git",{tag,"v1.12.0"}})
===> Analyzing applications...
===> Compiling pc
===> Fetching cowboy v2.9.0
===> Fetching graphql (from {git,"https://github.com/shopgun/graphql-erlang.git",
                   {ref,"fc76a2df7d1ddcf8bcc3bb46363e7ca8fca80939"}})
===> Fetching geas_rebar3 (from {git,"https://github.com/crownedgrouse/geas_rebar3.git",
                       {branch,"master"}})
===> Fetching geas v2.7.9
===> Fetching samovar v1.0.2
===> Analyzing applications...
===> Compiling samovar
===> Compiling geas
===> Compiling geas_rebar3
===> Fetching rebar3_hex v7.0.2
===> Fetching hex_core v0.8.4
===> Fetching verl v1.1.1
===> Analyzing applications...
===> Compiling verl
===> Compiling hex_core
===> Compiling rebar3_hex
===> Fetching gun v1.3.2
===> Fetching jiffy v1.0.8
===> Fetching prometheus v4.6.0
===> Fetching rebar3_archive_plugin v0.0.1
===> Fetching rebar3_elvis_plugin (from {git,"https://github.com/deadtrickster/rebar3_elvis_plugin.git",
                               "master"})
===> WARNING: It is recommended to use {branch, Name}, {tag, Tag} or {ref, Ref}, otherwise updating the dep may not work as expected.
===> Fetching katana_code v0.1.0
===> Fetching zipper v1.0.1
===> Fetching aleppo v1.0.0
===> Analyzing applications...
===> Compiling zipper
===> Compiling aleppo
_build/default/plugins/aleppo/src/aleppo.erl:6:2: Warning: record ale_context has field(s) without type information
_build/default/plugins/aleppo/src/aleppo.erl:12:1: Warning: missing specification for function process_file/1
_build/default/plugins/aleppo/src/aleppo.erl:15:1: Warning: missing specification for function process_file/2
_build/default/plugins/aleppo/src/aleppo.erl:25:1: Warning: missing specification for function process_tokens/1
_build/default/plugins/aleppo/src/aleppo.erl:32:1: Warning: missing specification for function process_tokens/2
_build/default/plugins/aleppo/src/aleppo.erl:307:1: Warning: missing specification for function scan_file/1

===> Compiling katana_code
===> Compiling rebar3_elvis_plugin
===> Analyzing applications...
===> Compiling rebar3_archive_plugin
===> Fetching prometheus_cowboy v0.1.8
===> Fetching prometheus_process_collector v1.6.0
===> Fetching rocksdb (from {git,"https://gitlab.com/arweave1/erlang-rocksdb.git",
                   {ref,"ed4d05d58d174485f883b5cd3e057c64d9e7ff3a"}})
===> Fetching cowlib v2.8.0
===> Fetching prometheus_httpd v2.1.11
===> Fetching ranch v1.8.0
===> Fetching accept v0.3.5
+ mkdir -p _build/cmake
+ cd _build/cmake
+ type cmake3
+ CMAKE=cmake
+ cmake ../../c_src
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- cmake args -DCMAKE_POSITION_INDEPENDENT_CODE=ON;-DCMAKE_CXX_COMPILER=/usr/bin/c++;-DCMAKE_AR=/usr/bin/ar;-DCMAKE_BUILD_TYPE=Release;-DFAIL_ON_WARNINGS=OFF;-DPORTABLE=ON;-DUSE_RTTI=1;-DWITH_TOOLS=OFF;-DWITH_TESTS=OFF;-DWITH_GFLAGS=OFF;-DWITH_JEMALLOC=OFF;-DWITH_TBB=;-DWITH_SNAPPY=;-DWITH_LZ4=;-DWITH_ZLIB=;-DWITH_BZ2=;-DWITH_ZSTD=
-- Found RocksDB library: /root/arweave/_build/default/lib/rocksdb/_build/cmake/rocksdb-prefix/src/rocksdb-build/librocksdb.a
-- Found RocksDB includes: /root/arweave/_build/default/lib/rocksdb/c_src/../deps/rocksdb/include
-- Using OTP lib: /usr/lib/erlang/lib - found
-- Using erl_interface version: erl_interface-5.3
-- Using erts version: erts-13.0.3
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /root/arweave/_build/default/lib/rocksdb/_build/cmake
+ echo done.
done.
make: Entering directory '/root/arweave/_build/default/lib/prometheus_process_collector/c_src'
g++ -O3 -finline-functions -fPIC -I /usr/lib/erlang/erts-13.0.3/include/ -I /usr/lib/erlang/lib/erl_interface-5.3/include -std=c++11 -Wall  -c -o prometheus_process_collector_nif.o prometheus_process_collector_nif.cc
g++ -O3 -finline-functions -fPIC -I /usr/lib/erlang/erts-13.0.3/include/ -I /usr/lib/erlang/lib/erl_interface-5.3/include -std=c++11 -Wall  -c -o prometheus_process_info_linux.o prometheus_process_info_linux.cc
cc prometheus_process_collector_nif.o prometheus_process_info_linux.o -shared -L /usr/lib/erlang/lib/erl_interface-5.3/lib -lei -lstdc++ -o /root/arweave/_build/default/lib/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
make: Leaving directory '/root/arweave/_build/default/lib/prometheus_process_collector/c_src'
=ERROR REPORT==== 17-Aug-2022::16:38:29.833856 ===
beam/beam_load.c(148): Error loading module enc:
  please re-compile this module with an Erlang/OTP 25 compiler


escript: exception error: undefined function enc:main/1
  in function  escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1 
  in call from init:do_boot/3 
===> Hook for compile failed!

@ThomasBlock
Copy link

So here are additional findings:

So it seems i also have to compile OTP 23 myself? ( Ubuntu 20.04 on all of my systems )

===> Compiling c_src/b64fast.c
===> Linking /root/arweave/_build/default/lib/b64fast/priv/b64fast.so
===> "/root/arweave/_build/default/lib/prometheus_cowboy/ebin/prometheus_cowboy.app" is missing description entry
make: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[1]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
Scanning dependencies of target randomx
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
[  2%] Building CXX object CMakeFiles/randomx.dir/src/aes_hash.cpp.o
[  5%] Building C object CMakeFiles/randomx.dir/src/argon2_ref.c.o
[  8%] Building C object CMakeFiles/randomx.dir/src/argon2_ssse3.c.o
[ 11%] Building C object CMakeFiles/randomx.dir/src/argon2_avx2.c.o
[ 14%] Building CXX object CMakeFiles/randomx.dir/src/bytecode_machine.cpp.o
[ 17%] Building CXX object CMakeFiles/randomx.dir/src/cpu.cpp.o
[ 20%] Building CXX object CMakeFiles/randomx.dir/src/dataset.cpp.o
[ 23%] Building CXX object CMakeFiles/randomx.dir/src/soft_aes.cpp.o
[ 26%] Building CXX object CMakeFiles/randomx.dir/src/virtual_memory.cpp.o
[ 29%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted.cpp.o
[ 32%] Building CXX object CMakeFiles/randomx.dir/src/allocator.cpp.o
[ 35%] Building CXX object CMakeFiles/randomx.dir/src/assembly_generator_x86.cpp.o
[ 38%] Building CXX object CMakeFiles/randomx.dir/src/instruction.cpp.o
[ 41%] Building CXX object CMakeFiles/randomx.dir/src/randomx.cpp.o
[ 44%] Building CXX object CMakeFiles/randomx.dir/src/superscalar.cpp.o
[ 47%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled.cpp.o
[ 50%] Building CXX object CMakeFiles/randomx.dir/src/vm_interpreted_light.cpp.o
[ 52%] Building C object CMakeFiles/randomx.dir/src/argon2_core.c.o
[ 55%] Building CXX object CMakeFiles/randomx.dir/src/blake2_generator.cpp.o
[ 58%] Building CXX object CMakeFiles/randomx.dir/src/instructions_portable.cpp.o
[ 61%] Building C object CMakeFiles/randomx.dir/src/reciprocal.c.o
[ 64%] Building CXX object CMakeFiles/randomx.dir/src/virtual_machine.cpp.o
[ 67%] Building CXX object CMakeFiles/randomx.dir/src/vm_compiled_light.cpp.o
[ 70%] Building C object CMakeFiles/randomx.dir/src/blake2/blake2b.c.o
[ 73%] Building CXX object CMakeFiles/randomx.dir/src/jit_compiler_x86.cpp.o
[ 76%] Building C object CMakeFiles/randomx.dir/src/jit_compiler_x86_static.S.o
[ 79%] Linking CXX static library librandomx.a
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 79%] Built target randomx
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
Scanning dependencies of target randomx-benchmark
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 82%] Building CXX object CMakeFiles/randomx-benchmark.dir/src/tests/benchmark.cpp.o
[ 85%] Building CXX object CMakeFiles/randomx-benchmark.dir/src/tests/affinity.cpp.o
[ 88%] Linking CXX executable randomx-benchmark
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 88%] Built target randomx-benchmark
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
Scanning dependencies of target randomx-codegen
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 91%] Building CXX object CMakeFiles/randomx-codegen.dir/src/tests/code-generator.cpp.o
[ 94%] Linking CXX executable randomx-codegen
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 94%] Built target randomx-codegen
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
Scanning dependencies of target randomx-tests
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make[2]: Entering directory '/root/arweave/apps/arweave/lib/RandomX/build'
[ 97%] Building CXX object CMakeFiles/randomx-tests.dir/src/tests/tests.cpp.o
[100%] Linking CXX executable randomx-tests
make[2]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
[100%] Built target randomx-tests
make[1]: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make: Leaving directory '/root/arweave/apps/arweave/lib/RandomX/build'
make: Entering directory '/root/arweave/apps/arweave/c_src'
g++ -O3 -finline-functions -Wall -fPIC -I /usr/lib/erlang/erts-11.0/include/ -I /usr/lib/erlang/lib/erl_interface-4.0/include -I ../lib/RandomX/src -std=c++11  -c -o /root/arweave/apps/arweave/c_src/feistel_msgsize_key_cipher.o /root/arweave/apps/arweave/c_src/feistel_msgsize_key_cipher.cpp
/root/arweave/apps/arweave/c_src/feistel_msgsize_key_cipher.cpp:1:10: fatal error: openssl/sha.h: No such file or directory
    1 | #include <openssl/sha.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:77: /root/arweave/apps/arweave/c_src/feistel_msgsize_key_cipher.o] Error 1
make: Leaving directory '/root/arweave/apps/arweave/c_src'
===> Hook for compile failed!

@ThomasBlock
Copy link

Ah that were just missing header files (libssl-dev). I found now a solution. Maybe you want to clarify README.md - i guess this is not intuitive for everyone.. Here my commandline to compile Arweave 2.5.3 on Ubuntu 20.04 :

sudo -s
apt install curl build-essential git libsqlite3-dev libgmp-dev cmake libssl-dev -y
echo "deb https://packages.erlang-solutions.com/ubuntu $(lsb_release -cs) contrib" | sudo tee /etc/apt/sources.list.d/erlang.list
curl -fsSL https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/erlang.gpg
apt update && apt upgrade -y
apt-get install esl-erlang=1:23.0-1 -y
apt-mark hold esl-erlang
git clone --recursive https://github.com/ArweaveTeam/arweave.git
cd arweave
./rebar3 as prod tar

@ldmberman
Copy link
Member

Good point, we need to add the OpenSSL dev headers requirement to the list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ldmberman @lhendre @ThomasBlock