编译问题 #309
Assignees
Labels
bug
Something isn't working
Comments
请问编译机器的配置是啥呀 |
@anakinxc 机器配置：64c 256G |
kill 掉然后重新开始试一下？感觉有可能 oom 了 |
@anakinxc kill掉 我执行了 bazel build //... -c opt |
|
可以问下 cpu 是哪种吗？ |
@Jamie-Cui cpu种类 |
试一下在这里 改成 def yacl_cmake_external(**attrs):
if "generate_args" not in attrs:
attrs["generate_args"] = ["-GNinja"]
if "build_args" not in attrs:
attrs["build_args"] = ["-j 8"]
return cmake(**attrs)bazel invoke cmake 的时候每个 cmake task 都可能 spawn #cores 个 build process。。。你这里同时有 7 个 cmake tasks，就可能会有 7*64 个 build process 有概率会拉爆 memory FYI: 改好这个重试你不用跑 |
|
@anakinxc |
那试一下 |
|
@anakinxc 进入bazel-bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/ 发现并没有任何log信息 |
那如果你手动执行 |
|
麻烦从 yacl 的目录执行 不要 cd 到 fmtlib_foreign_cc |
|
@anakinxc 执行结果 rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory and dependencies directory for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify --sandbox_debug Bazel command line flag.
rules_foreign_cc: Printing build logs:
_____ BEGIN BUILD LOGS _____
Bazel external C/C++ Rules. Building library fmtlib
Environment:______________
SHELL=/bin/bash
COLORTERM=truecolor
JAVA_PATH=/usr/local/jdk/bin:/usr/local/jdk/jre/bin
TERM_PROGRAM_VERSION=1.88.1
CONDA_EXE=/home/lhy/.miniconda3/bin/conda
_CE_M=
BUILD_SCRIPT=bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/build_script.sh
LANGUAGE=zh_CN:en
LC_ADDRESS=zh_CN
JAVA_HOME=/usr/local/jdk:-:-:-
LC_NAME=zh_CN
JRE_HOME=/usr/local/jdk/jre
EXT_BUILD_ROOT=/home/lhy/MPC/git_test/yacl
JAVA_LD_LIBRARY_PATH=/home/lhy/.miniconda3/envs/yacl/lib/jvm/lib/server:-:-:-
BUILD_LOG=bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/CMake.log
LC_MONETARY=zh_CN
GOBIN=/home/lhy/gowork/bin
PWD=/home/lhy/MPC/git_test/yacl
LOGNAME=lhy
CONDA_ROOT=/home/lhy/.miniconda3
XDG_SESSION_TYPE=tty
CONDA_PREFIX=/home/lhy/.miniconda3/envs/yacl
VSCODE_GIT_ASKPASS_NODE=/home/lhy/.vscode-server/cli/servers/Stable-e170252f762678dec6ca2cc69aba1570769a5d39/server/node
MOTD_SHOWN=pam
HOME=/root
LANG=zh_CN.UTF-8
LC_PAPER=zh_CN
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
BUILD_WRAPPER_SCRIPT=bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/wrapper_build_script.sh
SSL_CERT_DIR=/usr/local/ssl/certs
CONDA_PROMPT_MODIFIER=(yacl)
GIT_ASKPASS=/home/lhy/.vscode-server/cli/servers/Stable-e170252f762678dec6ca2cc69aba1570769a5d39/server/extensions/git/dist/askpass.sh
EXT_BUILD_DEPS=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib.ext_build_deps
SSH_CONNECTION=10.10.101.138 51910 10.20.9.4 22
GOROOT=/usr/local/go
VSCODE_GIT_ASKPASS_EXTRA_ARGS=
LESSCLOSE=/usr/bin/lesspipe %s %s
XDG_SESSION_CLASS=user
LC_IDENTIFICATION=zh_CN
TERM=xterm-256color
_CE_CONDA=
LESSOPEN=| /usr/bin/lesspipe %s
USER=lhy
VSCODE_GIT_IPC_HANDLE=/run/user/1003/vscode-git-1b85ab0c72.sock
CONDA_SHLVL=6
BUILD_TMPDIR=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib.build_tmpdir
SHLVL=4
LC_TELEPHONE=zh_CN
LC_MEASUREMENT=zh_CN
XDG_SESSION_ID=10910
PAPERSIZE=a4
CONDA_PYTHON_EXE=/home/lhy/.miniconda3/bin/python
CLASSPATH=.:/usr/local/jdk/lib:/usr/local/jdk/jre/lib:
XDG_RUNTIME_DIR=/run/user/1003
SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt
SSH_CLIENT=10.10.101.138 51910 22
CONDA_DEFAULT_ENV=yacl
LC_TIME=zh_CN
INSTALLDIR=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib
VSCODE_GIT_ASKPASS_MAIN=/home/lhy/.vscode-server/cli/servers/Stable-e170252f762678dec6ca2cc69aba1570769a5d39/server/extensions/git/dist/askpass-main.js
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
BROWSER=/home/lhy/.vscode-server/cli/servers/Stable-e170252f762678dec6ca2cc69aba1570769a5d39/server/bin/helpers/browser.sh
PATH=/home/lhy/MPC/git_test/yacl:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus
MAIL=/var/mail/root
CONDA_PREFIX_1=/home/lhy/.miniconda3
CONDA_PREFIX_2=/home/lhy/.miniconda3/envs/yacl
CONDA_PREFIX_3=/home/lhy/.miniconda3/envs/sf
CONDA_PREFIX_4=/home/lhy/.miniconda3/envs/yacl
CONDA_PREFIX_5=/home/lhy/.miniconda3/envs/sf
LC_NUMERIC=zh_CN
OLDPWD=/home/lhy/MPC/git_test/yacl/bazel-bin
GOPATH=/home/lhy/gowork
TERM_PROGRAM=vscode
VSCODE_IPC_HOOK_CLI=/run/user/1003/vscode-ipc-3d20f800-b505-4ad1-9b54-220264e65f13.sock
_=/usr/bin/env
__________________________
Can not copy /home/lhy/MPC/git_test/yacl/cmake
Can not copy /home/lhy/MPC/git_test/yacl/ninja
+ cmake -DFMT_TEST=OFF -DCMAKE_TOOLCHAIN_FILE=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib.build_tmpdir/crosstool_bazel.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib -DCMAKE_PREFIX_PATH=/home/lhy/MPC/git_test/yacl/bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib.ext_build_deps -DCMAKE_RANLIB= -GNinja -G Ninja /home/lhy/MPC/git_test/yacl/external/com_github_fmtlib_fmt
CMake Error: The source directory "/home/lhy/MPC/git_test/yacl/external/com_github_fmtlib_fmt" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt/bin/external/com_github_fmtlib_fmt/fmtlib_foreign_cc/CMake.log |
|
有点离奇，麻烦 check 一下 bazel 的版本？ 跑一下 |
root@ubuntu:/home/lhy/MPC/git_test/yacl# bazel info
WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE.
Starting local Bazel server and connecting to it...
bazel-bin: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/execroot/yacl/bazel-out/k8-fastbuild/bin
bazel-genfiles: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/execroot/yacl/bazel-out/k8-fastbuild/bin
bazel-testlogs: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/execroot/yacl/bazel-out/k8-fastbuild/testlogs
character-encoding: file.encoding = ISO-8859-1, defaultCharset = ISO-8859-1
command_log: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/command.log
committed-heap-size: 2147MB
execution_root: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/execroot/yacl
gc-count: 3
gc-time: 54ms
install_base: /root/.cache/bazel/_bazel_lhy/install/20da5ab742b8d3d499c34fdafcd3c8b8
java-home: /root/.cache/bazel/_bazel_lhy/install/20da5ab742b8d3d499c34fdafcd3c8b8/embedded_tools/jdk
java-runtime: OpenJDK Runtime Environment (build 11.0.6+10-LTS) by Azul Systems, Inc.
java-vm: OpenJDK 64-Bit Server VM (build 11.0.6+10-LTS, mixed mode) by Azul Systems, Inc.
max-heap-size: 32178MB
output_base: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8
output_path: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/execroot/yacl/bazel-out
package_path: %workspace%
release: release 6.5.0
repository_cache: /root/.cache/bazel/_bazel_lhy/cache/repos/v1
server_log: /root/.cache/bazel/_bazel_lhy/8c9e699abe134235f623e685831bf7f8/java.log.ubuntu.root.log.java.20240417-105956.1372454
server_pid: 1372454
used-heap-size: 50MB
workspace: /home/lhy/MPC/git_test/yacl |
|
@anakinxc ERROR: /home/sch/git_test/yacl/yacl/crypto/aead/BUILD.bazel:30:13: Linking yacl/crypto/aead/gcm_crypto_test failed: (Exit 1): gcc failed: error executing command (from target //yacl/crypto/aead:gcm_crypto_test) /usr/bin/gcc @bazel-out/k8-opt/bin/yacl/crypto/aead/gcm_crypto_test-2.params
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_globallookup: error: undefined reference to 'dlopen'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_globallookup: error: undefined reference to 'dlsym'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_globallookup: error: undefined reference to 'dlclose'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_bind_func: error: undefined reference to 'dlsym'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_bind_func: error: undefined reference to 'dlerror'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_load: error: undefined reference to 'dlopen'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_load: error: undefined reference to 'dlclose'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_load: error: undefined reference to 'dlerror'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_pathbyaddr: error: undefined reference to 'dladdr'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_pathbyaddr: error: undefined reference to 'dlerror'
bazel-out/k8-opt/bin/external/com_github_openssl_openssl/openssl/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o):dso_dlfcn.c:function dlfcn_unload: error: undefined reference to 'dlclose'
collect2: error: ld returned 1 exit status
INFO: Elapsed time: 1.141s, Critical Path: 0.56s
INFO: 9 processes: 9 internal.
FAILED: Build did NOT complete successfully |
跟这个 issue #313 一模一样的问题，看来不是个例，我们看一下，issue 里面有个临时 fix 的方案你先用一下 |
|
我去复现一下试一试看看 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
os：ubuntu20.04
依赖配置参照：https://github.com/secretflow/devtools/blob/main/dockerfiles/ubuntu-base-ci.DockerFile
执行编译命令：
bazel build //... -c opt产生问题：
在此处一直卡住不动
The text was updated successfully, but these errors were encountered: