Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support build with Bazel #286

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
161 changes: 161 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Bazel doesn't need more than 200MB of memory for local build based on memory profiling:
# https://docs.bazel.build/versions/master/skylark/performance.html#memory-profiling
# The default JVM max heapsize is 1/4 of physical memory up to 32GB which could be large
# enough to consume all memory constrained by cgroup in large host.
# Limiting JVM heapsize here to let it do GC more when approaching the limit to
# leave room for compiler/linker.
# The number 2G is chosen heuristically to both support large VM and small VM with RBE.
# Startup options cannot be selected via config.
startup --host_jvm_args=-Xmx2g

run --color=yes

build --color=yes
build --workspace_status_command="bash bazel/get_workspace_status"
build --incompatible_strict_action_env
build --host_force_python=PY3
build --java_runtime_version=remotejdk_11
build --tool_java_runtime_version=remotejdk_11

build --enable_platform_specific_config

# Allow tags to influence execution requirements
common --experimental_allow_tags_propagation

# Enable position independent code (this is the default on macOS and Windows)
# (Workaround for https://github.com/bazelbuild/rules_foreign_cc/issues/421)
build:linux --copt=-fPIC
build:linux --copt=-Wno-deprecated-declarations
build:linux --cxxopt=-std=c++17
build:linux --conlyopt=-fexceptions
build:linux --fission=dbg,opt
build:linux --features=per_object_debug_info
build:linux --action_env=BAZEL_LINKLIBS=-l%:libstdc++.a
build:linux --action_env=BAZEL_LINKOPTS=-lm

# We already have absl in the build, define absl=1 to tell googletest to use absl for backtrace.
build --define absl=1

# Pass PATH, CC, CXX and LLVM_CONFIG variables from the environment.
build --action_env=CC
build --action_env=CXX
build --action_env=LLVM_CONFIG
build --action_env=PATH

# Common flags for sanitizers
build:sanitizer --define tcmalloc=disabled
build:sanitizer --linkopt -ldl
build:sanitizer --build_tag_filters=-no_san
build:sanitizer --test_tag_filters=-no_san

# Common flags for Clang
build:clang --action_env=BAZEL_COMPILER=clang
build:clang --action_env=CC=clang --action_env=CXX=clang++
build:clang --linkopt=-fuse-ld=lld

# Flags for Clang + PCH
build:clang-pch --spawn_strategy=local

# Basic ASAN/UBSAN that works for gcc
build:asan --config=sanitizer
# ASAN install its signal handler, disable ours so the stacktrace will be printed by ASAN
build:asan --define signal_trace=disabled
build:asan --copt -fsanitize=address,undefined
build:asan --linkopt -fsanitize=address,undefined
# vptr and function sanitizer are enabled in clang-asan if it is set up via bazel/setup_clang.sh.
build:asan --copt -fno-sanitize=vptr,function
build:asan --linkopt -fno-sanitize=vptr,function
build:asan --copt -DADDRESS_SANITIZER=1
build:asan --copt -D__SANITIZE_ADDRESS__
build:asan --test_env=ASAN_OPTIONS=handle_abort=1:allow_addr2line=true:check_initialization_order=true:strict_init_order=true:detect_odr_violation=1
build:asan --test_env=UBSAN_OPTIONS=halt_on_error=true:print_stacktrace=1
build:asan --test_env=ASAN_SYMBOLIZER_PATH
# ASAN needs -O1 to get reasonable performance.
build:asan --copt -O1
build:asan --copt -fno-optimize-sibling-calls

# Clang ASAN/UBSAN
build:clang-asan --config=clang
build:clang-asan --config=asan
build:clang-asan --linkopt -fuse-ld=lld
build:clang-asan --linkopt --rtlib=compiler-rt
build:clang-asan --linkopt --unwindlib=libgcc

# macOS
build:macos --cxxopt=-std=c++17
build:macos --action_env=PATH=/usr/bin:/bin:/opt/homebrew/bin:/usr/local/bin:/opt/local/bin
build:macos --host_action_env=PATH=/usr/bin:/bin:/opt/homebrew/bin:/usr/local/bin:/opt/local/bin
build:macos --define tcmalloc=disabled

# macOS ASAN/UBSAN
build:macos-asan --config=asan
# Workaround, see https://github.com/bazelbuild/bazel/issues/6932
build:macos-asan --copt -Wno-macro-redefined
build:macos-asan --copt -D_FORTIFY_SOURCE=0
# Workaround, see https://github.com/bazelbuild/bazel/issues/4341
build:macos-asan --copt -DGRPC_BAZEL_BUILD
# Dynamic link cause issues like: `dyld: malformed mach-o: load commands size (59272) > 32768`
build:macos-asan --dynamic_mode=off

# Clang TSAN
build:clang-tsan --config=sanitizer
build:clang-tsan --copt -fsanitize=thread
build:clang-tsan --linkopt -fsanitize=thread
build:clang-tsan --linkopt -fuse-ld=lld
build:clang-tsan --build_tag_filters=-no_san,-no_tsan
build:clang-tsan --test_tag_filters=-no_san,-no_tsan
# Needed due to https://github.com/libevent/libevent/issues/777
build:clang-tsan --copt -DEVENT__DISABLE_DEBUG_MODE
# https://github.com/abseil/abseil-cpp/issues/760
# https://github.com/google/sanitizers/issues/953
build:clang-tsan --test_env="TSAN_OPTIONS=report_atomic_races=0"

# Clang MSAN - this is the base config for remote-msan and docker-msan. To run this config without
# our build image, follow https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo
# with libc++ instruction and provide corresponding `--copt` and `--linkopt` as well.
build:clang-msan --config=sanitizer
build:clang-msan --copt -fsanitize=memory
build:clang-msan --linkopt -fsanitize=memory
build:clang-msan --linkopt -fuse-ld=lld
build:clang-msan --copt -fsanitize-memory-track-origins=2
build:clang-msan --test_env=MSAN_SYMBOLIZER_PATH
# MSAN needs -O1 to get reasonable performance.
build:clang-msan --copt -O1
build:clang-msan --copt -fno-optimize-sibling-calls

# Clang with libc++
build:libc++ --config=clang
build:libc++ --action_env=CXXFLAGS=-stdlib=libc++
build:libc++ --action_env=LDFLAGS=-stdlib=libc++
build:libc++ --action_env=BAZEL_CXXOPTS=-stdlib=libc++
build:libc++ --action_env=BAZEL_LINKLIBS=-l%:libc++.a:-l%:libc++abi.a
build:libc++ --action_env=BAZEL_LINKOPTS=-lm:-pthread
build:libc++ --define force_libcpp=enabled

# Optimize build for binary size reduction.
build:sizeopt -c opt --copt -Os

# Test options
build --test_env=HEAPCHECK=normal --test_env=PPROF_PATH

# Coverage options
coverage --config=coverage
coverage --build_tests_only
build:coverage --action_env=BAZEL_USE_LLVM_NATIVE_COVERAGE=1
build:coverage --action_env=GCOV=llvm-profdata
build:coverage --copt=-DNDEBUG
# 1.5x original timeout + 300s for trace merger in all categories
build:coverage --test_timeout=390,750,1500,5700
build:coverage --define=dynamic_link_tests=true
build:coverage --test_env=HEAPCHECK=
build:coverage --combined_report=lcov
build:coverage --strategy=TestRunner=sandboxed,local
build:coverage --strategy=CoverageReport=sandboxed,local
build:coverage --experimental_use_llvm_covmap
build:coverage --collect_code_coverage
build:coverage --test_tag_filters=-nocoverage
build:coverage --instrumentation_filter="//src[/:],//include[/:]"
build:test-coverage --test_arg="-l trace"
build:fuzz-coverage --config=plain-fuzzer

test --enable_runfiles=True
31 changes: 31 additions & 0 deletions .github/workflows/bazel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: build-with-bazel

on:
# Trigger the workflow on push or pull request,
# but only for the main branch
push:
branches:
- master
- bazel

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove bazel branch.

pull_request:
branches:
- master
- bazel

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove


jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- uses: actions/checkout@v2

- name: Setup Bazel
run: |
sudo apt-get -qq install npm

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to pin the bazel version.

sudo npm install -g @bazel/bazelisk
bazel -h
- name: Build All Targets
run: bazel build -c dbg //...
- name: Run All Tests
run: bazel test -c dbg //tests/...
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,11 @@ nbproject/
*.pdb
*.exe
*.ilk
*.lib
*.lib
.cache
cmake-build-debug
compile_commands.json
bazel-bin
bazel-out
bazel-testlogs
bazel-NuRaft
26 changes: 26 additions & 0 deletions BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "api",
hdrs = glob(["include/**/*.hxx"]),
visibility = ["//visibility:public"],
strip_include_prefix = "//include/libnuraft",
)

cc_library(
name = "nuraft",
hdrs = glob(["src/*.hxx"]) + glob(["src/*.h"]),
srcs = glob(["src/*.cxx"]),
strip_include_prefix = "//src",
deps = [
":api",
"@asio//:asio",
"@boringssl//:ssl",
],
visibility = ["//visibility:public"],
)

exports_files([
"key.pem",
"cert.pem",
])
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,26 @@ C:\NuRaft\build> c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\V
```


Build with Bazel
----------
#### 1. Install Bazel
Follow [install instructions](https://docs.bazel.build/versions/5.0.0/install.html).
LTS versions(4.2.2, 5.0.0) are verified compatible. Bazelisk is preferred as it helps [manage Bazel versions](https://docs.bazel.build/versions/main/updating-bazel.html#managing-bazel-versions-with-bazelisk).

#### 2. Build static/shared libraries
```sh
bazel build -c opt //:nuraft
```
If debug version is desirable, run
```sh
bazel build -c dbg //:nuraft
```

#### 3. Run unit tests
```sh
bazel test -c dbg //tests/...
```

How to Use
----------
Please refer to [this document](./docs/how_to_use.md).
Expand Down
5 changes: 5 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
workspace(name = "com_github_ebay_nuraft")


load("@com_github_ebay_nuraft//bazel:repository.bzl", "nuraft_deps")
nuraft_deps()
Empty file added bazel/BUILD
Empty file.
36 changes: 36 additions & 0 deletions bazel/get_workspace_status
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/bin/bash

# This file was imported from https://github.com/bazelbuild/bazel at d6fec93.

# This script will be run bazel when building process starts to
# generate key-value information that represents the status of the
# workspace. The output should be like
#
# KEY1 VALUE1
# KEY2 VALUE2
#
# If the script exits with non-zero code, it's considered as a failure
# and the output will be discarded.
# If this SOURCE_VERSION file exists then it must have been placed here by a
# distribution doing a non-git, source build.
# Distributions would be expected to echo the commit/tag as BUILD_SCM_REVISION
if [ -f SOURCE_VERSION ]
then
echo "BUILD_SCM_REVISION $(cat SOURCE_VERSION)"
echo "STABLE_BUILD_SCM_REVISION $(cat SOURCE_VERSION)"
echo "BUILD_SCM_STATUS Distribution"
exit 0
fi

# The code below presents an implementation that works for git repository
git_rev=$(git rev-parse HEAD) || exit 1
echo "BUILD_SCM_REVISION ${git_rev}"
echo "STABLE_BUILD_SCM_REVISION ${git_rev}"

# Check whether there are any uncommitted changes
tree_status="Clean"
git diff-index --quiet HEAD -- || {
tree_status="Modified"
}
echo "BUILD_SCM_STATUS ${tree_status}"
echo "STABLE_BUILD_SCM_STATUS ${tree_status}"
28 changes: 28 additions & 0 deletions bazel/repository.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")

def nuraft_deps():
maybe(
http_archive,
name = "asio",
sha256 = "4cd5cd0ad97e752a4075f02778732a3737b587f5eeefab59cd98dc43b0dcadb3",
urls = [
"https://shutian.oss-cn-hangzhou.aliyuncs.com/cdn/asio/asio-1.20.0.tar.gz",
],
strip_prefix = "asio-1.20.0",
build_file = "@com_github_ebay_nuraft//third_party:asio.BUILD",
)

maybe(
http_archive,
name = "boringssl",
# Use github mirror instead of https://boringssl.googlesource.com/boringssl
# to obtain a boringssl archive with consistent sha256
sha256 = "534fa658bd845fd974b50b10f444d392dfd0d93768c4a51b61263fd37d851c40",
strip_prefix = "boringssl-b9232f9e27e5668bc0414879dcdedb2a59ea75f2",
urls = [
"https://shutian.oss-cn-hangzhou.aliyuncs.com/cdn/boringssl/boringssl-b9232f9e27e5668bc0414879dcdedb2a59ea75f2.tar.gz",
"https://storage.googleapis.com/grpc-bazel-mirror/github.com/google/boringssl/archive/b9232f9e27e5668bc0414879dcdedb2a59ea75f2.tar.gz",
"https://github.com/google/boringssl/archive/b9232f9e27e5668bc0414879dcdedb2a59ea75f2.tar.gz",
],
)
32 changes: 32 additions & 0 deletions cert.pem
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
-----BEGIN CERTIFICATE-----

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We (eBay security dept) will not allow to include any certificate file in the public repo, even though it is just a dummy. Is it possible to add a script like this?

NuRaft/CMakeLists.txt

Lines 199 to 213 in cce3af7

add_custom_command(
TARGET build_ssl_key
PRE_BUILD
COMMAND
openssl req
-new
-newkey rsa:4096
-days 365
-nodes
-x509
-subj "/C=AB/ST=CD/L=EFG/O=ORG/CN=localhost"
-keyout ${CMAKE_CURRENT_BINARY_DIR}/key.pem
-out ${CMAKE_CURRENT_BINARY_DIR}/cert.pem
2> /dev/null
)

MIIFdTCCA12gAwIBAgIULIMoUtM/TIzRdVT5NUCDHoNac+AwDQYJKoZIhvcNAQEL
BQAwSjELMAkGA1UEBhMCQUIxCzAJBgNVBAgMAkNEMQwwCgYDVQQHDANFRkcxDDAK
BgNVBAoMA09SRzESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIyMDIwODA2MjAyNloX
DTIzMDIwODA2MjAyNlowSjELMAkGA1UEBhMCQUIxCzAJBgNVBAgMAkNEMQwwCgYD
VQQHDANFRkcxDDAKBgNVBAoMA09SRzESMBAGA1UEAwwJbG9jYWxob3N0MIICIjAN
BgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAqDQc/RJn4nPWtLMdZ/53XIT8e6nQ
m/ydoK7RzlyR/gwAQMbBAsP1KnjTPeY6mdMwLkQ8Xas6SfwhtXM4hGIkoeA29S1J
5NhMBOr7joh4LZ0P9n1h1yLZC8zTioAKYOCVyq9Z1Of1mlqKFl6irDnmMxzLgCl2
6a9boR0lnl/q5gpbEBI9rlWNhKFlRxzZ1GXZQXZZf0DfwZIrAvSDMoRt2oCUMdNT
iCUybq8r+USfa/dCjtxZUia8Zkb4kV1Zm1ATL8RojeDIBz/7R3uM44gVh4i8uGJM
qB/RjjWN+2jaqOR+1v+JGFE9RGWDtvGnHtr4SQzh9RgJPwS65UGTioE5S/H5v3qP
fQdS2qboUeaApzbb6ZbnmKSfJxIHup4/mHiHXlDphbgI7MRn6voRLqG6QsNt4BGK
WxZQFHn0DRW0XS0YQMgcximw1RGy1d4350cX2LHPIRQWQSA11NX/UWzAHavRpKBQ
eA2pyawy2u2wqnP0zjZEHaTQqBC8XYsxCf6NQitbJ4tEzYG0tFWQj/BbF4gGiQn2
VPCp7eXqhj5/nwNzmeYhUvAitxvE9MP2qfoCwKGErLkwSLMdgBgGlCLjyo2DNwfO
WAGGRbKzzcdfVlzImtYOfclScv/CUrRNenIOmZCUAyZqEXSWy/gR7TF+o1BJCXFs
j9DhdS+kx7s1s80CAwEAAaNTMFEwHQYDVR0OBBYEFJbhLDc/1J87grw9YiLAVWcI
FoC0MB8GA1UdIwQYMBaAFJbhLDc/1J87grw9YiLAVWcIFoC0MA8GA1UdEwEB/wQF
MAMBAf8wDQYJKoZIhvcNAQELBQADggIBAA+ZVfWvBDsP6MfLQmWDPRhNgZ+ruLfG
v135hLoLmC9TU0/2LjpzLrl+RU3375xDtg6ZtsNJQxdSVxQzW6e1tD/7pbNSbjn2
Uc0noBo0v3+f5gHpL17RjthYEQiXDOcpwloUyE7833c/knozVi78R+BvF/3bpL4m
47c8nEF6erRkw7fxtJT1I+ndFAVbO5+uWHYr7FBRJh8U6o49wgSZaOuC5+00IoAH
DXEvvYjf7EwbLlbTMNkBO0Q40PA84GAAloKb2l55hKhWaC3v9+lAzO6DRfbCM3JU
sAwcNzz2LjkfQkwm9iEJh2ihqLZKF1IvqjO0DZVj8N0JlBz5CFo7kugiHVXkgT97
4O4IcwIkO4e/oxZoXuhT5A2F6/I73axzUhf0++/yIC4AHXc16lL5gNqlv9BSSiX4
uTygHncW7ccFiZBhntUdzlCPdvoxxITjCvGRgOiMSmUZGncFf4QLm+qLnJxwuR8a
w+0WnrVKQCkAGaDqe4ceu6Yi07UtvHpV5s+zn3lYKGDUdfUJdU5wxGvC7j4Gsc+X
BhD+HCsWSOSIHL3MgJjz9lGldbUilHDySFAl+uDi/t9jjPDTz6OBHguRsSowQUxF
9Vg2gdEMzO78lDzrue9I1ylhcLYiyCAfxmV+T7ZtlztHzQ4Hw3A8Y/dTvfo0LrJ3
KdIYJ7m7Ti/t
-----END CERTIFICATE-----