Skip to content

Build System

Jump to bottom
atnnn edited this page Mar 2, 2013 · 1 revision

Rewrite

The old Makefile used to build rethinkdb was very long, complicated and included a lot of extra functionality than just building the rethinkdb executable. The goal of the new build system is to be easier to use, understand and modify.

Split

The Makefile is now split into different sections, each section in its own file. These files are organized as follows:

  • Makefile -- The top-level Makefile. It delegates to mk/main.mk
  • configure -- A shell script.
  • src/build.mk -- Rules for building the rethinkdb executable.
  • src/Makefile -- Sets TOP = .. and includes the top-level Makefile
  • mk/main.mk -- This Makefile simply includes all the other Makefiles.
  • mk/support.mk -- Download and build the toolchain (node, v8, ...).
  • mk/install.mk -- make install
  • mk/packaging.mk -- make deb, make osx
  • mk/webui.mk -- make web-assets
  • ...

The configure script

The configure script creates the config.mk file. The current output of ./configure --help is:

Configure a RethinkDB build from source

Usage: ./configure [arguments]

  --help                  Display this help
  --config <file>         Output file (default config.mk)
  --continue              Do not stop after encountering an error

  --allow-fetch           Allow fetching missing dependencies
  --lib-path <dir>        Add dir to the library search path
  <var>=<val>             Set the value of a variable
                            CXXFLAGS  C++ compiler arguments
                            LDFLAGS   C++ linker arguments
  --force-fetch <dep>     Force fetching <dep>.
  <DEP>=<path>            Library or executable path. <dep> can be
                            cxx                      C++ Compiler
                            protoc                   Protobuf compiler
                            lessc                    LESS css
                            coffee                   CoffeScript
                            handlebars               Handlebars
                            npm                      Node.js package manager
                            boost_program_options    
                            protobuf                 
                            v8                       
                            unwind                   libunwind
                            tcmalloc_minimal         
  --static <lib>          Statically link some libraries. <lib> is a library <dep> or one of:
                            all          All libraries
                            recommended  Highly recommended libraries
                            default      Recommended libraries
                            none         No static libraries
  --prefix <dir>          Installation prefix. Defaults to /usr/local
  --sysconfdir <dir>      Configuration prefix. Defaults to /usr/local/etc
  --localstatedir <dir>   Runtime data prefix. Defaults to /usr/local/var
  --with-<module>
  --without-<module>      Enable or disable <module>. <module> can be one of
                            tcmalloc     Build with tcmalloc. (Default: enabled)
  --enable-<option>
  --disable-<option>      Enable or disable a build option
                            drivers          Build the client drivers (default: true)
                            precompiled-web  Use precompiled web assets located in precompiled/web (default: false)

Customizing the build

All variables that can be used to customize the build are listed and documented in mk/way/default.mk.

These variables can be changed on the command line. For example:

$ make DEBUG=1

Build settings local to the repository can be stored in the custom.mk file at the root of the repository.

Shared build configurations, called ways, are stored in mk/way. The WAY variable determines which way to use. For example, the osx way builds a portable, static osx package.

$ make WAY=osx

Useful features

When SHOW_COUNTDOWN=1 (the default), the makefile will print the fraction of remaining files left to build:

$ make
    [1/110] CC src/arch/io/disk.cc -o build/debug/obj/arch/io/disk.o
    [2/110] CC src/buffer_cache/mirrored/patch_memory_storage.cc -o build/debug/obj/buffer_cache/mirrored/patch_memory_storage.o

When a variable not in mk/way/default.mk is used, a warning gets printed:

$ make DEBUF=1
Makefile:47: Possibly unknown variables: DEBUF

Make can be called from any folder:

$ make src/clustering/Makefile
    [1/1] ECHO > src/clustering/Makefile
$ cd src/clustering
$ make
    [1/167] CC ../../src/arch/runtime/event_queue.cc -o ../../build/release/obj/arch/runtime/event_queue.o

Writing rules and recipes

The new Makefile uses the non-recursive make idiom and is located at the root of the source tree.

Although the makefiles are separate, they all share the same namespace. Variables should have unique, descriptive names.

All filenames should start with $(TOP), which is the path to the root of the source tree.

The new $P macro replaces the old VERBOSE/QUIET pattern and makes rules cleaner. All recipes should include a $P .

%.foo: %.bar
    $P QUUX "$@ -> $<"
    quux $< -o $@

By convention, a directory that previously had a Makefile would now have both a Makefile and a build.mk file. The Makefile is automatically generated. It simply sets TOP and includes $(TOP)/Makefile. The build.mk simply contains the make recipes and is included from mk/main.mk.

For example, src/Makefile and src/build.mk. src/Makefile was automatically generated by doing

$ make src/Makefile

Variables

Some of the variables available when writing recipes are:

  • TOP - The path to the top of the source tree

The configure file creates a variable for each binary and library it finds. For example:

  • PROTOC - The protoc binary
  • ...

Some paths are defined in mk/paths.mk

  • EXTERNAL_DIR - $(TOP)/external
  • ...

Build parameters set by the developer in custom.mk or on the command line are documented in mk/default.mk

Clone this wiki locally