Skip to content

Commit

Permalink
Convert WORKSPACE to MODULE.bazel, update README (#312)
Browse files Browse the repository at this point in the history
* Enable bzlmod, update README, fix Swift build

Rather than mess with the WORKSPACE rules, the shortest path to fixing
`blaze build //swift:tests` appeared to be introducing MODULE.bazel.

MODULE.bazel, a.k.a. bzlmod, appears to be the new hotness, so I'll also
try updating the other builds to use it as well.

- https://bazel.build/external/migration
- https://bazel.build/external/overview#workspace-shortcomings
- bazelbuild/bazel#18958

* Convert `bazel build //python/...` to MODULE.bazel

* Convert `bazel build //csharp/...` to MODULE.bazel

* Convert //go/..., gazelle to MODULE.bazel

* Migrate protobuf, rules_proto to MODULE.bazel

Also bumps to:

- protobuf: 23.1
- rules_proto: 6.0.0-rc2

* Add rules_java, rules_jvm_external to MODULE.bazel

rules_java was an implicit dependency before.

Bumps:

- rules_java: 7.4.0 => 7.5.0.
- rules_jvm_external: 4.4.2 => 6.0

The rules_jvm_external docs describe using `maven.install` in
MODULE.bazel as a replacement for `maven_install` in WORKSPACE:

- https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/bzlmod.md

However, our current `maven_install` depends on the definition of
IO_GRPC_GRPC_JAVA_ARTIFACTS from @io_grpc_grpc_java. I'll attempt that
migration next.

* MODULE.bazel: skylib, rules_proto_grpc, protobuf

`bazel build //java/...` and `bazel test //java/...` both work with
these changes.

* Move grpc-java to MODULE.bazel, bump to 1.62.2

grpc-java only got MODULE.bazel support as of this most recent version:

- grpc/grpc-java#11046
- bazelbuild/bazel-central-registry#1699

This grpc-java version bump exposed two issues that are fixed in this
commit:

1. The //java/com/engflow/notificationqueue:client target dependency on
   @maven//:io_netty_netty_handler broke.

   The original WORKSPACE import of io_grpc_grpc_java imported this
   dependency directly by passing IO_GRPC_GRPC_JAVA_ARTIFACTS directly
   to `maven_install`. The `maven.install` call from grpc/grpc-java's
   MODULE.bazel sets `strict_visibility = True`. Somehow the other
   dependencies registered by grpc-java's MODULE.bazel
   are accessible to notificationqueue:client, but netty-handler isn't.

   The solution was to add the `io.netty:netty-handler:4.1.100.Final`
   artifact to the `maven.install` call in this project's MODULE.bazel.
   It doesn't seem an optimal solution, but it works for now.

2. grpc/grpc-java removed `io.grpc.stub.MetadataUtils.attachHeaders()`
   in grpc/grpc-java#10443.

   This caused notificationqueue:client to fail to compile, but that PR
   revealed the replacement for the deprecated `attachHeaders` call.
   This commit applies that replacement.

* Move googleapis to MODULE.bazel

This appears to be a fairly recent development, and isn't yet 100%
officially supported in the googleapis/googleapis repo, but it works:

- googleapis/googleapis#855
- bazelbuild/bazel-central-registry#1699

* Move rules_kotlin to MODULE.bazel, bump to v1.9.5

* Move rules_perl to MODULE.bazel, bump to 0.2.0

There's actually a 0.2.1 release, but it hasn't been pushed to
https://registry.bazel.build/ yet.

* Add rules_scala GitHub issue links

rules_scala hasn't migrated to bzlmod yet, but discussion is underway.
These links will help track its progress.

* Migrate aspect_rules_ts to MODULE.bazel

* Replace deps.bzl with go_deps

This enables `bazel {build,test} //infra/...` to succeed using
MODULE.bazel. See:

- https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md#specifying-external-dependencies

* Set test sizes to "small" as appropriate

This eliminates warnings that these tests are sized too big, since the
default is "medium".

* Move http_{file,archive} calls to MODULE.bazel

This moves the following http_file calls:

- emacs
- ubuntu_20.04_1.3GB

and the following http_archive calls:

- com_engflow_engflowapis
- io_abseil_py

* Update //platform rules, remove dotnet constraints

The //dotnet/toolchain constraint was causing a Bazel error saying that
the @@rules_dotnet//dontent/toolchain package didn't exist. Removing
this constraint allowed remote execution to succeed anyway.

* Update README, explain swift incompatibility

Most of these changes are cosmetic, with the notable exception of the
explantion behind the inability to build //swift remotely.

Also added a `git` command to ignore python/requirements_lock.txt per:

- https://stackoverflow.com/a/73720550
  • Loading branch information
mbland committed Apr 15, 2024
1 parent abfc7d4 commit c9a30c2
Show file tree
Hide file tree
Showing 21 changed files with 15,928 additions and 422 deletions.
1 change: 0 additions & 1 deletion .bazelrc
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Options common for all EngFlow remote configurations.
common --@aspect_rules_ts//ts:skipLibCheck=honor_tsconfig
common --@aspect_rules_ts//ts:default_to_tsc_transpiler
common --noenable_bzlmod
common --noincompatible_check_sharding_support

build:engflow_common --jobs=40
Expand Down
2 changes: 1 addition & 1 deletion BUILD
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
load("@bazel_gazelle//:def.bzl", "gazelle")
load("@gazelle//:def.bzl", "gazelle")

gazelle(name = "gazelle")

Expand Down
170 changes: 170 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
"""EngFlow example project"""

module(name = "engflow-example", version = "0.0.0")
http_file = use_repo_rule(
"@bazel_tools//tools/build_defs/repo:http.bzl", "http_file"
)
http_archive = use_repo_rule(
"@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive"
)

# Some file dependencies
http_file(
name = "emacs",
sha256 = "1439bf7f24e5769f35601dbf332e74dfc07634da6b1e9500af67188a92340a28",
urls = [
"https://storage.googleapis.com/engflow-tools-public/emacs-28.1.tar.gz",
"https://mirror.its.dal.ca/gnu/emacs/emacs-28.1.tar.gz",
"https://mirrors.kernel.org/gnu/emacs/emacs-28.1.tar.gz",
],
)

http_file(
name = "ubuntu_20.04_1.3GB",
sha256 = "5035be37a7e9abbdc09f0d257f3e33416c1a0fb322ba860d42d74aa75c3468d4",
urls = [
"https://storage.googleapis.com/engflow-tools-public/ubuntu-20.04.5-live-server-amd64.iso",
"https://mirror.math.princeton.edu/pub/ubuntu-iso/focal/ubuntu-20.04.5-live-server-amd64.iso",
"https://mirror.pit.teraswitch.com/ubuntu-releases/focal/ubuntu-20.04.5-live-server-amd64.iso",
],
)

http_archive(
name = "com_engflow_engflowapis",
sha256 = "8721f7a0ec52c5bc120119aac090eedd671ca3b708652f88b82b44bea2b6c278",
strip_prefix = "engflowapis-44fcd39598f223e8e5f6c7cbf2f73c870b2a6341",
urls = [
"https://github.com/EngFlow/engflowapis/archive/44fcd39598f223e8e5f6c7cbf2f73c870b2a6341.zip",
],
)

# Abseil Python can be imported through pip_import, but it has native Bazel support too.
# bzlmod: https://github.com/abseil/abseil-py/issues/263
http_archive(
name = "io_abseil_py",
sha256 = "8a3d0830e4eb4f66c4fa907c06edf6ce1c719ced811a12e26d9d3162f8471758",
strip_prefix = "abseil-py-2.1.0",
url = "https://github.com/abseil/abseil-py/archive/refs/tags/v2.1.0.tar.gz",
)

bazel_dep(name = "bazel_skylib", version = "1.5.0")
bazel_dep(
name = "protobuf",
version = "23.1",
repo_name = "com_google_protobuf",
)

# https://github.com/googleapis/googleapis/pull/855
# https://github.com/bazelbuild/bazel-central-registry/pull/1699
bazel_dep(
name = "googleapis",
version = "0.0.0-20240326-1c8d509c5",
repo_name = "com_google_googleapis",
)

bazel_dep(name = "rules_python", version = "0.31.0")

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")

pip.parse(
hub_name = "pip",
python_version = "3.11",
requirements_lock = "//python:requirements_lock.txt",
)

use_repo(pip, "pip")

bazel_dep(name = "rules_proto", version = "6.0.0-rc2")

bazel_dep(
name = "apple_support",
version = "1.15.1",
repo_name = "build_bazel_apple_support",
)

bazel_dep(name = "rules_dotnet", version = "0.15.1")

dotnet = use_extension("@rules_dotnet//dotnet:extensions.bzl", "dotnet")
dotnet.toolchain(dotnet_version = "8.0.200")
use_repo(dotnet, "dotnet_toolchains")

register_toolchains("@dotnet_toolchains//:all")

# https://github.com/bazelbuild/rules_go/blob/master/docs/go/core/bzlmod.md
bazel_dep(name = "rules_go", version = "0.46.0")
bazel_dep(name = "gazelle", version = "0.36.0")

GO_PLATFORMS = [
("darwin", "amd64"),
("darwin", "arm64"),
("linux", "amd64"),
("linux", "arm64"),
("windows", "amd64"),
]

GO_VERSION = "1.21.6"

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")

[
go_sdk.download(
name = "go_{}_{}".format(goos, goarch),
goarch = goarch,
goos = goos,
version = GO_VERSION,
)
for goos, goarch in GO_PLATFORMS
]

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")

use_repo(go_deps, "com_github_google_go_cmp")

bazel_dep(name = "rules_java", version = "7.5.0")

# https://github.com/bazelbuild/rules_jvm_external/blob/master/docs/bzlmod.md
bazel_dep(name = "rules_jvm_external", version = "6.0")
bazel_dep(
name = "grpc-java",
version = "1.62.2",
repo_name = "io_grpc_grpc_java",
)

# Loads rules required to compile proto files
bazel_dep(name = "rules_proto_grpc", version = "5.0.0-alpha2")

maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")
maven.install(
artifacts = [
"commons-cli:commons-cli:1.5.0",
"com.google.oauth-client:google-oauth-client:1.34.1",
"io.netty:netty-handler:4.1.100.Final",
],
strict_visibility = True,
repositories = [
"https://repo.maven.apache.org/maven2/",
],
)
use_repo(maven, "maven")

bazel_dep(name = "rules_kotlin", version = "1.9.5")
bazel_dep(name = "rules_perl", version = "0.2.0")

bazel_dep(
name = "rules_swift",
version = "1.18.0",
repo_name = "build_bazel_rules_swift"
)

bazel_dep(name = "aspect_rules_ts", version = "2.2.0")

rules_ts_ext = use_extension(
"@aspect_rules_ts//ts:extensions.bzl",
"ext",
dev_dependency = True,
)

rules_ts_ext.deps()

use_repo(rules_ts_ext, "npm_typescript")

0 comments on commit c9a30c2

Please sign in to comment.