The pbf Go package implements an interpreter for a custom bytecode format. A bytecode program defines a filter for a Protocol Buffers (protobuf) message type. It can be successively applied to protobuf message data, yielding boolean results.
The application doing the filtering doesn't need any prior information about the specific message types, as the bytecode includes all necessary details. In other words, the application doesn't need to include code generated by the protobuf compiler.
Conversely, when constructing the bytecode program, message type information is necessary. The field and op Go packages, and the pbf Python module contain definitions helpful to bytecode generators, but comprehensive program construction tools are not provided by this project.
The test code includes a bytecode program example.
Filtering using PBF (BenchmarkFilter) is faster than using generated Go code (BenchmarkProtocGo), but slower than C++ (BenchmarkProtocCXX):
goos: linux
goarch: amd64
pkg: github.com/ninchat/pbf
cpu: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
BenchmarkPrepare 1212697 4905 ns/op 6532 B/op 19 allocs/op
BenchmarkFilter 4460156 1348 ns/op 148.39 MB/s 0 B/op 0 allocs/op
BenchmarkProtocGo 2612586 2299 ns/op 86.99 MB/s 776 B/op 26 allocs/op
BenchmarkProtocCXX 6486204 898.1 ns/op 222.70 MB/s
Official protoc 3.6.1 and protoc-gen-go 1.27.1 were used for C++ and Go code generation.
PBF doesn't add pressure on the garbage collector due to being virtually allocation-free. It needs to allocate maps to keep track of unpacked repeated fields, but they are shared between iterations.
BenchmarkPrepare parses and verifies the filter bytecode. It needs to be done only once per program (message type).
Developed by Ninchat Terveystalo.