Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BPF Masquerade test #2384

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

tommyp1ckles
Copy link
Contributor

@tommyp1ckles tommyp1ckles commented Mar 11, 2024

Adds test coverage for bpf-masquerade (as well as any other IP masquerading mode) via connectivity tests. Currently this is done via Ginkgo runtime tests in cilium/cilium.

To accommodate this, and to improve overall test debugging this also replaces the normal json-mock based echo server implementation, used by connectivity tests, with a Go based implementation.

Currently, the node based json-mock server is used for creating test servers for running various tests.
Unfortunately this has some shortcomings, namely:

  • Mock image is built outside of CLI repo thus making it difficult to make changes to.
  • JSON Mock is designed to provide a server that returns some static schema of data. Cilium connectivity tests don't really this functionality, (we only really need to define some extra http routes to test route based policy). Instead a more flexible implementation would make it easier to adapt to future use cases.
  • JSON Mock doesn't log on the index server as it's not really part of the schema. Better logging from the servers would be useful for diagosing test l7 failures/flakes (ex. CI: Conformance E2E: client-egress-l7-named-port/pod-to-pod: command terminated with exit code 28 (timeout) cilium#27762) as it would let us correlate timestamped events on the server side.

This adds a new standalone server called "echoserver" which is a small Go http server with the following routes:

GET / -> Index route, used for general l4/l7 connectivity testing.
GET /private -> Private route, used for l7 network policy.
GET /public -> Private route, used for l7 network policy.
GET /echo -> Responds with a json payload the remote address (i.e. :<client_port>) and timestamp.

These routes will cover existing test cases, with the latter being used to implement a bpf masquerade connectivity test.

This will be used for implementing bpf-masquerade tests in subsequent commits.

Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>
The new echoserver server test image will echo back the remote client ip when requesting GET /echo.
When running a node without Ciliu, we use this to implement a masquerading connectivity test to curl
out to the external node to find out if the correct client remote IP was used.

Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>
Currently, the node based json-mock server is used for creating test servers for running various tests.
Unfortunately this has some shortcomings, namely:

* Mock image is built outside of CLI repo thus making it difficult to make changes to.
* JSON Mock is designed to provide a server that returns some static schema of data. Cilium connectivity tests don't really this functionality, (we only really need to define some extra http routes to test route based policy). Instead a more flexible implementation would make it easier to adapt to future use cases.
* JSON Mock doesn't log on the index server as it's not really part of the schema. Better logging from the servers would be useful for diagosing test l7 failures/flakes (ex. cilium/cilium#27762) as it would let us correlate timestamped events on the server side.

This adds a new standalone server called "echoserver" which is a small Go http server with the following routes:

GET / -> Index route, used for general l4/l7 connectivity testing.
GET /private -> Private route, used for l7 network policy.
GET /public -> Private route, used for l7 network policy.
GET /echo -> Responds with a json payload the remote address (i.e. <ip>:<client_port>) and timestamp.

These routes will cover existing test cases, with the latter being used to implement a bpf masquerade connectivity test.

Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>
Previous commits introduced a Go based implementation of the echoserver.
This also provides more detailed request logging, this may be useful when debugging connectivity tests.
This adds a set of sysdump steps that are run when invoked from 'cilium connectivity test' which will dump logs from test namespace echo server Pods (i.e. label:kind=echo).

Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>
@brb
Copy link
Member

brb commented Mar 12, 2024

Neat! We could extend this test to cover not only BPF masq, but also the connectivity to outside. Currently, by default, we use 1.1.1.1 as the endpoint to outside, which is flaky. We could use that echo server as the endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants