libvirt: e2e test for attestation for sample tee #1824

huoqifeng · 2024-04-28T07:37:47Z

This PR is based on the azure approach: #1735

Add AA_KBC_PARAMS in libvirt provider
Wait kbs deployment ready before retrieve the endpoint from it
Add test case for attestation secret
Revise a little for the caa ds waiting, as pod ready is enough as the condition
Change the KBS opa policy to allow_all

huoqifeng · 2024-04-30T07:23:57Z

Created a cluster and run e2e test like below:

Make sure file id_rsa exists like below: (which is created when creating the k8s cluster)

# ls /root/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/install/overlays/libvirt/
cri_runtime_endpoint.yaml  id_rsa  id_rsa.pub  kustomization.yaml  tls_certs_volume_mount.yaml

Created a properties file /root/libvirt.properties like below:
libvirt_uri="qemu+ssh://root@192.168.122.1/system?no_verify=1"
libvirt_ssh_key_file="id_rsa"
CLUSTER_NAME = "peer-pods"
KBS_IMAGE = "ghcr.io/confidential-containers/staged-images/kbs"
KBS_IMAGE_TAG = "dc01f454264fb4350e5f69eba05683a9a1882c41"
Clone trustee and build kbs-client

pushd test/e2e
git clone https://github.com/confidential-containers/trustee.git
pushd trustee
git checkout 65ee7e1acccd13dcb515058e71c5f8bfb4281e35
pushd kbs
make cli
popd
popd
popd

Note, we got trustee/target/release/kbs-cli

Run commamd:

export TEST_PROVISION_FILE="/root/libvirt.properties"
export CLOUD_PROVIDER=libvirt
export DEPLOY_KBS=true
export TEE_CUSTOMIZED_OPA="/root/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e/trustee/kbs/sample_policies/allow_all.rego"
export TEST_INSTALL_CAA=yes
export TEST_TEARDOWN=no
make test-e2e

Result:

# make test-e2e
go test -v -tags=libvirt -timeout 60m -count=1 ./test/e2e
time="2024-04-30T07:16:00Z" level=info msg="Do setup"
time="2024-04-30T07:16:00Z" level=info msg="Deploying kbs"
time="2024-04-30T07:16:00Z" level=info msg="creating key.bin"
/root/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e
time="2024-04-30T07:16:00Z" level=info msg="Creating kbs install overlay"
time="2024-04-30T07:16:00Z" level=info msg="Customize the overlay yaml file"
time="2024-04-30T07:16:00Z" level=info msg="Updating kbs image with \"ghcr.io/confidential-containers/staged-images/kbs\""
time="2024-04-30T07:16:00Z" level=info msg="Updating kbs image tag with \"dc01f454264fb4350e5f69eba05683a9a1882c41\""
time="2024-04-30T07:16:00Z" level=info msg="Creating kbs install overlay"
time="2024-04-30T07:16:00Z" level=info msg="Install Kbs"
Wait for the kbs deployment be available
time="2024-04-30T07:16:05Z" level=info msg="KBS PARAMS: cc_kbc::http://192.168.122.123:31581"
time="2024-04-30T07:16:05Z" level=info msg=EnableKbsAllowAllPolicy
time="2024-04-30T07:16:05Z" level=info msg="Install Cloud API Adaptor"
time="2024-04-30T07:16:05Z" level=info msg="Deploy the Cloud API Adaptor"
time="2024-04-30T07:16:05Z" level=info msg="Install the controller manager"
Wait for the cc-operator-controller-manager deployment be available
time="2024-04-30T07:16:13Z" level=info msg="Customize the overlay yaml file"
time="2024-04-30T07:16:15Z" level=info msg="Install the cloud-api-adaptor"
Wait for the pod cloud-api-adaptor-daemonset-wvd4h be ready
Wait for the kata-remote runtimeclass be created
time="2024-04-30T07:17:05Z" level=info msg="Installing peerpod-ctrl"
time="2024-04-30T07:17:11Z" level=info msg="Wait for the peerpod-ctrl deployment to be available"
time="2024-04-30T07:17:26Z" level=info msg="Creating namespace 'coco-pp-e2e-test-db63d831'..."
time="2024-04-30T07:17:26Z" level=info msg="Wait for namespace 'coco-pp-e2e-test-db63d831' be ready..."
time="2024-04-30T07:17:31Z" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-db63d831'..."
time="2024-04-30T07:17:31Z" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-db63d831' is ready for use"
=== RUN   TestLibvirtKbsKeyRelease
=== PAUSE TestLibvirtKbsKeyRelease
=== CONT  TestLibvirtKbsKeyRelease
time="2024-04-30T07:17:31Z" level=info msg="Do test kbs key release"
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-04-30T07:18:16Z" level=info msg="Success to get key.bin This is my cluster name: "
    assessment_runner.go:415: Output when execute test commands:This is my cluster name: 
time="2024-04-30T07:18:16Z" level=info msg="Deleting pod busybox-wget..."
time="2024-04-30T07:18:21Z" level=info msg="Pod busybox-wget has been successfully deleted within 60s"
--- PASS: TestLibvirtKbsKeyRelease (50.57s)
    --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test (50.57s)
        --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.53s)
PASS
ok  	github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e	141.284s

huoqifeng · 2024-04-30T07:24:56Z

I'll use a new PR to enable this test case in libvirt_e2e GHA.

src/cloud-api-adaptor/test/provisioner/provision.go

stevenhorsman

I think some of the commits need squashing and we need some information in the commit messages about the changes.

src/cloud-api-adaptor/test/e2e/libvirt_test.go

src/cloud-api-adaptor/test/provisioner/provision.go

src/cloud-api-adaptor/test/e2e/main_test.go

stevenhorsman · 2024-05-07T14:43:14Z

In the instructions can we use: make CLI_FEATURES=sample_only cli to reduce all the extra packages we need?

stevenhorsman · 2024-05-07T14:47:13Z

I tried to test this locally and hit an error:

=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-07T14:45:09Z" level=error msg="Failed to access key.bin: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }

I'm not sure of the cause though at the moment.

stevenhorsman · 2024-05-07T14:47:49Z

Can we also add a negative test here, such that we confirm that set-ups without using the permissive policy get rejected as expected?

huoqifeng · 2024-05-08T03:54:31Z

In the instructions can we use: make CLI_FEATURES=sample_only cli to reduce all the extra packages we need?

Yes, agreed

huoqifeng · 2024-05-08T03:55:44Z

I tried to test this locally and hit an error:

=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-07T14:45:09Z" level=error msg="Failed to access key.bin: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }

I'm not sure of the cause though at the moment.

I think it's because the parameter TEE_CUSTOMIZED_OPA is missed.
Updated the steps above #1824 (comment), also added negative test cases when TEE_CUSTOMIZED_OPA is missing.

huoqifeng · 2024-05-08T05:08:15Z

lint / govulncheck error here https://github.com/confidential-containers/cloud-api-adaptor/actions/runs/8996124825/job/24712137125?pr=1824 is tracked in #1836

src/cloud-api-adaptor/test/e2e/README.md

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

stevenhorsman · 2024-05-09T10:39:19Z

It is probably user error, but the KeyReleaseFailure test is failing for me:

=== RUN   TestLibvirtKbsKeyRelease
time="2024-05-09T10:03:39Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/deny_all.rego"
time="2024-05-09T10:03:39Z" level=trace msg="./kbs-client --url http://192.168.122.76:30992 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/deny_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gZmFsc2UK\n"
=== PAUSE TestLibvirtKbsKeyRelease
=== CONT  TestLibvirtKbsKeyRelease
time="2024-05-09T10:03:39Z" level=info msg="Do test kbs key release failure case"
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget-failure are ready
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
time="2024-05-09T10:04:24Z" level=error msg="Failed to faliure case as: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-09T10:04:24Z" level=info msg="Deleting pod busybox-wget-failure..."
time="2024-05-09T10:04:29Z" level=info msg="Pod busybox-wget-failure has been successfully deleted within 60s"
time="2024-05-09T10:04:29Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/allow_all.rego"
time="2024-05-09T10:04:30Z" level=trace msg="./kbs-client --url http://192.168.122.76:30992 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/allow_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gdHJ1ZQoK\n"
time="2024-05-09T10:04:30Z" level=info msg="Do test kbs key release"
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-09T10:05:15Z" level=error msg="Failed to access key.bin: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-09T10:05:15Z" level=info msg="Deleting pod busybox-wget..."
time="2024-05-09T10:05:20Z" level=info msg="Pod busybox-wget has been successfully deleted within 60s"
--- FAIL: TestLibvirtKbsKeyRelease (100.40s)
    --- FAIL: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test (50.17s)
        --- FAIL: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed (5.12s)
    --- FAIL: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test (50.20s)
        --- FAIL: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.16s)

It also doesn't report PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test like I would expect. Maybe as they are running in the same Test function?

huoqifeng · 2024-05-09T11:28:16Z

I run it again with after with new trustee repo cloned, looks good:

# cat /root/libvirt.properties
libvirt_uri="qemu+ssh://root@192.168.122.1/system?no_verify=1"
libvirt_ssh_key_file="id_rsa"
CLUSTER_NAME = "peer-pods"
KBS_IMAGE = "ghcr.io/confidential-containers/staged-images/kbs"
KBS_IMAGE_TAG = "dc01f454264fb4350e5f69eba05683a9a1882c41"

~/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e/trustee/kbs# cd ../
~/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e/trustee# git branch
* (HEAD detached at dc01f45)
  main
  
export TEST_PROVISION_FILE="/root/libvirt.properties"
export CLOUD_PROVIDER=libvirt
export DEPLOY_KBS=true
export TEST_INSTALL_CAA=yes
export TEST_TEARDOWN=no
make test-e2e
go test -v -tags=libvirt -timeout 60m -count=1 ./test/e2e
time="2024-05-09T11:22:16Z" level=info msg="Do setup"
time="2024-05-09T11:22:16Z" level=info msg="Deploying kbs"
time="2024-05-09T11:22:16Z" level=info msg="creating key.bin"
/root/src/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e
time="2024-05-09T11:22:16Z" level=info msg="Creating kbs install overlay"
time="2024-05-09T11:22:16Z" level=info msg="Customize the overlay yaml file"
time="2024-05-09T11:22:16Z" level=info msg="Updating kbs image with \"ghcr.io/confidential-containers/staged-images/kbs\""
time="2024-05-09T11:22:16Z" level=info msg="Updating kbs image tag with \"dc01f454264fb4350e5f69eba05683a9a1882c41\""
time="2024-05-09T11:22:16Z" level=info msg="Creating kbs install overlay"
time="2024-05-09T11:22:16Z" level=info msg="Install Kbs"
Wait for the kbs deployment be available
time="2024-05-09T11:22:21Z" level=info msg="KBS PARAMS: cc_kbc::http://192.168.122.123:31126"
time="2024-05-09T11:22:21Z" level=info msg="Install Cloud API Adaptor"
time="2024-05-09T11:22:21Z" level=info msg="Deploy the Cloud API Adaptor"
time="2024-05-09T11:22:21Z" level=info msg="Install the controller manager"
Wait for the cc-operator-controller-manager deployment be available
time="2024-05-09T11:22:39Z" level=info msg="Customize the overlay yaml file"
time="2024-05-09T11:22:41Z" level=info msg="Install the cloud-api-adaptor"
Wait for the pod cloud-api-adaptor-daemonset-jz9lv be ready
Wait for the kata-remote runtimeclass be created
time="2024-05-09T11:23:31Z" level=info msg="Installing peerpod-ctrl"
time="2024-05-09T11:23:32Z" level=info msg="Wait for the peerpod-ctrl deployment to be available"
time="2024-05-09T11:23:47Z" level=info msg="Creating namespace 'coco-pp-e2e-test-00b04760'..."
time="2024-05-09T11:23:47Z" level=info msg="Wait for namespace 'coco-pp-e2e-test-00b04760' be ready..."
time="2024-05-09T11:23:52Z" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-00b04760'..."
time="2024-05-09T11:23:52Z" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-00b04760' is ready for use"
=== RUN   TestLibvirtKbsKeyRelease
time="2024-05-09T11:23:52Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/deny_all.rego"
=== PAUSE TestLibvirtKbsKeyRelease
=== CONT  TestLibvirtKbsKeyRelease
time="2024-05-09T11:23:52Z" level=info msg="Do test kbs key release failure case"
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget-failure are ready
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
time="2024-05-09T11:24:38Z" level=info msg="Pass failure case as: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: request unautorized\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: request unautorized", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-09T11:24:38Z" level=info msg="Deleting pod busybox-wget-failure..."
time="2024-05-09T11:24:43Z" level=info msg="Pod busybox-wget-failure has been successfully deleted within 60s"
time="2024-05-09T11:24:43Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/allow_all.rego"
time="2024-05-09T11:24:43Z" level=info msg="Do test kbs key release"
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-09T11:25:29Z" level=info msg="Success to get key.bin This is my cluster name: "
    assessment_runner.go:415: Output when execute test commands:This is my cluster name: 
time="2024-05-09T11:25:29Z" level=info msg="Deleting pod busybox-wget..."
time="2024-05-09T11:25:34Z" level=info msg="Pod busybox-wget has been successfully deleted within 60s"
--- PASS: TestLibvirtKbsKeyRelease (101.70s)
    --- PASS: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test (51.19s)
        --- PASS: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed (6.15s)
    --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test (50.50s)
        --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.46s)
PASS
ok  	github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e	197.629s

I'll check your log later @stevenhorsman

huoqifeng · 2024-05-09T11:32:56Z

@stevenhorsman can you help cat your libvirt.properties file and the the commit in trustee repo?

stevenhorsman · 2024-05-09T11:58:08Z

@stevenhorsman can you help cat your libvirt.properties file and the the commit in trustee repo?

I think my settings are the same as yours. The only difference I've made is update the test-e2e target to only run the KBS test:

# cat libvirt.properties
libvirt_uri="qemu+ssh://root@10.250.129.5/system?no_verify=1"
libvirt_ssh_key_file="id_rsa"
CLUSTER_NAME = "peer-pods"
KBS_IMAGE = "ghcr.io/confidential-containers/staged-images/kbs"
KBS_IMAGE_TAG = "dc01f454264fb4350e5f69eba05683a9a1882c41"
```
```
root@sh-libvirt-switch-main:~/go/src/github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e/trustee# git log -1 --oneline
dc01f45 (HEAD) k8s-config: Add support for NodePort service type
```
```
export TEST_PROVISION_FILE="$PWD/libvirt.properties"
export TEST_PROVISION="no"
export CLOUD_PROVIDER=libvirt
export DEPLOY_KBS=true
export TEST_INSTALL_CAA=yes
export TEST_TEARDOWN=no
export KUBECONFIG=$HOME/.kcli/clusters/peer-pods/auth/kubeconfig
make test-e2e
go test -v --run TestLibvirtKbsKeyRelease  -tags=libvirt -timeout 50m -count=1 ./test/e2e
time="2024-05-09T11:50:27Z" level=info msg="Do setup"
time="2024-05-09T11:50:27Z" level=info msg="Deploying kbs"
time="2024-05-09T11:50:27Z" level=info msg="creating key.bin"
/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e
time="2024-05-09T11:50:27Z" level=info msg="Creating kbs install overlay"
time="2024-05-09T11:50:27Z" level=info msg="Customize the overlay yaml file"
time="2024-05-09T11:50:27Z" level=info msg="Updating kbs image with \"ghcr.io/confidential-containers/staged-images/kbs\""
time="2024-05-09T11:50:27Z" level=info msg="Updating kbs image tag with \"dc01f454264fb4350e5f69eba05683a9a1882c41\""
time="2024-05-09T11:50:27Z" level=info msg="Creating kbs install overlay"
time="2024-05-09T11:50:27Z" level=info msg="Install Kbs"
Wait for the kbs deployment be available
time="2024-05-09T11:50:32Z" level=info msg="KBS PARAMS: cc_kbc::http://192.168.122.76:30992"
time="2024-05-09T11:50:32Z" level=info msg="Podvm uploading"
time="2024-05-09T11:50:32Z" level=trace msg="UploadPodvm()"
time="2024-05-09T11:50:35Z" level=info msg="Install Cloud API Adaptor"
time="2024-05-09T11:50:35Z" level=info msg="Deploy the Cloud API Adaptor"
time="2024-05-09T11:50:35Z" level=info msg="Install the controller manager"
time="2024-05-09T11:50:37Z" level=trace msg="/usr/local/bin/kubectl apply -k github.com/confidential-containers/operator/config/release?ref=v0.8.0, output: namespace/confidential-containers-system configured\ncustomresourcedefinition.apiextensions.k8s.io/ccruntimes.confidentialcontainers.org configured\nserviceaccount/cc-operator-controller-manager unchanged\nrole.rbac.authorization.k8s.io/cc-operator-leader-election-role unchanged\nclusterrole.rbac.authorization.k8s.io/cc-operator-manager-role configured\nclusterrole.rbac.authorization.k8s.io/cc-operator-metrics-reader unchanged\nclusterrole.rbac.authorization.k8s.io/cc-operator-proxy-role unchanged\nrolebinding.rbac.authorization.k8s.io/cc-operator-leader-election-rolebinding unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cc-operator-manager-rolebinding unchanged\nclusterrolebinding.rbac.authorization.k8s.io/cc-operator-proxy-rolebinding unchanged\nconfigmap/cc-operator-manager-config unchanged\nservice/cc-operator-controller-manager-metrics-service unchanged\ndeployment.apps/cc-operator-controller-manager unchanged\n"
Wait for the cc-operator-controller-manager deployment be available
time="2024-05-09T11:50:42Z" level=info msg="Customize the overlay yaml file"
time="2024-05-09T11:51:05Z" level=trace msg="/usr/local/bin/kubectl apply -k github.com/confidential-containers/operator/config/samples/ccruntime/peer-pods?ref=v0.8.0, output: ccruntime.confidentialcontainers.org/ccruntime-peer-pods configured\n"
time="2024-05-09T11:51:05Z" level=info msg="Install the cloud-api-adaptor"
Wait for the cc-operator-daemon-install DaemonSet be available
Wait for the pod cc-operator-daemon-install-8hc2n be ready
Wait for the cloud-api-adaptor-daemonset DaemonSet be available
Wait for the pod cloud-api-adaptor-daemonset-r2ktq be ready
Wait for the kata-remote runtimeclass be created
time="2024-05-09T11:51:30Z" level=info msg="Installing peerpod-ctrl"
time="2024-05-09T11:51:32Z" level=trace msg="/usr/bin/make -C ../peerpod-ctrl deploy, output: make[1]: Entering directory '/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl'\ntest -s /root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl/bin/controller-gen || GOBIN=/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.14.0\n/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl/bin/controller-gen rbac:roleName=manager-role crd webhook paths=\"./...\" output:crd:artifacts:config=config/crd/bases\ncd config/manager && /root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl/bin/kustomize edit set image controller=quay.io/confidential-containers/peerpod-ctrl:latest\n/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl/bin/kustomize build config/default | kubectl apply -f -\n# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.\nnamespace/confidential-containers-system configured\ncustomresourcedefinition.apiextensions.k8s.io/peerpods.confidentialcontainers.org unchanged\nserviceaccount/peerpod-ctrl-controller-manager unchanged\nrole.rbac.authorization.k8s.io/peerpod-ctrl-leader-election-role unchanged\nclusterrole.rbac.authorization.k8s.io/peerpod-ctrl-manager-role unchanged\nclusterrole.rbac.authorization.k8s.io/peerpod-ctrl-metrics-reader unchanged\nclusterrole.rbac.authorization.k8s.io/peerpod-ctrl-proxy-role unchanged\nrolebinding.rbac.authorization.k8s.io/peerpod-ctrl-leader-election-rolebinding unchanged\nclusterrolebinding.rbac.authorization.k8s.io/peerpod-ctrl-manager-rolebinding unchanged\nclusterrolebinding.rbac.authorization.k8s.io/peerpod-ctrl-proxy-rolebinding unchanged\nservice/peerpod-ctrl-controller-manager-metrics-service unchanged\ndeployment.apps/peerpod-ctrl-controller-manager unchanged\nmake[1]: Leaving directory '/root/go/src/github.com/confidential-containers/cloud-api-adaptor/src/peerpod-ctrl'\n"
time="2024-05-09T11:51:32Z" level=info msg="Wait for the peerpod-ctrl deployment to be available"
time="2024-05-09T11:51:37Z" level=info msg="Creating namespace 'coco-pp-e2e-test-bf40ab10'..."
time="2024-05-09T11:51:37Z" level=info msg="Wait for namespace 'coco-pp-e2e-test-bf40ab10' be ready..."
time="2024-05-09T11:51:42Z" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-bf40ab10'..."
time="2024-05-09T11:51:42Z" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-bf40ab10' is ready for use"
=== RUN   TestLibvirtKbsKeyRelease
time="2024-05-09T11:51:42Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/deny_all.rego"
time="2024-05-09T11:51:42Z" level=trace msg="./kbs-client --url http://192.168.122.76:30992 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/deny_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gZmFsc2UK\n"
=== PAUSE TestLibvirtKbsKeyRelease
=== CONT  TestLibvirtKbsKeyRelease
time="2024-05-09T11:51:42Z" level=info msg="Do test kbs key release failure case"
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget-failure are ready
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
time="2024-05-09T11:52:27Z" level=error msg="Failed to faliure case as: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-09T11:52:27Z" level=info msg="Deleting pod busybox-wget-failure..."
time="2024-05-09T11:52:32Z" level=info msg="Pod busybox-wget-failure has been successfully deleted within 60s"
time="2024-05-09T11:52:32Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/allow_all.rego"
time="2024-05-09T11:52:32Z" level=trace msg="./kbs-client --url http://192.168.122.76:30992 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/allow_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gdHJ1ZQoK\n"
time="2024-05-09T11:52:32Z" level=info msg="Do test kbs key release"
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-09T11:53:17Z" level=error msg="Failed to access key.bin: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \\\"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\\\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
    assessment_runner.go:417: Command [wget -q -O- http://127.0.0.1:8006/cdh/resource/reponame/workload_key/key.bin] running in container busybox produced unexpected output on stdout: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: get token failed: Attestation Agent token provider error: cal ttrpc failed: rpc status: Status { code: INTERNAL, message: \"[ERROR:attestation-agent] AA-KBC get token failed: Unsupported token type: Matching variant not found\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-09T11:53:17Z" level=info msg="Deleting pod busybox-wget..."
time="2024-05-09T11:53:22Z" level=info msg="Pod busybox-wget has been successfully deleted within 60s"
--- FAIL: TestLibvirtKbsKeyRelease (100.43s)
    --- FAIL: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test (50.22s)
        --- FAIL: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed (5.18s)
    --- FAIL: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test (50.20s)
        --- FAIL: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.16s)
FAIL
FAIL	github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e	175.820s
FAIL
make: *** [Makefile:95: test-e2e] Error 1
```

Let me know if there is anything else you need, or I could give you access to my test box to try yourself?

src/cloud-api-adaptor/test/e2e/README.md

huoqifeng · 2024-05-10T23:57:44Z

It's identified that the PeerPod VM image from quay.io/confidential-containers/podvm-generic-ubuntu-amd64:e5a6fb8fdb34943caceea738770f79b9db87faa1 works well but the image from quay.io/confidential-containers/podvm-generic-ubuntu-amd64:v0.8.2 failed the test. It's interesting.
v0.8.2 and latest has same guest-components commit 277617af60c32661819c1132ffbf3db8dc6e1b9f, I'll investigate the reason...

huoqifeng · 2024-05-11T00:16:00Z

# journalctl -t kata-agent |grep kbc
May 11 00:06:28 podvm-busybox-92b37f81 kata-agent[828]: {"msg":"announce","level":"INFO","ts":"2024-05-11T00:06:28.531813269Z","subsystem":"root","version":"0.1.0","source":"agent","pid":"828","name":"kata-agent","api-version":"0.0.1","agent-commit":"3.2.0-rc0-d0df91935b8840036c2891b1f93dd8059ebe486a","config":"AgentConfig { debug_console: false, dev_mode: false, log_level: Info, hotplug_timeout: 3s, debug_console_vport: 0, log_vport: 0, container_pipe_size: 0, server_addr: \"unix:///run/kata-containers/agent.sock\", unified_cgroup_hierarchy: false, tracing: false, endpoints: AgentEndpoints { allowed: {\"ExecProcessRequest\", \"GetMetricsRequest\", \"OnlineCPUMemRequest\", \"RemoveContainerRequest\", \"StartContainerRequest\", \"GuestDetailsRequest\", \"AddSwapRequest\", \"MemHotplugByProbeRequest\", \"CloseStdinRequest\", \"ReseedRandomDevRequest\", \"GetOOMEventRequest\", \"ResumeContainerRequest\", \"StopTracingRequest\", \"TtyWinResizeRequest\", \"UpdateContainerRequest\", \"SignalProcessRequest\", \"WaitProcessRequest\", \"CreateContainerRequest\", \"CopyFileRequest\", \"DestroySandboxRequest\", \"ListInterfacesRequest\", \"PullImageRequest\", \"CreateSandboxRequest\", \"AddARPNeighborsRequest\", \"UpdateRoutesRequest\", \"StartTracingRequest\", \"ListRoutesRequest\", \"PauseContainerRequest\", \"UpdateInterfaceRequest\", \"SetGuestDateTimeRequest\", \"StatsContainerRequest\", \"ReadStreamRequest\", \"SetPolicyRequest\", \"WriteStreamRequest\"}, all_allowed: false }, supports_seccomp: true, container_policy_path: \"\", aa_kbc_params: \"cc_kbc::http://192.168.122.123:32601\", rest_api: \"\", https_proxy: \"\", no_proxy: \"\", data_integrity: false, enable_signature_verification: false, image_policy_file: \"\", image_registry_auth_file: \"\", simple_signing_sigstore_config: \"\" }","agent-version":"3.2.0-rc0"}
May 11 00:06:30 podvm-busybox-92b37f81 kata-agent[925]: [2024-05-11T00:06:30Z WARN  kms::plugins::kbs::cc_kbc] KBS_CERT get failed: NotPresent. Use no KBS public key certs.
May 11 00:06:33 podvm-busybox-92b37f81 kata-agent[876]: [2024-05-11T00:06:33Z ERROR attestation_agent::rpc::getresource::ttrpc] Call AA-KBC to get resource failed: AA does not support the given KBC module! Module: cc_kbc
May 11 00:06:35 podvm-busybox-92b37f81 kata-agent[876]: [2024-05-11T00:06:35Z ERROR attestation_agent::rpc::getresource::ttrpc] Call AA-KBC to get resource failed: AA does not support the given KBC module! Module: cc_kbc

Seems cc_kbc was not enabled in kata-agent/cdh/asr in the PodVM image in v0.8.2.

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng · 2024-05-11T02:09:43Z

Updated the readme. I'll add this AA_KBC=cc_kbc parameter when build the PodVM image when run libvirt_e2e GHA in a new PR.

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

stevenhorsman · 2024-05-13T09:03:49Z

I've added the test_e2e_libvirt label now that there is a podvm change to ensure we don't cause issues to existing tests, but am happy that the libvirt workflow changes to deploy the KBS and therefore test the new functions can be done in a subsequent PR.

stevenhorsman · 2024-05-13T14:03:04Z

When I re-built the podvm image with the cc_kbs the tests passed:

=== RUN   TestLibvirtKbsKeyRelease
time="2024-05-13T13:45:40Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/deny_all.rego"
time="2024-05-13T13:45:40Z" level=trace msg="./kbs-client --url http://192.168.122.242:31597 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/deny_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gZmFsc2UK\n"
=== PAUSE TestLibvirtKbsKeyRelease
=== CONT  TestLibvirtKbsKeyRelease
time="2024-05-13T13:45:40Z" level=info msg="Do test kbs key release failure case"
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget-failure are ready
=== RUN   TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed
time="2024-05-13T13:46:51Z" level=info msg="Pass failure case as: rpc status: Status { code: INTERNAL, message: \"[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: request unautorized\", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }"
    assessment_runner.go:415: Output when execute test commands:rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Get Resource failed: get resource failed: get rersource failed: Kbs client error: get resource failed: request unautorized", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } }
time="2024-05-13T13:46:51Z" level=info msg="Deleting pod busybox-wget-failure..."
time="2024-05-13T13:46:56Z" level=info msg="Pod busybox-wget-failure has been successfully deleted within 60s"
time="2024-05-13T13:46:56Z" level=info msg="EnableKbsCustomizedPolicy: ../../kbs/sample_policies/allow_all.rego"
time="2024-05-13T13:46:56Z" level=trace msg="./kbs-client --url http://192.168.122.242:31597 config --auth-private-key ../../kbs/config/kubernetes/base/kbs.key set-resource-policy --policy-file ../../kbs/sample_policies/allow_all.rego, output: Set resource policy success \n policy: CnBhY2thZ2UgcG9saWN5CgpkZWZhdWx0IGFsbG93ID0gdHJ1ZQoK\n"
time="2024-05-13T13:46:56Z" level=info msg="Do test kbs key release"
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test
    assessment_runner.go:265: Waiting for containers in pod: busybox-wget are ready
=== RUN   TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful
time="2024-05-13T13:47:41Z" level=info msg="Success to get key.bin This is my cluster name: "
    assessment_runner.go:415: Output when execute test commands:This is my cluster name:
time="2024-05-13T13:47:41Z" level=info msg="Deleting pod busybox-wget..."
time="2024-05-13T13:47:46Z" level=info msg="Pod busybox-wget has been successfully deleted within 60s"
--- PASS: TestLibvirtKbsKeyRelease (126.63s)
    --- PASS: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test (76.31s)
        --- PASS: TestLibvirtKbsKeyRelease/DoTestKbsKeyReleaseForFailure_test/Kbs_key_release_is_failed (6.26s)
    --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test (50.31s)
        --- PASS: TestLibvirtKbsKeyRelease/KbsKeyReleasePod_test/Kbs_key_release_is_successful (5.27s)

src/cloud-api-adaptor/test/e2e/README.md

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng · 2024-05-14T01:14:33Z

azure e2e test action https://github.com/confidential-containers/cloud-api-adaptor/blob/main/.github/workflows/azure-e2e-test.yml#L224 put trustee repo in ${cloud-api-adaptor-repo-dir}/src/cloud-api-adaptor/test in stead of ${cloud-api-adaptor-repo-dir}/src/cloud-api-adaptor/test/e2e, make correspond change in readme and TRUSTEE_REPO_PATH.

huoqifeng force-pushed the libvirt-kbs branch 3 times, most recently from 529cde5 to 51d3607 Compare April 28, 2024 10:16

huoqifeng mentioned this pull request Apr 28, 2024

azure-e2e-test: Add test for key release in kbs #1735

Merged

huoqifeng force-pushed the libvirt-kbs branch 3 times, most recently from f3183e0 to 8a45203 Compare April 30, 2024 06:21

huoqifeng requested review from mkulke and kartikjoshi21 April 30, 2024 07:08

huoqifeng marked this pull request as ready for review April 30, 2024 07:08

huoqifeng requested review from stevenhorsman and bpradipt April 30, 2024 07:08

mkulke reviewed Apr 30, 2024

View reviewed changes

src/cloud-api-adaptor/test/provisioner/provision.go Outdated Show resolved Hide resolved

src/cloud-api-adaptor/test/provisioner/provision.go Outdated Show resolved Hide resolved

huoqifeng force-pushed the libvirt-kbs branch 2 times, most recently from 6aadeca to f4e7810 Compare May 6, 2024 05:08

huoqifeng mentioned this pull request May 6, 2024

kbs: customize OPA policy when deploy KBS confidential-containers/trustee#380

Closed

stevenhorsman reviewed May 7, 2024

View reviewed changes

src/cloud-api-adaptor/test/e2e/libvirt_test.go Outdated Show resolved Hide resolved

src/cloud-api-adaptor/test/provisioner/provision.go Outdated Show resolved Hide resolved

src/cloud-api-adaptor/test/e2e/main_test.go Outdated Show resolved Hide resolved

huoqifeng force-pushed the libvirt-kbs branch 3 times, most recently from ba38655 to ee5f9d4 Compare May 8, 2024 03:51

huoqifeng force-pushed the libvirt-kbs branch 2 times, most recently from 51798d9 to 435a6c4 Compare May 8, 2024 04:10

stevenhorsman reviewed May 8, 2024

View reviewed changes

src/cloud-api-adaptor/test/e2e/README.md Outdated Show resolved Hide resolved

libvirt: enable attestation test cases and fix comments

2fe3343

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng force-pushed the libvirt-kbs branch 5 times, most recently from 305b81b to 9829d26 Compare May 9, 2024 05:17

libvirt: enable attestation test cases and run negative test also

1698635

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng force-pushed the libvirt-kbs branch from 9829d26 to 1698635 Compare May 9, 2024 05:22

stevenhorsman reviewed May 9, 2024

View reviewed changes

src/cloud-api-adaptor/test/e2e/README.md Outdated Show resolved Hide resolved

libvirt: enable attestation test cases and update readme for AA_KBC

8b3aafc

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

libvirt: change AA_KBC default to cc_kbc

0a103cc

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

stevenhorsman added the test_e2e_libvirt Run Libvirt e2e tests label May 13, 2024

huoqifeng force-pushed the libvirt-kbs branch from 19cc833 to ea1c95f Compare May 13, 2024 14:45

stevenhorsman approved these changes May 13, 2024

View reviewed changes

src/cloud-api-adaptor/test/e2e/README.md Outdated Show resolved Hide resolved

src/cloud-api-adaptor/test/e2e/README.md Outdated Show resolved Hide resolved

libvirt: update readme

42e5d7e

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng force-pushed the libvirt-kbs branch from ea1c95f to 42e5d7e Compare May 14, 2024 00:00

libvirt: enable e2e test and rollback TRUSTEE_REPO_PATH

512f858

Fixes: confidential-containers#1825 Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>

huoqifeng mentioned this pull request May 14, 2024

Add test case for libvirt secret retrieve for attestation based on sample TEE #1825

Closed

3 tasks

huoqifeng merged commit 6e467c5 into confidential-containers:main May 14, 2024
27 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libvirt: e2e test for attestation for sample tee #1824

libvirt: e2e test for attestation for sample tee #1824

huoqifeng commented Apr 28, 2024 •

edited

huoqifeng commented Apr 30, 2024 •

edited

huoqifeng commented Apr 30, 2024

stevenhorsman left a comment

stevenhorsman commented May 7, 2024

stevenhorsman commented May 7, 2024

stevenhorsman commented May 7, 2024

huoqifeng commented May 8, 2024

huoqifeng commented May 8, 2024

huoqifeng commented May 8, 2024

stevenhorsman commented May 9, 2024 •

edited

huoqifeng commented May 9, 2024 •

edited

huoqifeng commented May 9, 2024

stevenhorsman commented May 9, 2024

huoqifeng commented May 10, 2024

huoqifeng commented May 11, 2024

huoqifeng commented May 11, 2024

stevenhorsman commented May 13, 2024

stevenhorsman commented May 13, 2024

huoqifeng commented May 14, 2024

libvirt: e2e test for attestation for sample tee #1824

libvirt: e2e test for attestation for sample tee #1824

Conversation

huoqifeng commented Apr 28, 2024 • edited

huoqifeng commented Apr 30, 2024 • edited

huoqifeng commented Apr 30, 2024

stevenhorsman left a comment

Choose a reason for hiding this comment

stevenhorsman commented May 7, 2024

stevenhorsman commented May 7, 2024

stevenhorsman commented May 7, 2024

huoqifeng commented May 8, 2024

huoqifeng commented May 8, 2024

huoqifeng commented May 8, 2024

stevenhorsman commented May 9, 2024 • edited

huoqifeng commented May 9, 2024 • edited

huoqifeng commented May 9, 2024

stevenhorsman commented May 9, 2024

huoqifeng commented May 10, 2024

huoqifeng commented May 11, 2024

huoqifeng commented May 11, 2024

stevenhorsman commented May 13, 2024

stevenhorsman commented May 13, 2024

huoqifeng commented May 14, 2024

huoqifeng commented Apr 28, 2024 •

edited

huoqifeng commented Apr 30, 2024 •

edited

stevenhorsman commented May 9, 2024 •

edited

huoqifeng commented May 9, 2024 •

edited