Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR: quote3_error_t=SGX_QL_ERROR_INVALID_PRIVILEGE #205

Open
zhouyong0115 opened this issue Apr 21, 2023 · 11 comments
Open

ERROR: quote3_error_t=SGX_QL_ERROR_INVALID_PRIVILEGE #205

zhouyong0115 opened this issue Apr 21, 2023 · 11 comments

Comments

@zhouyong0115
Copy link

Issue description

I run the "remote_attestation" in sample in cloud service, however there were something wrong with in it which are shown below:
EGo v1.2.0 (f272b3f)
[erthost] loading enclave ...
[erthost] entering enclave ...
[ego] starting application ...
[load_qe ../qe_logic.cpp:698] Error, call sgx_create_enclave QE fail [load_qe], SGXError:4004.
ERROR: quote3_error_t=SGX_QL_ERROR_INVALID_PRIVILEGE
(oe_result_t=OE_PLATFORM_ERROR) [openenclave-src/host/sgx/sgxquote.c:oe_sgx_qe_get_target_info:714]
ERROR: SGX Plugin _get_report(): failed to get ecdsa report. OE_PLATFORM_ERROR (oe_result_t=OE_PLATFORM_ERROR) [openenclave-src/enclave/sgx/attester.c:_get_report:324]
OE_PLATFORM_ERROR
listening ...

When i run attested_tls sample, the error code is same. Thanks in advance.

@thomasten
Copy link
Member

Hi,
There may be some config issue with your system. Please follow the instructions at https://github.com/edgelesssys/sgx-troubleshoot#sgx-troubleshooting and post the output of the tool.

@zhouyong0115
Copy link
Author

thank you very much, we can see the output below.
root@SGX-Server-Blockchain-Lab:~/SGX# ./sgx-troubleshoot -v -test-all
SGX troubleshooter by Edgeless Systems (build timestamp: 1662455973)

/etc/sgx_default_qcnl.conf
PCCS_URL=https://sgx-dcap-server-vpc.cn-hangzhou.aliyuncs.com/sgx/certification/v3/
USE_SECURE_CERT=FALSE
ERROR: sgx_default_qcnl.conf: invalid character 'P' looking for beginning of value

lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 57 bits virtual
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 106
Model name: Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
Stepping: 6
CPU MHz: 2699.998
BogoMIPS: 5399.99
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 48 KiB
L1i cache: 32 KiB
L2 cache: 1.3 MiB
L3 cache: 48 MiB
NUMA node0 CPU(s): 0,1
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, RSB filling
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good
nopl nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor la
hf_lm abm 3dnowprefetch cpuid_fault invpcid_single ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx
512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat avx512vbmi pku ospke avx512_vbmi2 gfni vaes vpclmul
qdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid arch_capabilities

sh -c dmesg | grep microcode
exit status 1

sh -c lsmod | grep -i sgx
isgx 53248 2

sh -c dmesg | grep -i sgx
[ 7550.974238] intel_sgx: Intel SGX Driver v2.11.0
[ 7550.974287] intel_sgx intel_sgx.0: EPC bank 0x140000000-0x238000000

service aesmd status
● aesmd.service - Intel(R) Architectural Enclave Service Manager
Loaded: loaded (/lib/systemd/system/aesmd.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-04-19 22:28:30 CST; 2 days ago
Main PID: 106396 (aesm_service)
Tasks: 4 (limit: 4419)
Memory: 5.3M
CGroup: /system.slice/aesmd.service
└─106396 /opt/intel/sgx-aesm-service/aesm/aesm_service

Apr 19 22:28:30 SGX-Server-Blockchain-Lab systemd[1]: Starting Intel(R) Architectural Enclave Service Manager...
Apr 19 22:28:30 SGX-Server-Blockchain-Lab aesm_service[106376]: aesm_service: warning: Turn to daemon. Use "--no-daemon" option to execute in foreground.
Apr 19 22:28:30 SGX-Server-Blockchain-Lab systemd[1]: Started Intel(R) Architectural Enclave Service Manager.
Apr 19 22:28:30 SGX-Server-Blockchain-Lab aesm_service[106396]: [ADMIN]White List update requested
Apr 19 22:28:30 SGX-Server-Blockchain-Lab aesm_service[106396]: [load_qe ../qe_logic.cpp:698] Error, call sgx_create_enclave QE fail [load_qe], SGXError:4004.
Apr 19 22:28:30 SGX-Server-Blockchain-Lab aesm_service[106396]: Failed to load QE3: 0x4004
Apr 19 22:28:30 SGX-Server-Blockchain-Lab aesm_service[106396]: The server sock is 0x55b133420990
Apr 19 22:28:31 SGX-Server-Blockchain-Lab aesm_service[106396]: [ADMIN]White list update request successful for Version: 116

sh -c apt list --installed | grep -e sgx -e dcap
libsgx-ae-epid/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-id-enclave/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-ae-le/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-pce/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-qe3/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-ae-qve/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-aesm-ecdsa-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-aesm-epid-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-aesm-launch-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-aesm-pce-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-aesm-quote-ex-plugin/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-dcap-default-qpl/unknown,now 1.16.100.2-focal1 amd64 [installed]
libsgx-dcap-ql/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-dcap-quote-verify/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-enclave-common/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-launch/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-pce-logic/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-qe3-logic/unknown,now 1.16.100.2-focal1 amd64 [installed,automatic]
libsgx-quote-ex/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
libsgx-urts/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
sgx-aesm-service/unknown,now 2.19.100.3-focal1 amd64 [installed,automatic]
sgx-dcap-pccs/unknown,now 1.16.100.2-focal1 amd64 [installed]

stdbuf -oL ./testapp_host enclave.signed
CPUSVN: 0b0b100fffff00000000000000000000
[load_qe ../qe_logic.cpp:698] Error, call sgx_create_enclave QE fail [load_qe], SGXError:4004.
2023-04-22T08:09:26+0800.558956Z [(H)ERROR] tid(0x7ff6f624f280) | quote3_error_t=SGX_QL_ERROR_INVALID_PRIVILEGE
(oe_result_t=OE_PLATFORM_ERROR) [/openenclave/host/sgx/sgxquote.c:oe_sgx_qe_get_target_info:706]
2023-04-22T08:09:26+0800.559005Z [(H)ERROR] tid(0x7ff6f624f280) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/quote.c:sgx_get_qetarget_info:37]
2023-04-22T08:09:26+0800.559039Z [(E)ERROR] tid(0x7ff6f624f280) | enclave.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:oe_get_remote_report:283]
2023-04-22T08:09:26+0800.559066Z [(E)ERROR] tid(0x7ff6f624f280) | enclave.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:_oe_get_report_internal:388]
2023-04-22T08:09:26+0800.559104Z [(E)ERROR] tid(0x7ff6f624f280) | enclave.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:oe_get_report_v2_internal:443]
2023-04-22T08:09:26+0800.559135Z [(E)ERROR] tid(0x7ff6f624f280) | enclave.signed:SGX Plugin _get_report(): failed to get ecdsa report. OE_PLATFORM_ERROR (oe_result_t=OE_PLATFORM_ERROR) [/openenclave/enclave/sgx/attester.c:_get_report:324]
2023-04-22T08:09:26+0800.559159Z [(E)ERROR] tid(0x7ff6f624f280) | enclave.signed::OE_PLATFORM_ERROR [/openenclave/enclave/sgx/report.c:oe_get_report_v2:192]
ERROR: get remote report: OE_PLATFORM_ERROR

stdbuf -oL ./testapp_host enclave.signed
2023-04-22T08:09:26+0800.628615Z [(H)ERROR] tid(0x7f178b5c2280) | enclave_initialize failed (err=0x6) (oe_result_t=OE_PLATFORM_ERROR) [/openenclave/host/sgx/sgxload.c:oe_sgx_initialize_enclave:745]
2023-04-22T08:09:26+0800.628631Z [(H)ERROR] tid(0x7f178b5c2280) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_sgx_build_enclave:1134]
oe_create_helloworld_enclave(): result=21 (OE_PLATFORM_ERROR)
2023-04-22T08:09:26+0800.628775Z [(H)ERROR] tid(0x7f178b5c2280) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_create_enclave:1329]

docker run --rm -t -v/var/run/aesmd:/var/run/aesmd --device /dev/isgx ghcr.io/edgelesssys/sgx-troubleshoot/testapp enclave_debug.signed
Unable to find image 'ghcr.io/edgelesssys/sgx-troubleshoot/testapp:latest' locally
latest: Pulling from edgelesssys/sgx-troubleshoot/testapp
675920708c8b: Pulling fs layer
156ed6238e3a: Pulling fs layer
84260b97905a: Pulling fs layer
067252080310: Pulling fs layer
067252080310: Waiting
84260b97905a: Download complete
067252080310: Verifying Checksum
067252080310: Download complete
675920708c8b: Verifying Checksum
675920708c8b: Download complete
675920708c8b: Pull complete
156ed6238e3a: Verifying Checksum
156ed6238e3a: Download complete
156ed6238e3a: Pull complete
84260b97905a: Pull complete
067252080310: Pull complete
Digest: sha256:47d1c049682a4272d2d88d789342c537706c1b4600b2dfb78a18716a5c997151
Status: Downloaded newer image for ghcr.io/edgelesssys/sgx-troubleshoot/testapp:latest
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libsgx-dcap-default-qpl.
(Reading database ... 4917 files and directories currently installed.)
Preparing to unpack .../libsgx-dcap-default-qpl_1.14.100.3-focal1_amd64.deb ...
Unpacking libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Setting up libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
PCCS_URL: https://172.17.0.1:8081/sgx/certification/v3/
CPUSVN: 0b0b100fffff00000000000000000000
[load_qe ../qe_logic.cpp:698] Error, call sgx_create_enclave QE fail [load_qe], SGXError:4004.
2023-04-22T00:29:36+0000.051227Z [(H)ERROR] tid(0x7ff25e78bf40) | quote3_error_t=SGX_QL_ERROR_INVALID_PRIVILEGE
(oe_result_t=OE_PLATFORM_ERROR) [/openenclave/host/sgx/sgxquote.c:oe_sgx_qe_get_target_info:706]
2023-04-22T00:29:36+0000.051328Z [(H)ERROR] tid(0x7ff25e78bf40) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/quote.c:sgx_get_qetarget_info:37]
2023-04-22T00:29:36+0000.051400Z [(E)ERROR] tid(0x7ff25e78bf40) | enclave_debug.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:oe_get_remote_report:283]
2023-04-22T00:29:36+0000.051461Z [(E)ERROR] tid(0x7ff25e78bf40) | enclave_debug.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:_oe_get_report_internal:388]
2023-04-22T00:29:36+0000.051510Z [(E)ERROR] tid(0x7ff25e78bf40) | enclave_debug.signed::OE_PLATFORM_ERROR [/openenclave/enclave/core/sgx/report.c:oe_get_report_v2_internal:443]
2023-04-22T00:29:36+0000.051544Z [(E)ERROR] tid(0x7ff25e78bf40) | enclave_debug.signed:SGX Plugin _get_report(): failed to get ecdsa report. OE_PLATFORM_ERROR (oe_result_t=OE_PLATFORM_ERROR) [/openenclave/enclave/sgx/attester.c:_get_report:324]
2023-04-22T00:29:36+0000.051557Z [(E)ERROR] tid(0x7ff25e78bf40) | enclave_debug.signed::OE_PLATFORM_ERROR [/openenclave/enclave/sgx/report.c:oe_get_report_v2:192]
ERROR: get remote report: OE_PLATFORM_ERROR

docker run --rm -t -v/var/run/aesmd:/var/run/aesmd --device /dev/isgx ghcr.io/edgelesssys/sgx-troubleshoot/testapp enclave.signed
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libsgx-dcap-default-qpl.
(Reading database ... 4917 files and directories currently installed.)
Preparing to unpack .../libsgx-dcap-default-qpl_1.14.100.3-focal1_amd64.deb ...
Unpacking libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Setting up libsgx-dcap-default-qpl (1.14.100.3-focal1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
PCCS_URL: https://172.17.0.1:8081/sgx/certification/v3/
2023-04-22T00:29:38+0000.216306Z [(H)ERROR] tid(0x7f046b594f40) | enclave_initialize failed (err=0x6) (oe_result_t=OE_PLATFORM_ERROR) [/openenclave/host/sgx/sgxload.c:oe_sgx_initialize_enclave:745]
2023-04-22T00:29:38+0000.216684Z [(H)ERROR] tid(0x7f046b594f40) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_sgx_build_enclave:1134]
2023-04-22T00:29:38+0000.217086Z [(H)ERROR] tid(0x7f046b594f40) | :OE_PLATFORM_ERROR [/openenclave/host/sgx/create.c:oe_create_enclave:1329]
oe_create_helloworld_enclave(): result=21 (OE_PLATFORM_ERROR)

CPU name Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
CPU supports SGX true
CPU supports SGX-FLC true
SGX enabled in BIOS/Hypervisor true
SGX2 true
EPC size MiB 3968
SMT/Hyper-threading true
uname Linux SGX-Server-Blockchain-Lab 5.4.0-106-generic #120-Ubuntu SMP Fri Mar 18 12:42:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Cloud
/dev mount options rw,nosuid,relatime,size=1885776k,nr_inodes=471444,mode=755
Current user root
Users of group sgx_prv
AESM status active
AESM socket Srwxrwxrwx
Value of SGX_AESM_ADDR (not set)
PCCS URL
PCCS use secure cert
PCSS API version
PCCS connection URL not set
sys_vendor Alibaba Cloud
board_vendor open /sys/devices/virtual/dmi/id/board_vendor: no such file or directory
board_name open /sys/devices/virtual/dmi/id/board_name: no such file or directory
board_version open /sys/devices/virtual/dmi/id/board_version: no such file or directory
bios_vendor EFI Development Kit II / OVMF
bios_version 0.0.0
bios_date 02/06/2015
bios_release open /sys/devices/virtual/dmi/id/bios_release: no such file or directory
/dev drwxr-xr-x
/dev/sgx lstat /dev/sgx: no such file or directory
/dev/sgx_enclave lstat /dev/sgx_enclave: no such file or directory
/dev/sgx/enclave lstat /dev/sgx/enclave: no such file or directory
/dev/sgx_provision lstat /dev/sgx_provision: no such file or directory
/dev/sgx/provision lstat /dev/sgx/provision: no such file or directory
/dev/isgx Dcrw-rw-rw-
Debug enclave exit code 8 (attestation failed: get_remote_report)
Debug enclave CPUSVN 0b0b100fffff00000000000000000000
Debug enclave TCB status Unknown (unknown status)
Production enclave exit code 1 (failed to launch enclave)
Production enclave TCB status Unknown (unknown status)
Debug Docker enclave exit code 8 (attestation failed: get_remote_report)
Debug Docker enclave CPUSVN 0b0b100fffff00000000000000000000
Debug Docker enclave TCB status Unknown (unknown status)
Production Docker enclave exit code 1 (failed to launch enclave)
Production Docker enclave TCB status Unknown (unknown status)

Quote providers:
/usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so => /usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so.1.13.103.2
/usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so.1 => /usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so.1.13.103.2
/usr/lib/x86_64-linux-gnu/libdcap_quoteprov.so.1.13.103.2

@thomasten
Copy link
Member

You have the legacy isgx driver installed. First, you need to uninstall it. (There should be an uninstall script somewhere in /opt/intel).

Then you can either upgrade your kernel to >= 5.11 (recommended) or install the dcap driver.

To upgrade your kernel on Ubuntu 20.04, you can apt install linux-generic-hwe-20.04.

If you can't upgrade your kernel, you can get the dcap driver here: https://download.01.org/intel-sgx/latest/dcap-latest/linux/distro/

If you still get (the same or another) error after doing this, please run sgx-troubleshoot again and post the new output.

@zhouyong0115
Copy link
Author

zhouyong0115 commented Apr 23, 2023 via email

@thomasten
Copy link
Member

I'm unsure what you want to achieve. And I don't know what exactly you mean with node and host. Maybe you can give a small example that makes things clearer.

@zhouyong0115
Copy link
Author

zhouyong0115 commented Apr 25, 2023 via email

@thomasten
Copy link
Member

I got a rough idea of it now, but I'm not sure which problem is not solved yet. Once the client verified the certificate with remote attestation, it can securely connect to the server and they can exchange arbitrary data.

@zhouyong0115
Copy link
Author

zhouyong0115 commented Apr 25, 2023 via email

@thomasten
Copy link
Member

The remote_attestation sample also has a client app: https://github.com/edgelesssys/ego/blob/master/samples/remote_attestation/ra_client/client.go
Have you already looked at it? Anything unclear regarding this?

@zhouyong0115
Copy link
Author

zhouyong0115 commented Apr 26, 2023 via email

@thomasten
Copy link
Member

Yes, that's right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants