Metadata magic word does not match certificate errors when run on an encrypted device

[idea--list]

Describe the bug
Repo runs as expected on an unencrypted device when following the getting started guide here:
https://github.com/FreeRTOS/iot-reference-esp32c3/blob/main/GettingStartedGuide.md

However i am struggling for 3 days now to enable security features despite trying to follow the guide for enabling security features found here
https://github.com/FreeRTOS/iot-reference-esp32c3/blob/main/UseSecurityFeatures.md

I always get "metadata magic word does not match" and certificate errors when run on an encrypted device.

Reading the guide about enabling security features i face several issues:

• the guide tells there is an option called "Use DS Peripheral" under "Featured FreeRTOS IoT Integration" in menuconfig... well there is no such option
  (found a similar option under Component config -> ESP-TLS and even under ESP Secure Cert configuration options, even tried all possible combinations of those, resulting in no difference)

• to create the esp_secure_crt partition binary this command is not more valid:
  python managed_components/espressif__esp_secure_cert_mgr/tools/configure_esp_secure_cert.py -p PORT --configure_ds --keep_ds_data_on_host --ca-cert CA_CERT_FILEPATH --device-cert DEVICE_CERT_FILEPATH --private-key PRIVATE_KEY_FILEPATH --target_chip esp32s3 --secure_cert_type cust_flash
  (as a newer version of cert manager will be pulled, which also requires adding --priv_key_algo parameter. Also tried to add that, but still no difference)

• Section 5 has the title "Configure Secure Boot" but then just above point 5.1 the guide all of a sudden is speaking about secure boot V2. If so the title of section 5 should rather be "Configure Secure Boot 2"
  (Did not even try to activate secure boot as i do not want to risk bricking my board)

To sum it: it is clear that the guide is outdated and for sure is not compliant with the master repo compiled on IDF V5.1.2

A clear and current description about how to run this repo on an encrypted board and if secure boot 2 is a requirement or only optional would be highly appreciated!

System information

• Hardware board: Adafruit ESP32-S3 Feather
• IDE used: [ VS Code ]
• Operating System: [ Windows ]
• Code version: (run git describe --tags to find it)
• Project/Demo: [ any demo of this repo ]

Expected behavior
I expect the repo working when i follow the description about securing this repo found here:
https://github.com/FreeRTOS/iot-reference-esp32c3/blob/main/UseSecurityFeatures.md

Screenshots or console output

Steps to reproduce bug

1. Build and run the repo on an unencrypted ESP32S3 board
2. Follow the "Getting Started With Security Features" for this repo exactly as written at the link above
3. Notice you can not succeed as something important has changed since that guide has been put together (or some breaking change(s) have been introduced in IDF itself?).

Code to reproduce bug
Current master of this repo.

Additional context
Include any additional context relevant to the issue here.

[aggarg]

Thank you for reporting this issue. We are taking a look.

[xuelix]

Thanks for reporting the issue and sorry for the confusion caused in the provisioning process.

The guide is only good for ESP-IDF 4.x version. I'll raise a PR to update the UseSecurityFeatures.md to support latest ESP-IDF.

As for the support on ESP32S3 board, I am contacting an engineer from Espressif team to see if the same code would support ESP32-S3 board.

[idea--list]

@aggarg , @xuelix
Meanwhile i figured out that secure boot v2 is a requirement in order to enable security features of the repo.
However i also managed to brick my board while trying to figure out things...

Will stay tuned how to enable security features on IDF 5.x

[xuelix]

I got the confirmation from Espressif, ESp32-s3 is supported with the same code. I am waiting for a fresh new board arriving to try out the process before merging in the PR.

I actually saw exact the same error messages you saw with my esp32-s3 board. I managed to get out that state by re-flashing the secure cert using this command:

1. you should have the secure_cert on host, it should be located at managed_components/esp_secure_cert_data
2. -- encrypt should be added at the end of the write_flash command, like shown below

esptool.py --no-stub --port PORT write_flash 0xD000 esp_secure_cert_data/esp_secure_cert.bin --encrypt

The above step actually should be removed from the guide.
Thanks for the patience.

[idea--list]

@xuelix
Thanks for the suggestion!
esptool.py --no-stub --port PORT write_flash 0xD000 esp_secure_cert_data/esp_secure_cert.bin --encrypt
Solves the metadata magic word does not match issue.

But right after that i face the next one:

I try to follow the documentation, but something seems to be missing and i can not figure it out.

The problem is that my certificate that signs the OTA binaries has expired. I had to create a new OTA signing profile with a new certificate in IoT Core. I also had to replace the code signing certificate on the device and ever since then thigs do not work as expected. I can not reproduce this on unencrypted boards, so there is something wrong/missing in the documentation of how to secure these examples.

[xuelix]

The PR is finally merged. I would suggest to follow the doc and start from the scratch before diving into debugging a board might not be programmed correctly.

[xuelix]

I will close the issue for now. If you have more questions, please open a new issue. Thanks