How to specify firmware version?

[debloper]

Probably a noob question, but how does buildroot know which firmware commit to fetch? It's doing something weird, and I'm not sure how to ask it nicely to use latest release and not a random version.

What am I trying to do?

I'm trying to build it for RPi Zero (without wifi), to have Camera Module v3 support. Here's the changes I've made so far (after a few trials and errors, mostly to use the latest stable & interop code as much as possible):

diff --git a/buildroot b/buildroot
index 170f42e..954aeb7 160000
--- a/buildroot
+++ b/buildroot
@@ -1 +1 @@
-Subproject commit 170f42eb6b891e06fd500483c92a5e00f77d1dc6
+Subproject commit 954aeb7c721bf7edd56832650e8ebfb3b5c6fbd3
diff --git a/configs/config b/configs/config
index e6083c2..31427dc 100644
--- a/configs/config
+++ b/configs/config
@@ -1,6 +1,6 @@
 BR2_arm=y
 BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
-BR2_KERNEL_HEADERS_5_10=y
+BR2_KERNEL_HEADERS_6_1=y
 BR2_TOOLCHAIN_BUILDROOT_CXX=y
 BR2_TARGET_GENERIC_HOSTNAME="piwebcam"
 BR2_TARGET_GENERIC_ISSUE=""
@@ -13,7 +13,7 @@ BR2_ROOTFS_POST_IMAGE_SCRIPT="$(BR2_EXTERNAL_PICAM_PATH)/board/post-image.sh"
 BR2_ROOTFS_POST_SCRIPT_ARGS="--configure-picam"
 BR2_LINUX_KERNEL=y
 BR2_LINUX_KERNEL_CUSTOM_TARBALL=y
-BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,raspberrypi,linux,6af8ae321a801a4e20183454c65eb0d23069d8ac)/linux-6af8ae321a801a4e20183454c65eb0d23069d8ac.tar.gz"
+BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,raspberrypi,linux,a984fda6b2c24dbf1ca21924f99c8f9418f5765e)/linux-a984fda6b2c24dbf1ca21924f99c8f9418f5765e.tar.gz"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="$(BR2_EXTERNAL_PICAM_PATH)/board/linux.config"
 BR2_LINUX_KERNEL_DTS_SUPPORT=y

What's happening

• The genimage part of post-image.sh script is failing, complaining about not finding the rpi-firmware/bootcode.bin
  • I can see output/raspberrypi0/images or its subdir rpi-firmware don't have bootcode.bin or fixup.dat or start.elf files
  • It also doesn't have imx708's dtoverlay in the overlay subdir, meaning it's not gonna work to meet my expectation anyway
• I can see it has got rpi-firmware-3f20b832b27cd730deb6419b570f31a98167eef6 snapshot downloaded in output/raspberrypi0/build path, which has all the necessary files (including the dtoverlay for imx708)
  • I'm assuming that setting the kernel headers to 6.1 may have some effect on this, as that's when it was moved to support kernel 6.1 (in that case, from where is it finding this info, and how to override it?)
  • In that case, why the images dir has different (size & hash) bcm2708-rpi-zero.dtb file? Is it from before I updated the kernel headers version? Does it not update it automatically? Should I clean cache and start a rebuild?
• Looking at Buildroot update / Raspberry Pi 2 Support #197 I can see that @dgsiegel has added the fixup & start files explicitly in the config... and I might give that branch a try, but is that going to be necessary?
  • I'm avoiding that branch as it seems to introduce too many variables at the same time, and I don't fully understand it all
  • It also seems to stick with kernel 5.10 (not sure if that's a hard limit & 6.x can't work for some reason)
  • Which also might mean related firmware for it may not support the new Camera Module v3
  • It also seems to be work in progress, and not done yet (so I'm not sure if I'll succeed)

I'm going to clean cache and start fresh with the changes made so far - hopefully that clears up some of the confusion (especially firmware version discrepancy between build and image). But while I throw dart on the wall in the dark, I'll appreciate any help/guidance on this.

[dgsiegel]

Much has changed in Buildroot, rpi-firmware in the past few years, so it's quite normal that you run into trouble combining newer stuff with older software. As you already discovered, I've tried to update everything and add libcamera support in #197. I think that would be the best place to start.

Probably a noob question, but how does buildroot know which firmware commit to fetch? It's doing something weird, and I'm not sure how to ask it nicely to use latest release and not a random version.

The version is set in buildroot/package/rpi-firmware/rpi-firmware.mk which will be defined by the Buildroot version. So either use a specific Buildroot version or make a copy of that Buildroot package.

What am I trying to do?

I'm trying to build it for RPi Zero (without wifi), to have Camera Module v3 support. Here's the changes I've made so far (after a few trials and errors, mostly to use the latest stable & interop code as much as possible):

Be careful as a new kernel will also include new firmware which might or might not be compatible with the selecte rpi-firmware version and your setup. That's one reason why I was still using 5.10 in my PR.

* The `genimage` part of `post-image.sh` script is failing, complaining about not finding the `rpi-firmware/bootcode.bin`
  
  * I can see `output/raspberrypi0/images` or its subdir `rpi-firmware` don't have bootcode.bin or fixup.dat or start.elf files
  * It also doesn't have imx708's dtoverlay in the `overlay` subdir, meaning it's not gonna work to meet my expectation anyway

As stated above be sure to use a matching rpi-firmware version for your kernel.

* I can see it has got `rpi-firmware-3f20b832b27cd730deb6419b570f31a98167eef6` snapshot downloaded in `output/raspberrypi0/build` path, which has all the necessary files (including the dtoverlay for imx708)
  
  * I'm assuming that setting the kernel headers to 6.1 may have some effect on this, as that's when it was moved to support kernel 6.1 (in that case, from where is it finding this info, and how to override it?)
  * In that case, why the `images` dir has different (size & hash) `bcm2708-rpi-zero.dtb` file? Is it from before I updated the kernel headers version? Does it not update it automatically? Should I clean cache and start a rebuild?

Make sure you have BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS=y set in order to have the overlays available during build time. Then you have to manually copy them over to your image in board/genimage-raspberrypi0.cfg. See my PR for an example.

* Looking at [Buildroot update / Raspberry Pi 2 Support #197](https://github.com/showmewebcam/showmewebcam/pull/197) I can see that @dgsiegel has added the fixup & start files explicitly in the config... and I might give that branch a try, but is that going to be necessary?

Yes, please do. The current master only supports old kernel, old rpi-firmware and old Raspberry Pi firmware. Feel free to remove all the libcamera stuff. Also please note that you should confirm whether the Camera 3 module actually works with the legacy setup (there is a setting for this in the Raspberry config) or whether you need the new firmware and/or libcamera. See #223 (comment) for details.

  * I'm avoiding that branch as it seems to introduce too many variables at the same time, and I don't fully understand it all

Right, the main difference is old vs. new Raspberry firmware, where the old firmware supports direct hardware access for V4L devices and the new one requires libcamera. Have a look at recent issues and some discussions about this on the Discord channel.

  * It also seems to stick with kernel 5.10 (not sure if that's a hard limit & 6.x can't work for some reason)

Same as above.

  * Which also might mean related firmware for it may not support the new Camera Module v3

Yeah, that's something you'd have to check out. Or maybe @kbingham knows something about this.

  * It also seems to be work in progress, and not done yet (so I'm not sure if I'll succeed)

It works, but performance is very poor and therefore I didn't continue working on it.

I'm going to clean cache and start fresh with the changes made so far - hopefully that clears up some of the confusion (especially firmware version discrepancy between build and image). But while I throw dart on the wall in the dark, I'll appreciate any help/guidance on this.

Hope my comments helped, but feel free to join the Discord channel, where you can find many other talented people.

[kbingham]

What am I trying to do?

I'm trying to build it for RPi Zero (without wifi), to have Camera Module v3 support. Here's the changes I've made so far (after a few trials and errors, mostly to use the latest stable & interop code as much as possible):

Be careful as a new kernel will also include new firmware which might or might not be compatible with the selecte rpi-firmware version and your setup. That's one reason why I was still using 5.10 in my PR.

* The `genimage` part of `post-image.sh` script is failing, complaining about not finding the `rpi-firmware/bootcode.bin`
  
  * I can see `output/raspberrypi0/images` or its subdir `rpi-firmware` don't have bootcode.bin or fixup.dat or start.elf files
  * It also doesn't have imx708's dtoverlay in the `overlay` subdir, meaning it's not gonna work to meet my expectation anyway

As stated above be sure to use a matching rpi-firmware version for your kernel.

* I can see it has got `rpi-firmware-3f20b832b27cd730deb6419b570f31a98167eef6` snapshot downloaded in `output/raspberrypi0/build` path, which has all the necessary files (including the dtoverlay for imx708)
  
  * I'm assuming that setting the kernel headers to 6.1 may have some effect on this, as that's when it was moved to support kernel 6.1 (in that case, from where is it finding this info, and how to override it?)
  * In that case, why the `images` dir has different (size & hash) `bcm2708-rpi-zero.dtb` file? Is it from before I updated the kernel headers version? Does it not update it automatically? Should I clean cache and start a rebuild?

Make sure you have BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS=y set in order to have the overlays available during build time. Then you have to manually copy them over to your image in board/genimage-raspberrypi0.cfg. See my PR for an example.

* Looking at [Buildroot update / Raspberry Pi 2 Support #197](https://github.com/showmewebcam/showmewebcam/pull/197) I can see that @dgsiegel has added the fixup & start files explicitly in the config... and I might give that branch a try, but is that going to be necessary?

Yes, please do. The current master only supports old kernel, old rpi-firmware and old Raspberry Pi firmware. Feel free to remove all the libcamera stuff. Also please note that you should confirm whether the Camera 3 module actually works with the legacy setup (there is a setting for this in the Raspberry config) or whether you need the new firmware and/or libcamera. See #223 (comment) for details.

  * I'm avoiding that branch as it seems to introduce too many variables at the same time, and I don't fully understand it all

Right, the main difference is old vs. new Raspberry firmware, where the old firmware supports direct hardware access for V4L devices and the new one requires libcamera. Have a look at recent issues and some discussions about this on the Discord channel.

  * It also seems to stick with kernel 5.10 (not sure if that's a hard limit & 6.x can't work for some reason)

Same as above.

6.x should work and getting it running is worthwhile

  * Which also might mean related firmware for it may not support the new Camera Module v3

Yeah, that's something you'd have to check out. Or maybe @kbingham knows something about this.

The Rpi firmware itself doesn't manage the V3 camera, but you very likely need an updated kernel. I can't remember when the V3 came in.

5.10 is really old now and not supported by rpi

  * It also seems to be work in progress, and not done yet (so I'm not sure if I'll succeed)

It works, but performance is very poor and therefore I didn't continue working on it.

If someone has time, I think we can fix this. But I haven't had time. I'll help anyone who can work on it though.

I'm going to clean cache and start fresh with the changes made so far - hopefully that clears up some of the confusion (especially firmware version discrepancy between build and image). But while I throw dart on the wall in the dark, I'll appreciate any help/guidance on this.

Hope my comments helped, but feel free to join the Discord channel, where you can find many other talented people.

Yup, feel free to ping me there for more real-time discussion too.