Kernel build re-architecture + Pi 0 2w support + Pi 0 Universal build #173

htruong · 2021-11-14T04:56:06Z

This is gonna be a heavyweight one. While trying to support Pi 0 2w support, I realized we have been maintaining our kernel configs in a way that is not easy to keep track of: We have been hand-crafting kernel configs. This makes it hard to port support to new boards, because we have to figure out (manually) the diff from Pi0 which is the base kernel config and add stuff on top of that. This isn't something our small team could realistically support in the long run: Hand maintaining those configs and how they change is time consuming -- even when we just want to uprev the kernel.

The more sensible way to do it would be to respect the in-tree defconfig (which is maintained by the Raspberry Foundation) and tell it to "add more modules" (or remove them) on top of their work to support our embedded configuration. This simplifies the kernel config maintenance by a lot: No longer we have to maintain thousands of lines of kernel configs we don't understand, just maintain stuff that we care about and we do understand.

I also add a (poor man's) way to overlay/override vanilla buildroot packages -- in this case, we want to use a custom rpi-firmware package in order to support the pi0_2w.

sazarkin · 2021-11-29T17:59:04Z

build-showmewebcam.sh

-    echo "usage: BUILDROOT_DIR=buildroot $0 (boardname)"
-    echo "boardname: raspberrypi0, raspberrypi0w, raspberrypi4"
+    echo "usage: BUILDROOT_DIR=buildroot $0 (boardname) [component]"
+    echo "boardname: raspberrypi0, raspberry0_2, raspberrypi4"


Typo, should be raspberrypi0_2

dgsiegel · 2021-11-29T20:15:06Z

.github/workflows/release.yml

@@ -20,12 +20,12 @@ jobs:
  build:
    runs-on: ubuntu-20.04
    env:
-      BUILDROOT_VERSION: '2021.02.4'
+      BUILDROOT_VERSION: '2021.08.2'


AFAIK the buildroot people are still working on the Pi2 support, shall we wait for their stable release?

dgsiegel · 2021-11-29T20:24:07Z

.github/workflows/release.yml

      BUILDROOT_DIR: 'buildroot'

    strategy:
      matrix:
-        raspberry: ['raspberrypi0', 'raspberrypi0w', 'raspberrypi4']
+        raspberry: ['raspberrypi0', 'raspberrypi0_2', 'raspberrypi4']


It might make sense to rework the naming to keep them consistent. Buildroot uses raspberrypizero2w, so we might use zero too instead of 0?

Or else raspberrypi0, raspberrypi0v2, raspberrypi4 etc.

dgsiegel · 2021-11-29T20:24:23Z

.github/workflows/release.yml

@@ -129,8 +129,7 @@ jobs:
          name: release

      - name: Create checksums file
-        run: |
-          sha256sum "sdcard*" > CHECKSUMS.sha256
+        run: sha256sum sdcard* > CHECKSUMS.sha256


This should be a separate PR

dgsiegel · 2021-11-29T20:26:11Z

board/config_0.txt

@@ -0,0 +1,29 @@
+# Please note that this is only a sample, we recommend you to change it to fit


We don't really need this file, it's basically one from buildroot/board/raspberrypi/, so we might just copy it from there.

dgsiegel · 2021-11-29T20:26:24Z

board/config_4.txt

@@ -0,0 +1,29 @@
+# Please note that this is only a sample, we recommend you to change it to fit


See comment above

dgsiegel · 2021-11-29T20:32:45Z

build-showmewebcam.sh

+if [ $# -ge 1 ] ; then 
+  make -C "output/$BOARDNAME" "$@"
+else
+  make -C "output/$BOARDNAME" all


dgsiegel · 2021-11-29T20:33:33Z

configs/config

@@ -1,6 +1,6 @@
 BR2_arm=y
 BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
-BR2_KERNEL_HEADERS_5_10=y
+BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_5_10=y


What is the difference between those two?

dgsiegel · 2021-11-29T20:34:52Z

configs/config

-BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
-BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="$(BR2_EXTERNAL_PICAM_PATH)/board/linux.config"
+BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="$(call github,raspberrypi,linux,2a297702b85d53863458d698b60af4d8c3fc853f)/linux-2a297702b85d53863458d698b60af4d8c3fc853f.tar.gz"
+BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="$(BR2_EXTERNAL_PICAM_PATH)/board/linux-additional-modules.config"


Oh this is nice, might even be easier to use our existing config overrides.

dgsiegel · 2021-11-29T20:37:01Z

configs/raspberrypi0

+BR2_PACKAGE_RPI_FIRMWARE_BOOTCODE_BIN=y
+BR2_PACKAGE_RPI_FIRMWARE_VARIANT_PI=y
+BR2_PACKAGE_RPI_FIRMWARE_VARIANT_PI_X=y
+BR2_PACKAGE_RPI_FIRMWARE_CONFIG_FILE="$(BR2_EXTERNAL_PICAM_PATH)/board/config_0.txt"


Most of these are currently not existing in our setup, can you explain the changes?

dgsiegel · 2021-11-29T20:37:25Z

package-override/rpi-firmware/Config.in

@@ -0,0 +1,109 @@
+config BR2_PACKAGE_RPI_FIRMWARE


See my comment above about package overrides

This kernel is needed for Raspberry Zero 2/2w support, yet doesn't break traditional camera module. It includes the device tree files for this board.

This change adds the following functionalities: - Add Raspberry Pi Zero 2W support - Consolidate Raspberry Zero support to one image - Add the ability to override buildroot packages (in this case, we override rpi-firmware package due to buildroot upstream hasn't added support for Raspberry Pi Zero 2W) - Allow build-showmewebcam.sh to take optional parameters The build-showmewebcam.sh can now be called with a optional buildroot package like so: ./build-showmewebcam.sh raspberrypi0 rpi-firmware ./build-showmewebcam.sh raspberrypi0 menuconfig And so on. This simplifies manually calling those commands when we just try to build a single package. Fixes showmewebcam#168

It turns out that maintaining our linux configs is not a good idea. This makes adding new board supports very hard, while we can mirror what is buildroot does for the pis: Just use the foundation's defconfig. So this change deletes all of our linux defconfigs and just add the modules that we want for showmewebcam to function, namely board/linux-additional-modules.config. This also add the post-image changes that is needed for this almost-stock kernel to boot quickly and with the modules we desire.

The double quote doesn't allow the wildcard expansion, causing the release to fail, so remove the double quotes. Fixes showmewebcam#170

dgsiegel · 2022-05-09T14:00:18Z

I am currently reworking this patch and preparing our setup to allow adding new boards (like the Pi0 2w) more easily in this new PR #197.

With the switch to libcamera, this is unfortunately harder than anticipated, so any help is highly appreciated!

kbingham · 2023-07-11T22:02:44Z

I've now discovered that there /is/ a hardware JPEG encoder on the new stack - I'm not sure how we missed it last time - so if anyone wants to try to resume this work - let me know and we can figure out how we make use of the hardware encoders on the Zero/Zero2.

dgsiegel · 2023-07-12T07:40:14Z

I've now discovered that there /is/ a hardware JPEG encoder on the new stack - I'm not sure how we missed it last time - so if anyone wants to try to resume this work - let me know and we can figure out how we make use of the hardware encoders on the Zero/Zero2.

libcamera support is here: #197
How can one access the hardware encoders? Feel free to comment in the other PR.

This was referenced Nov 14, 2021

Support Raspberry Pi Zero 2 W #168

Open

Build universal Raspberry Pi 0 image #44

Closed

htruong requested a review from dgsiegel November 14, 2021 05:10

htruong marked this pull request as draft November 14, 2021 08:18

sazarkin reviewed Nov 29, 2021

View reviewed changes

dgsiegel requested changes Nov 29, 2021

View reviewed changes

HunterAP23 mentioned this pull request Dec 5, 2021

Support for the Nvidia Jetson Nano (2GB and 4GB) #174

Open

dgsiegel mentioned this pull request Dec 8, 2021

getting error while building image file as mentioned in Development & building section #175

Closed

dgsiegel linked an issue Dec 8, 2021 that may be closed by this pull request

Support Raspberry Pi Zero 2 W #168

Open

htruong added 4 commits December 16, 2021 11:52

Kernel: Upgrade kernel to ffdfc6e6

0c3ea9e

This kernel is needed for Raspberry Zero 2/2w support, yet doesn't break traditional camera module. It includes the device tree files for this board.

gh workflow: Fix checksum step

cf4b288

The double quote doesn't allow the wildcard expansion, causing the release to fail, so remove the double quotes. Fixes showmewebcam#170

htruong force-pushed the user/htruong/pi0_2_kernelconfig branch from c4f7142 to cf4b288 Compare December 16, 2021 19:54

dgsiegel mentioned this pull request Jan 14, 2022

Allow configuration of resolution, framerate, white balance, brightness geerlingguy/pi-webcam#4

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kernel build re-architecture + Pi 0 2w support + Pi 0 Universal build #173

Kernel build re-architecture + Pi 0 2w support + Pi 0 Universal build #173

htruong commented Nov 14, 2021 •

edited

sazarkin Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel Nov 29, 2021

dgsiegel commented May 9, 2022

kbingham commented Jul 11, 2023

dgsiegel commented Jul 12, 2023

		@@ -0,0 +1,29 @@
		# Please note that this is only a sample, we recommend you to change it to fit

Kernel build re-architecture + Pi 0 2w support + Pi 0 Universal build #173

Are you sure you want to change the base?

Kernel build re-architecture + Pi 0 2w support + Pi 0 Universal build #173

Conversation

htruong commented Nov 14, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgsiegel commented May 9, 2022

kbingham commented Jul 11, 2023

dgsiegel commented Jul 12, 2023

htruong commented Nov 14, 2021 •

edited