Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boards/RPi: switch to the u-boot aarch64 combined image #192

Open
wants to merge 2 commits into
base: development
Choose a base branch
from

Conversation

mupuf
Copy link

@mupuf mupuf commented Sep 19, 2022

The current version of the raspberryPi-aarch64 image only works on a couple of RaspberryPi boards: 3b and 4b.

Adding support for other boards can be achieved by using u-boot's rpi_arm64_defconfig, which supports:

  • Raspberry Pi 3b
  • Raspberry Pi 3b+
  • Raspberry Pi 4b
  • Raspberry Pi 400
  • Raspberry Pi CM 3
  • Raspberry Pi CM 3+
  • Raspberry Pi CM 4
  • Raspberry Pi zero 2 w

This however requires shipping the .dtb files along with the overlays inside the shared image, which this commit also adds.

Unfortunately, the raspberrypifw needs to be updated to achieve this goal, which explains why this commit is marked at WIP:

  • some of the .dtb files are currently missing
  • the start4.elf file is too old for the CM4

@Strit
Copy link
Contributor

Strit commented Sep 19, 2022

You should probably rebase against the development branch, instead of the release branch.

@samueldr samueldr changed the base branch from released to development September 19, 2022 19:55
@samueldr
Copy link
Contributor

samueldr commented Sep 19, 2022

(changing the branch brough yet-to-be merged documentation commits, don't worry too much until it's "ready")

Converting to draft since it's WIP.

@samueldr samueldr marked this pull request as draft September 19, 2022 19:56
@mupuf
Copy link
Author

mupuf commented Sep 20, 2022

Thanks @Strit and @samueldr :)

My commit is pretty much ready (aside from the lack of documentation update), but what I don't know how to go about updating NixOS's raspberrypifw package. Is this something I have to ask from NixOS/you guys, or something I can do myself?

@samueldr
Copy link
Contributor

samueldr commented Nov 1, 2022

Hi, sorry it took a while to come back to this.

but what I don't know how to go about updating NixOS's raspberrypifw package

If we directly depend on the "thing", we should import the package locally. Like we're doing with a few things like TF-A and firmwarey bits. I think we should own a copy of the firmware packages to ensure that we are testing specific versions, and that we can upgrade them without causing other changes.

The main things we should rely on from Nixpkgs should be the toolchain, e.g. compilers and generic libraries for toolchain use mainly.

@samueldr samueldr added this to the [blocked or WIP] milestone Nov 1, 2022
@mupuf
Copy link
Author

mupuf commented Nov 1, 2022

I think we should own a copy of the firmware packages to ensure that we are testing specific versions, and that we can upgrade them without causing other changes.

That sounds like the most sensible approach, indeed! I didn't want to sound rude though, so I did not propose it :p

The current version of the raspberryPi-aarch64 image only works on
a couple of RaspberryPi boards: 3b and 4b.

Adding support for other boards can be achieved by using u-boot's
`rpi_arm64_defconfig`, which supports:

 - Raspberry Pi 3b
 - Raspberry Pi 3b+
 - Raspberry Pi 4b
 - Raspberry Pi 400
 - Raspberry Pi CM 3
 - Raspberry Pi CM 3+
 - Raspberry Pi CM 4
 - Raspberry Pi CM 4s
 - Raspberry Pi zero 2 w

This however requires shipping the .dtb files along with the overlays
inside the shared image, which this commit also adds.

Unfortunately, the raspberrypifw package is now outdated, so rather
than depending on nix OS packages, we also vendor our own copy of it.
Tow-boot's user interface requires USB support to interact with the
boot configuration menu, which is unfortunately disabled by default
on the CM4, as can be seen in page 7 of the CM4 datasheet[1]:

    The firmware disables the USB interface by default to save power.
    In recent versions of Raspberry Pi OS (Bullseye) it is
    automatically enabled by the otg_mode=1 setting in the config.txt
    file. If you are using a different OS, or an older version of
    Raspberry Pi OS, you will need to add this to config.txt to enable
    the USB interface.

In my experience on a CM4, setting otg_mode=1 was not sufficient to get
my keyboard to work in both Linux or Tow-Boot, while using
worked in all cases `dtoverlay=dwc2,dr_mode=host`. This is why I opted
to use this setting.

As per this comment[2], it also seems to apply to the CM4S.

[1] https://datasheets.raspberrypi.com/cm4/cm4-datasheet.pdf
[2] https://www.jeffgeerling.com/comment/31588#comment-31588
@mupuf mupuf changed the title WIP: boards/RPi: switch to the u-boot aarch64 combined image boards/RPi: switch to the u-boot aarch64 combined image Jan 31, 2023
@mupuf
Copy link
Author

mupuf commented Jan 31, 2023

@samueldr: Sorry for the delay!

I think I have done what you wanted me to do, although I must admit that I am wholly unfamiliar with the nix build system. I merely duplicated what you did with the armlogic, and pretty-much copy/pasted the upstream raspberrypi-firmware package (I just kept the relevant files and dropped all the rest).

In any case, this is a nice improvement: it works both on my cm4 and my raspberry Pi 3+

@mupuf mupuf marked this pull request as ready for review January 31, 2023 09:56
@mupuf
Copy link
Author

mupuf commented Feb 13, 2023

@samueldr: gentle ping :)

@oneingan
Copy link

oneingan commented Mar 4, 2023

Trying to boot a recent nixos iso image: https://hydra.nixos.org/build/211623536/download/1/nixos-minimal-new-kernel-no-zfs-23.05pre460365.3c5319ad3aa-aarch64-linux.iso using your PR in a Rpi B3+.

OS Stage 1 boot hangs in:

waiting for device /dev/root to appear...................
Timed out waiting for device /dev/root, trying to mount anyway.

EDIT:

More tests:

  • same above ISO in a Raspberry Pi 3 Model B with Tow-Boot 2021.10-005 Release works ok,
  • same ISO, Rpi 3B with your PR, hangs exactly as in Rpi B3+ (above error).

EDIT 2:

Adding upstream_kernel=1 to config.txt make system boot but weird keyboard behaviour

@mupuf
Copy link
Author

mupuf commented Mar 6, 2023

Thank you for testing the series!

Adding upstream_kernel=1 to config.txt make system boot but weird keyboard behaviour

upstream_kernel=1 will be trying to use the upstream dtbs filenames which I believe are missing from raspberrypi-firmware (source). A great read is this issue in the rpi firmware project.

Not sure what defaults the tow-boot project should go for (upstream or vendor). I think we should prefer upstream, given that the vendor-recommended way of booting the vendor-recommended kernel would be directly through config.txt.

If we go this route, this would mean we need to create another nix package to download kernel-generated dtbs for the RPi, and we'll need to keep them relatively up to date.

Anyway, mind trying to add the upstream dtbs and see if this solves your "weird" keyboard behaviour?

@oneingan
Copy link

oneingan commented Mar 6, 2023

Thanks for reply, some interesting things I've discovered:

I put the mainline dtbs in upstream/ but they were ignored.

Reading docs seems like upstream_kernel=1 in Tow-Boot is not loading mainline dts, because kernel= is declared. So it only load upstream.dtbo from overlays directory. I've confirmed this by deleting that file and boot hangs again waiting for /dev/root.

Also reading this comment #53 (comment) I'm starting to be confused about how Tow-Boot wants to manage this.

@mupuf
Copy link
Author

mupuf commented Mar 6, 2023

Thanks for reply, some interesting things I've discovered:

I put the mainline dtbs in upstream/ but they were ignored.

Ha, good that you are starting to figure out what is going on!

Reading docs seems like upstream_kernel=1 in Tow-Boot is not loading mainline dts, because kernel= is declared. So it only load upstream.dtbo from overlays directory. I've confirmed this by deleting that file and boot hangs again waiting for /dev/root.

I see! So, the adaptation overlay is not compatible with newer kernels anymore? I can't say I am surprised, as this definitely feels like a hack.

Would be good if you could figure out if using the upstream dtbs actually solve your keyboard weirdness!

Also reading this comment #53 (comment) I'm starting to be confused about how Tow-Boot wants to manage this.

I agree with the objective of the project. This BS should not go in the way of users!

According to https://elinux.org/Device_Tree_Linux#forward_and_backward_dts_compatibility, newer kernels need to work with old dtbs, but the opposite is not guaranteed. So, from this PoV, we want the kernel we want to boot to come with its own dtbs (so that they are the most feature-complete), but we could also select a kernel that we consider good-enough for most boards and vendor its dtb.

The problem with asking kernels to come with their own dtbs is that, of course, the firmware doesn't know ahead of time which dtbs to load... so that means tow-boot would have to load the right one once it would have selected the kernel to boot. I wonder if there is a way for us to know which board we are running on so that we could be responsible for loading the right dtb (and overlays) at runtime (there must be an RPC for that, but not sure if it is documented).

Additionally, if we can't rely on config.txt to select which overlays to load, we'll have to add a way for users to select which overlays they want to load at boot time, akin to what a BIOS config screen would be. This may be a little too much to ask from tow-boot?

@mupuf
Copy link
Author

mupuf commented Mar 6, 2023

@samueldr: May we ask for your thoughts on this matter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants