-
-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pine64-rock64: Add support for ROCK64 #155
base: development
Are you sure you want to change the base?
pine64-rock64: Add support for ROCK64 #155
Conversation
Hi! 👋 Thank you for your contribution! I know there's no procedure to follow for contributions (yet), so please bear with these few questions: Can you tell me what was tested?
cc @Absolucy who asked about ROCK64 support previously. |
Updated New changes happens in the |
I actually have to admit I'm having some trouble compiling it. Wanted to open the PR so that people watching #151 would be aware it's in progress, and to ask for help. It's failing with the following config messages:
|
I see two general issues in there, for which I need to ask questions:
cc @mirrexagon, I kinda forget what the actual state of what you were doing is. |
|
Regarding SPI flash:
[1] I don't remember exactly where I saw it, but maybe one of the patches referenced in a review someone did of my patches notes that the |
Thank you for the brain dump. About [1], The documentation bit was previously in README.rockchip, Which added support for RK3288. |
Tentatively putting in the milestone for the next release. It might be possible to do this as I was provided with a ROCK64 by someone (not sure if they wanted to stay anonymous) to support the platform. I'll try poking at it soon, once I have the Tow-Boot patches rebased on top of |
About HDMI: there's no rk3328 HDMI driver in U-Boot. So this won't be a trivial thing to get going. I don't really know the platform yet, but if the subsystems on the 3328 are close enough to 3399, it might be possible to handle. Though a quick glance at the Linux driver makes me think it's going to take some work or someone that already is familiar with it. |
Note that I'm getting the same build failure as @moosingin3space was, and I can't really test this as a result. |
I'll post an update soon once I better get the subtleties of SPI on the Rock64. I have the shared image going already. |
This is useful for targets where there's no display support, rather than dig in the implementation details (e.g. withLogo).
Their upstream configs may make this awkward, as the required configs for these configs to be available will not be active.
@moosingin3space I went ahead and:
I took the liberty of force-pushing on this PR branch, your changes (except for the rock64 commit) are untouched. The rock64 commit only has relevant additions. If you have any concern with the changes, don't hesitate to ask. |
What works
What is untested
What doesn't work
|
They end up working better when power is provided. How does this make it work? 1. Make the DWC3 driver work with rk3328 (by stating it will) 2. Add the necessary supply nodes to the DT Note that the innate FDT Linux will not enable USB ports. So while Linux will boot, it will likely not be able to mount what's on the USB stick, unless it is loading another FDT. This is because the U-Boot FDT conflicts with the upstream drivers. To work around the issue, use any upstream method to load another FDT, some listed here: - https://source.denx.de/u-boot/u-boot/-/blob/v2022.04/doc/develop/distro.rst Since the issue stems from the innate FDT being the U-Boot-modified FDT, the core issue is not specific to the Rock64 board. A generic solution will need to be explored, as using the innate FDT may not work as expected.
I've looked quickly at HDMI, and reading the Linux drivers, it differs too much from the existing RK3288 and RK3399 implementations to be a trivial thing to implement. So for the time being, it'll have to be used through serial :/.
They have a totally different driver model for video in the Rockchip BSP. It's named |
Okay, that's pretty interesting. With this fix, I was able to boot the Fedora Server 36 netinstall image when plugged into one of the USB 2 ports. USB 3, indeed, does not work, but my drive is powered on this time. Looks like it's a bug in the u-boot XHCI driver. Thankfully, it appears as though someone has already found a solution to the Linux bug: https://artemis.sh/2022/05/16/everfrees-armferno.html (mentioned under the |
Aside, it appears as though JTAG headers are exposed on the Rock64: https://crwulff.blogspot.com/p/rock64.html This could help significantly with debugging some of these drivers. |
The underlying issue is a bit more subtle than implied in that post. The problem is that the innate FDT (the built-in device tree) is built for U-Boot, with quirks added. There is a quirk added by U-Boot disabling the always-on USB power. This is what the patch from ayufan works around within U-Boot. So a workaround is to load the device tree from a mainline Linux A solution would be to implement a generic method of "unquirking" a for-U-Boot device tree before booting an operating system. Doing so would be pretty much equivalent to loading the mainline Linux device trees as they are regularly synced in U-Boot with the mainline Linux trees. This would be helpful to keep the concerns separate about the U-Boot-specific device tree, and the generic innate device tree. |
I may be dumb, but I can't seem to get any serial output? Tried flashing both spi.installer.img and shared.disk-image.img to a microSD. ROCK64 turns on, but no output over UART. 112500 baud rate and all. edit: i think my cables are just too long for uart? |
It's 115200, if that wasn't a typo. Though it's plausible that length can be an issue for UART, too. Also check if RX/TX weren't swapped, it's IMO always weird with the labeling. |
Thanks for the explanation. I have a number of questions, as my experience with U-Boot isn't that extensive.
So basically, U-Boot loads a device tree that has certain quirks, ostensibly to account for differences between U-Boot and Linux drivers, and then Linux loads the same one from some specified address?
To clarify: in the case where I have a UEFI-booting distribution image (i.e. Fedora), this implies that U-boot must load the device tree? I was under the impression that in the case of something like the Fedora AArch64 generic image, GRUB (the bootloader it ships with) would handle booting Linux with the mainline device tree. Perhaps I just need to build a minimal distro with Yocto (or even Nix -- I noticed your work on
This would be something done upstream, I assume. Would it basically result in the same effect (loading the mainline device tree when booting Linux)? Or would it move the responsibility into the OS? |
Those are "heavy" questions, and this will derail the discussion a bit, and possibly exponentially grow. Though let's try and answer them. Some of them will stretch my verified knowledge and might slip into assumptions. A lot of the answers will be about my opinionated ideals that are currently mostly fiction in the ecosystem.
Yeah~ish. Here U-Boot does the loading, and tells the OS (Linux here) where the FDT is in memory. When there is no "loading" done, e.g. with UEFI by default, U-Boot will re-use what I call the "innate" FDT, the one that U-Boot was built with. We'll see that Linux completely absolving itself from the task of loading the correct dtb file is a recurring theme, and a problem.
Any component before the OS can load the FDT in memory. Though there is a severe dysfunction in that there is no way to know which dtb file to load. So when targeting generic boot, you can't get GRUB to e.g. load the Pinebook Pro dtb on the Pinebook Pro, GRUB (and pretty much all tools except U-Boot) have no way to ask "who am I loading this for?". I have not verified what the actual Fedora iso does, I suspect either it does nothing special, or it adds the " So, how does U-Boot does it? "Heuristics"! Generally speaking, the platform name is either entirely hardcoded, or there is a small detection routine in a board-specific file that will distinguish between few compatible platforms (e.g. Pinephone A64).
Development could happen here, but it should probably be coordinated with upstream so that it is acceptable to them. The effect wouldn't exactly be "loading the mainline device tree when booting Linux", but it sure would look similar to it. The effect would be that we'd undo the "quirky" changes U-Boot applies (e.g. see
You hit the nail on the head. We're now diving into my opinion on the subject matter. I think there is one deep flaw that needs to be solved with the trifecta Device Tree, Linux, Generic Boot: the kernel wants its own device tree representations, but does absolutely nothing to make it easy or possible. Everything has been authored with an extremely embedded mindset, where the platform is tightly vertically integrated. I might be missing some things, but basically I think that dtb files shouldn't be handled by the bootloader stages, since they have no way to know which file to load (or a protocol devised to be implemented by the bootloaders). One way I've thought up to handle this would be to bundle all dtb files from a kernel build in a cpio archive, and load it with the kernel "before" the initramfs. The kernel in its earliest init would look at the current "facts" available about the platform, and somehow select the right dtb file. That somehow is load bearing. Note that the same exact system could be used for ACPI platforms to implement quirks with an "overlay" instead of peppering the whole tree with What facts are there? I don't exactly know, but I have a few in mind. First, a new DT property could be added to short-circuit facts-checking e.g. With such a scheme, you would resolve I believe all Linux-side pains with ARM and Generic Boot. It would be possible to implement the same scheme in a bootloader step, the same strategy is used for WoA laptops [citation needed], but the solution needs to be applied as close as possible to the kernel or else it continues being an issue to any platforms not using the special solution. Another issue with a bootloader-y step is that for OS with generations like NixOS, you might face an issue where you need to first load the "correct" dtb file, but the "correct" dtb file will vary depending on which kernel/system generation you boot next. This would also affect other similar strategies, like I think SUSE does with Snapper. I moved this from the response, as I don't know how to interpret this:
This seems unrelated to the FDT/boot questions. Or do you mean that you want to know how to manually boot an OS using the U-Boot console? It won't be as helpful as you'd hope if you want to boot with EFI, as you'd basically load the EFI program, and run it. And otherwise the boot process is different enough when booted through U-Boot native methods that it won't exactly map to EFI boot semantics. Note that the knowledge learned with the Sorry for the long rambling! If anything needs to be discussed still... I don't know where it should be... But there's the matrix room (#Tow-Boot:matrix.org), where it might be easier to bounce smaller bite-sized questions/answers instead of huge essays :). |
Thank you for the incredibly detailed response, you are teaching me quite a bit about the way that these SBCs work. My experience in this area has been heavily focused on the embedded Linux side, after U-Boot loads an ITB-formatted image with kernel, initramfs, and device tree. That's a very... embedded systems way to do it, versus the generic boot mechanism that you're describing, and that Tow-Boot aims to provide. Bringing this conversation back to this specific PR's goal, bringing up Tow-Boot on a ROCK64, and the questions regarding booting a Fedora Server ISO, I opened the ISO with
I can't tell, from this, how it decides what device tree to load. Decided to look at the boot loader for the raw image:
This doesn't appear to specify a device tree either. It looks to me as if Fedora, when booted with UEFI on AArch64, expects to be able to find ACPI tables. There's definitely a device tree binary located on the boot partition at So I guess the next question is: what's next for this PR? |
I'm not sure. I don't like the fact that the innate FDT is "wrong" as far as Linux is concerned, but it's not like it's something that's caused by changes from Tow-Boot. So I should ignore that detail since it's the same with upstream. I don't like that HDMI out doesn't work... but same story, upstream problem. So really, it's probably as ready as can be. |
Fedora AFAIK on AArch64 "expects" ACPI tables just as much as other "standards-based boot" distros do, really DT is fine too. Is the boot partition you're speaking of the ESP? If so, U-Boot itself (and thus Tow-Boot) should load the There are no mechanisms available after U-Boot relinquishes control to determine the "correct" dtb file name to load. So any UEFI program loading a dtb file would need to hardcode the platform, so it's not an option for generic boot images (and should be seen as a workaround for device-specific systems). |
It is not - the ESP contains grub with a grub config that mounts a different (XFS) partition. That XFS partition contains the dtb. I am not sure why it doesn't just use the ESP for storing all the boot resources. Does this imply that if I were booting a distribution that placed the dtbs on the ESP, U-Boot should load Linux's device tree according to heuristics? |
Yes, this is precisely what the code aims to do. Note that the |
❌ This can't be merged as-is. A discussion with @mirrexagon and other peeps in the NixOS on ARM room floated up an issue I had forgotten about: there are variants that needs slower RAM init, or else the board might exhibit issues. See this discussion: So this will require fishing-up the proprietary ram training bits from other WIP branches. |
Hi, sorry for the silence. I will have to look through ayufan's commits to see if there are any patches relating to RAM init across variants, or even detection of the variants with this requirement. |
Moving this in Also noting that downstream-upstream there are further changes related to RAM init that we might want to look at Namely, I wasn't aware of it for lpddr3, mainline U-Boot already has multiple blobs for RAM init with different speeds available. I assumed there wasn't from experience with #54 where it's not the case for lpddr4. So I want to take proper care to verify this. We'll also need to figure out how we handle differing RAM init... Do we build multiple builds? For now I guess we can build with the single slower RAM init and push that question as something to be answered later. Maybe an "applet" within the future configuration menu that can be used to replace the blob in-situ? Though that would be hard and prone to errors. |
Is there documentation from Pine64 (or Rockchip) about how the different variants require different raminit? |
I don't know of any. Tried looking, and no official word AFAIK. It would be from Pine64. There are multiple locations stating v2 is the problematic revision. ¯\_(ツ)_/¯ |
At this time, what are the remaining issues before this is in an acceptable state? From what I understood:
Looking at https://wiki.pine64.org/wiki/ROCK64#Information,_Schematics_and_Certifications, I can see two board variants ( I have a ROCK64 2GB, but I'm not sure which variant it is; is there anything I can help with? |
As mainline U-Boot contains support for the ROCK64, I added support for it to Tow-Boot.
Fixes #151