Kernel panic - not syncing

[liwde]

Platform

I’m using: Raspberry Pi 1 (Community-supported, but I hope that I can find help here)

Observed behavior

I initially set up my gokrazy instance in September, which worked flawlessly. Updating all packages this week, the instance didn't boot properly any more. Booting into Raspian works without problems, so the hardware doesn't seem to be an issue.

Rather, the problem seems to be within gokrazy/init:

Run gokrazy/init as init process
Kernel panic - not syncing: Attempted to kill init! exitcode 0x00000004
CPU: 0 PID: 1 Comm: init Not tainted 6.1.21 #1

I've tried to overwrite the sd card, but that didn't fix the issue. I also triple-checked that I set the correct go-arch and didn't change anything else in my configuration. Unfortunately, I don't have a backup of the builddir to restore the old versions.

Can you point me to what I should be trying to debug this? Is there a good way to downgrade individual packages of gokrazy to older versions to bisect where the problem could have come from?

Thank you for your support and for maintaining gokrazy!

[damdo]

Hey @liwde gokrazy officially supports only Raspberry Pi 3 and greater.

Raspberry Pi 1 is not officially supported and its support is instead a community effort at: https://github.com/gokrazy-community so this question may be best directed at https://github.com/gokrazy-community/kernel-rpi-os-32

That said, among other gokrazy devices, I also run gokrazy on a Raspberry Pi 1, and I've also been facing panic issues trying to upgrade its packages recently. I ended up downgrading back to the last known working configuration.

Here is the version I'm pinning things to:

github.com/gokrazy-community/kernel-rpi-os-32 v1.0.2-6.1.21

github.com/gokrazy/gokrazy v0.0.0-20230417151844-a7649541429e
github.com/gokrazy/internal v0.0.0-20220129150711-9ed298107648

Also I'm using go 1.20 for this rather than 1.21, but maybe that works anyway.

[stapelberg]

In your picture, it looks like the gokrazy init output might have gone to the serial port, but you don’t have a serial adapter connected?

Can you set "SerialConsole": "disabled" in your instance config? https://gokrazy.org/userguide/instance-config/#serialconsole

That might get us a better error message on your HDMI output.

[liwde]

@damdo, thank you for the advice, I'm trying to downgrade the versions, but haven't managed to succeed, yet... I changed the version in all go.mod files I found in the builddir and deleted the go.sum files, but that seemingly wasn't enough? How did you do the downgrade? I also tried this with a completely fresh gok project, and tried to use the replace directive to specify the versions you gave me, but that didn't change the output.

@stapelberg, that setting is already in place, I never had a serial adapter for my pi. What output should I be expecting? Is it this here? https://github.com/gokrazy/tools/blob/80a59f115332a41206487afc6038beaaed48ce70/internal/packer/buildinit.go#L36

[stapelberg]

Changing the versions in the go.mod files in the builddir should be sufficient. Deleting go.mod is not required. Using the replace directive is not required.

It’s weird that you don’t get any init output. I’m wondering if we’re compiling with a too-new GOARM setting or something like that, which would result in programs that can’t be run on the target hardware.

[connickshields]

Just adding here that I am having the same experience with a kernel panic on both an RPI 3B+ and 4B, using the default config and also a custom config I was working on.

I need to find a mini HDMI cable to view the HDMI output on the 4B, but the 3B+ was showing a kernel panic. On both devices the green SD card LED turned off 15-20 seconds after power on.

Also I will note that I am building my SD cards on macOS, not Linux, if that might make a difference. I will post here again once I have a photo to share.

[stapelberg]

That’s very strange. I’m running a bunch of Pi 4s without problems.

Yes, let us know when you have an error message to share. Please also provide a tar or zip archive of ~/gokrazy/hello, or whereever your config.json + builddir lives. That will allow us to build your instance to reproduce the problem.

[liwde]

I will try to find time to check with different version of the gok cli and go itself, but haven't found the time yet... Just to keep you updated.