Pi 5: i2c: i2c_designware: controller timed out, requiring power cycle to fix #5784
Comments
|
This bug has just happened to me on 6.6.5 too. |
|
After the first instance, before I had the logic analyser attached (running a simple program to mirror the levels of GPIOs 40&41 onto GPIOs 6&7), I can't get this to fail. Whether I load the overlay from config.txt or at runtime using "sudo dtoverlay hyperportal10" (*) doesn't seem to make a difference - it's still solid. The only errors I see are: The first of these is because the Goodix controller NAKs the first read, not surprising since it happens less than a millisecond after the reset is released. The retry that happens about 37ms later is successful. It may also be worth saying that I see the backlight turn on, and a flickering band of largely white pixels about 6cm wide across one side (in landscape) of the display. (*) The overlay needs some sanitisation before the kernel will accept it at runtime, but it ends up looking like this: |
|
And then it failed hard, with SCL high and SDA held low: Until I flipped the catch on the touchscreen cable and it sprang into life: In other words, I think the touchscreen controller (or perhaps the PWM/GPIO chip) is holding the SDA line low and causing the bus to malfunction. It's hard to tell which, since they are both on the other end of an I2C bus and my bucket has a hole. |
|
Oh good grief. Thank you for looking into this. I should have known it would be our hardware. This would explain why it took a full power-cycle to recover. The display flickering is expected. It works on Pi 4 (possibly due to timings being more forgiving) but I have not yet found the right magic incantations to get it working on 5. At a guess I'd pin it on the touchscreen controller. It's quite possible that using a vreg instead of a proper reset pin assignment causes much more trouble than I'd imagined. I will have to dig deeper into the reset logic of the driver and reconcile that with #5777 |
|
@Gadgetoid Hi, I have a similar issue with i2c_designware on Pi 5 when plugging in a Waveshare 4.3inch Capacitive on a custom kernel 6.1.77 I built from here. I was wondering if you had any update on this. This doesn't happen on Raspberry Pi OS Bookworm, but only happens when I custom compile the kernel. Even tried copying and using exact dtbo from Bookworm, no help. @pelwell I also saw contributions by you, Phil, regarding these topics, it would be great to have some insight on the issue I am having, which seems similar here. SCL held high and SDA held low. Help is greately appreciated. Thanks |
|
You're asking me to guess what you have got wrong in your configuration, in a forum which is intended for reporting problems in our kernel, not for general help requests. You need to put a logic analyser on the I2C lines to see what is going wrong, which may then help you figure out why. But if you are getting the same timeout errors as above then it's likely that the device is holding one or both of the lines low when it shouldn't. |
|
@pelwell Okay, thanks regardless. Will do. |
For general help requests, the forums are the right place to ask. forums.raspberrypi.com. You are likely to get a faster response as well, since there are more people there with knowledge than hang around here. |
|
The problem I have described above (not the original) is fixed when Linux kernel is downgraded to 6.1.63 (official Bookworm kernel version that worked). I guess the fixes in i2c_designware that was introduced after 6.1.63 regarding I2C timings doesn't really get along with some of the devices, causing "i2c timed out" issues. |
|
I have the exact same problem with the 4.3 inch screen from waveshare. Most of the times it works but sometimes the display will not show anything and I will just keep getting these i2c timeouts. |
|
As I said above:
If you are also on a Pi 5 then you can check the state of the I2C lines with: Run this command next time the display doesn't work. One of the pairs - probably 40&41 - will be SDA and SCL, and all of the signal levels should be |
|
Is there any fix for that, I mean how would you know waveshare hardware but maybe there is a way? Or just don't buy waveshare things ever again? |
|
If (and it currently is an "if") the hardware is permanently holding one of the I2C lines low, possibly as a result of an incomplete reset, then there is nothing we can do. If it turns out that the device is intentionally using clock stretching and that it is exceeding some time limit programmed into the I2C controller then there may be a possibility to increase the time limit. |
|
Is there any way to tell the pi to actually reset the screen? Like a "dsi-reinit" thing? |
|
Also the funny thing is that for some reason even after 50 reboots the display won't reset. In some cases it crashes so bad I have to format the SD card for it to work again. |
|
I don't know if there is a generic way to reset a display, but I doubt it. The DSI interface is only for pixel data, the control signals must go via I2C or SPI etc. I feel the only reliable reset mechanism is with a dedicated reset line driven by a GPIO. The driver for the display must be told which GPIO to use, usually via Device Tree. |
Hmm what do you think of that? I understand the crash of the controller due to the display but even if I unplug power let it sit there for a bit for the capacitors to run out, still the display won't show anything. It's like it remembers not to work or something. Also because I am fed up with waveshare and their products, is the support for displays like the hyperpixel one better? Would you recommend that driver-wise? |
|
Whilst I don't have the Waveshare 4.3" panel, I do have a couple of the others (2.8", 7.9", and 10.1") You have several largely independent items making up the panel.
There are no commands sent over DSI, just image data. I will check the latest kernels against my panels on a Pi5 to see if they also show issues. HyperPixel displays are DPI, not DSI, so you lose almost all your GPIOs. |
|
Yeah it takes all the gpio pins. Well everything leans towards the official display for me or a solution to the waveshare panel. If you have any hardware mod for an x solution that I could try on the screen part I will be happy to do so. |
The first of those statements makes it sound like an occasional problem, the second makes it sound almost permanent. What's the reality? |
|
It's kind of both. When the problem appears (usually after a reboot) I can't do anything, if I am lucky after 2-3 reboots or unplugging the cable it works if not I just format the SD card because I can't do anything else. The weird thing is that it doesn't seem to happen at all with a pi4. |
|
If the panel isn't responding, do some basic diagnostics of why.
Now that you say it, I do vaguely recall warm boots on that panel causing problems on I2C at one point. It's just possible that the order in which signals come up are being misinterpreted as an I2C start by the Waveshare MCU. |
|
So the problems I have found are a) controller will just keep failing and vnc will not even open b) |
|
Soo it happened again. Here is the output of the Dmesg: |
If that state is semi-permanent (you can monitor the line for changes with |
|
Is there any way to reset it? Reboot doesn't do anything... |
|
You are asking for help with a third-party display I don't possess. Any suggestions, @waveshare? |
|
Yeah indeed, well I sent an email to waveshare detailing everything, hopefully they know how to fix their own hardware... |
|
Oops, thought you were using the Waveshare displays that don't emulate the Pi display, but you're not (it's the 800x480 Pi panel clone). Sorry, no idea on those panels. If the pictures on https://www.waveshare.com/4.3inch-DSI-LCD.htm are correct, then it's also using the ICN6211. I don't see a MCU on there though, so I don't know what they're doing there. It's also possible that they have updated their code to avoid I2C issues. |
|
Unfortunately waveshare's product quality is so good that its one leg broke of because apparently you are not supposed to screw it in all they way for it to balance? And the other screw pad just fell off, tried to solder it, no luck. Here is a picture of @waveshare quality:
to be honest tho I tried to screw it using a small drill because it wouldn't screw in all they way but I barely clicked the button, it was from these analog drills that turn as much as you press the button. But still it only did one turn, one turn and it broke, I have done this with other displays and they were ok! |
|
I don't want this issue to degenerate into a criticism of Waveshare - it's chatty enough as it is. |
|
Yeah anyway I don't have anything to test on so thank you for your help. |
|
I'm encountering a similar issue to steveiliop56 with a Raspberry Pi 5 and the Waveshare 7inch DSI LCD (C) https://www.waveshare.com/wiki/7inch_DSI_LCD_(C) on the Raspbian OS Lite.
After approximately every fifth reboot, the display stops working, and subsequent reboots fail to bring it back up. Upon failure, the pinctrl output reads: The screen resumes working only after physically reconnecting the Raspberry Pi power supply. After hours of attempting to troubleshoot how to bring the display up after a reboot that didn't start the screen, I discovered that the only solution is to execute the following command: Subsequent reboots then successfully bring the display back. Nonetheless, I assume this is not an ideal solution. |
That's an interesting find, and may help others in the same situation. If you have the time, I would be interested to hear if
No, it's definitely not ideal. |
|
I am able to reproduce this symptom using different slave devices (MCP23017, PCA9675, and GP8403). For me it results repeated "controller timed out" and "i2c_dw_handle_tx_abort: lost arbitration" messages in dmesg. After running Using |
|
I can confirm that SDA1 remains low after executing pinctrl 2-3 ip, and the display did not come up after the reboot. Here are the results: |
|
I have a similar problem when AQM0802-based LCD is connected to Pi 5 with recent kernel (6.6.20+).
In my case, I could avoid this tourble by adding the following configuration in /boot/firmware/config.txt
I felt that 20000 is more stable. Timing might be severe on Pi 5 with recent kernel. |
|
FYI there's a PR (#6050) that may help with this issue. Similar failures are seen when the device fails to ACK data in time, and the PR changes the mark/space ratio on SCL to give the device longer to respond at any given clock speed. |
|
Dear pelwell, Thank you for creating the pull request. I tried "sudo rpi-update pulls/6050". When my device stops responding to my program, only the folloiwng error appeared in dmesg.
After that, I tried "i2cdetect -y 1", and the following output appeared. When "--" is shown, the following error appeared.
So, after "i2cdetect" finishes, the above error appeared 24 times. |
|
That's not a failure mode I've encountered. At this point I think you should open a new issue, explaining the behaviour you were seeing. |
|
So is the issue fixed with the new 6.6 kernel? Because that's the response I got from waveshare was to try with 6.1 kernel and replace the dtb file with one from here (that was before a week tho). |
|
@steveiliop56 I just tried copying over the dtbo file from 6.6 I am running into a different type of error. Anyone there to help?
Output of Output of My config.txt file: Linux kernel version: Confirmed that I have waveshare overlay: Here is the Problem: my screen is blank. My Qt app says "No screens available" or "No DRM device" depending on which vc4-* overlay I load. Ahh, I'm out of ideas and lost. If anyone can give me any sort of hints that helped them, I'd appreciate it! |
|
@talksik 2 things:
The original config.txt that you posted is MASSIVE! My memory says there is a maximum size the bootloader will load, so trim it down substantially. |
On Pi4 the config.txt file should be <= 4KB, otherwise, network boot will truncate it. On Pi5 there an individual config.txt file (or include) can be up to 16KB,. |
|
I'm not aware of any outstanding Pi 5 I2C problems using the latest kernel, so I'm tentatively closing this issue. |
Describe the bug
When using i2c on either of the display connectors to drive an IO expander and touch, I'm getting frequent hard crashes of the i2c controller, requiring a full power cycle to fix:
Steps to reproduce the behaviour
I don't have a minimum repro since I'm connecting and trying to talk to i2c hardware, normally I'd chalk this up as user-error but I'm concerned that a fault like this should be cleared on a soft reboot...
This is the dtoverlay that may or may not be triggering the problem-
Device (s)
Raspberry Pi 5 Model B Rev 1.0
System
Logs
Full dmesg output of a failure state:
Additional context
No response
The text was updated successfully, but these errors were encountered: