[GDB] Debugger seems to crash when stepping inside an IRQ handler

[lutrampal]

Hello there,

Describe the bug
When setting a breakpoint inside an IRQ handler in my application, I'm experiencing the following crash each time I try to continue, step over, step into or step out of the handler:

/__w/_temp/workspace/build/.build/arm-zephyr-eabi/src/gdb/gdb/infrun.c:5825: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

Quit this debugging session? 
(y or n) [answered Y; input not from terminal]

This is a bug, please report it.
  For instructions, see:
<https://github.com/zephyrproject-rtos/sdk-ng/issues>.

/__w/_temp/workspace/build/.build/arm-zephyr-eabi/src/gdb/gdb/infrun.c:5825: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.

Create a core file of GDB? 
(y or n) [answered Y; input not from terminal]
GDB session ended unexpectedly. exit-code: none, signal: SIGABRT

Here's the GDB coredump: core.zip

Please also mention any information which could help others to understand
the problem you're facing

• Target CPU: NRF5340 Application core
• Zephyr SDK revision: 0.16.4 (gdb v12.1)
• Zephyr OS revision: v3.4.99-ncs1-1 (Nordic's current fork revision on NRF SDK v2.5.1)
• Debugging probe: SEGGER JLink BASE Compact with J-Link GDB Server V7.94f (latest)
• I'm in the process of updating the NRF SDK and I did not encounter this issue in the previous version used in my project (v2.4.1). I'm also reporting this crash on Nordic DevZone in case this should be handled by Nordic.
• I have tried to replicate the crash with another debugger (SEGGER Ozone) but couldn't. So this seems to be a GDB related issue.
• This may be related to this other issue: gdb/infrun.c:5300: internal-error riscvarchive/riscv-binutils-gdb#230

Steps to reproduce

1. Set a breakpoint inside an IRQ handler
2. Have your code hit the breakpoint
3. Try to step to next instruction after the breakpoint
   => GDB crashes with the reported error
   NB: I was able to consistently reproduce the crash on my setup by setting a breakpoint in several distinct IRQ handlers. The crash does not occur when debugging regular code which is why I deduced that the issue is somehow linked to IRQ handlers.

Expected behaviour
Stepping inside function should not crash GDB.

[sprhawk]

same here, also working with nRF5340-DK, with same SDK version.

I set breakpoint inside zephyr uart async api, which is set by uart_callback_set(), it breaks at UART_RX_RDY, and can inspect variables struct uart_event *evt, and crashed when I use next command.

Thread 2 hit Breakpoint 1, uart_lte_callback (dev=0x36f8c <__device_dts_ord_137>, evt=0x20008fe8 <z_interrupt_stacks+1480>, user_data=0x0) at lte_tracker/src/lte.cxx:74
74            if (evt->data.rx.len > 0) {
(gdb) p evt
$1 = (uart_event *) 0x20008fe8 <z_interrupt_stacks+1480>
(gdb) p evt->data
$2 = {tx = {buf = 0x20008380 <rx_buf> "^boot.rom\001'v", len = 0}, rx = {
    buf = 0x20008380 <rx_buf> "^boot.rom\001'v", offset = 0, len = 20},
  rx_buf = {buf = 0x20008380 <rx_buf> "^boot.rom\001'v"}, rx_stop = {
    reason = (unknown: 0x80), data = {buf = 0x0, offset = 20, len = 0}}}
(gdb) p evt->data.rx
$3 = {buf = 0x20008380 <rx_buf> "^boot.rom\001'v", offset = 0, len = 20}
(gdb) n
[New Remote target]
/__w/_temp/workspace/build/.build/arm-zephyr-eabi/src/gdb/gdb/infrun.c:5825: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
----- Backtrace -----
0x55e42716a85d ???
0x55e42745c81c ???
0x55e42745c9fa ???
0x55e42754a3de ???
0x55e4272c5409 ???
0x55e4272c5ea8 ???
0x55e4272c73ea ???
0x55e4273a33cd ???
0x55e42754ae9c ???
0x55e42754b050 ???
0x55e4272f4bdc ???
0x55e4272f6634 ???
0x55e4270a546a ???
0x7f96f6ef8249 __libc_start_call_main
        ../sysdeps/nptl/libc_start_call_main.h:58
0x7f96f6ef8304 __libc_start_main_impl
        ../csu/libc-start.c:360
0x55e4270b5e79 ???
0xffffffffffffffff ???
---------------------
/__w/_temp/workspace/build/.build/arm-zephyr-eabi/src/gdb/gdb/infrun.c:5825: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)

core.zip

[sprhawk]

I tried to build gdb-14.2 (gdb/configure --target=arm-zephyr-eabi) from sourceware.org, same crash but with debugging information:

infrun.c:6265: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
----- Backtrace -----
0x55ad2130a60b gdb_internal_backtrace_1
        gdb-14.2/gdb/bt-utils.c:122
0x55ad2130a60b _Z22gdb_internal_backtracev
        gdb-14.2/gdb/bt-utils.c:168
0x55ad216591fc internal_vproblem
       gdb-14.2/gdb/utils.c:396
0x55ad2165946a _Z15internal_verrorPKciS0_P13__va_list_tag
       gdb-14.2/gdb/utils.c:476
0x55ad217c1ca1 _Z18internal_error_locPKciS0_z
       gdb-14.2/gdbsupport/errors.cc:58
0x55ad21484a72 finish_step_over
       gdb-14.2/gdb/infrun.c:6265
0x55ad21484a72 handle_signal_stop
       gdb-14.2/gdb/infrun.c:6401
0x55ad21485bea handle_inferior_event
      gdb/infrun.c:6114
0x55ad21486e73 _Z20fetch_inferior_eventv
      infrun.c:4466
0x55ad2159679d run_async_handler_and_reschedule
       gdb-14.2/gdb/ser-base.c:138
0x55ad217c2545 gdb_wait_for_event
        gdb-14.2/gdbsupport/event-loop.cc:716
0x55ad217c3032 _Z16gdb_do_one_eventi
       gdb-14.2/gdbsupport/event-loop.cc:264
0x55ad214b00e9 start_event_loop
        gdb-14.2/gdb/main.c:407
0x55ad214b00e9 captured_command_loop
       gdb-14.2/gdb/main.c:471
0x55ad214b2954 captured_main
       gdb-14.2/gdb/main.c:1324
0x55ad214b2954 _Z8gdb_mainP18captured_main_args
      gdb-14.2/gdb/main.c:1343
0x55ad2124f1b7 main
        gdb-14.2/gdb/gdb.c:39
---------------------
infrun.c:6265: internal-error: finish_step_over: Assertion `ecs->event_thread->control.trap_expected' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)

[dastarling]

I also seem to be experiencing the same issue, on nrf5340, single stepping crashes gdb with the same error messages. Any update on this case?

[ufanders]

Any update?

[D-Gulose]

Hey, I experience the same bug working on the nrf5340. I can circumvent it by not setting -Og optimization level, but instead set the configs manually:
CONFIG_DEBUG_OPTIMIZATIONS=y CONFIG_DEBUG_THREAD_INFO=y
Using -Og or the configs should be equivalent, but apparently causes an error.