Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warning: closing connection to zebra because of an I/O error! #15860

Open
DS819src opened this issue Apr 27, 2024 · 0 comments
Open

Warning: closing connection to zebra because of an I/O error! #15860

DS819src opened this issue Apr 27, 2024 · 0 comments
Labels
triage Needs further investigation

Comments

@DS819src
Copy link

Description

The following error message occurs occasionally. I work with network namespaces, instead of vrf.

amenonuhoko# sh ip route vrf all
Warning: closing connection to zebra because of an I/O error!
Warning: connecting to zebra...failed!
amenonuhoko# sh ip route vrf all
Warning: connecting to zebra...failed!
zebra is not running
amenonuhoko# sh ip route vrf all
Warning: connecting to zebra...success!
Warning: closing connection to mgmtd because of an I/O error!
Warning: connecting to mgmtd...success!
Warning: closing connection to ospfd because of an I/O error!
Warning: connecting to ospfd...success!
Warning: closing connection to ospf6d because of an I/O error!
Warning: connecting to ospf6d...success!
Warning: closing connection to bgpd because of an I/O error!
Warning: connecting to bgpd...success!
Warning: closing connection to isisd because of an I/O error!
Warning: connecting to isisd...success!
Warning: closing connection to staticd because of an I/O error!
Warning: connecting to staticd...success!

Version

FRRouting 10.0 (amenonuhoko) on Linux(6.1.0-20-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--sbindir=/usr/lib/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--enable-rpki' '--disable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

How to reproduce

It occurs occasionally.

Expected behavior

Zebra doesn't terminate the connection. This behavior doesn't appear in version 9.1.

Actual behavior

# coredumpctl 
TIME                            PID  UID  GID SIG     COREFILE EXE                          SIZE
Sun 2024-04-21 22:42:05 CEST 969587  111  145 SIGSEGV missing  /usr/lib/frr/zebra              -
Fri 2024-04-26 17:10:09 CEST 825488  111  145 SIGSEGV present  /usr/lib/frr/zebra         762.7K

coredumpctl dump 825488

       PID: 825488 (zebra)
       UID: 111 (frr)
       GID: 145 (frr)
    Signal: 11 (SEGV)
 Timestamp: Fri 2024-04-26 17:10:09 CEST (1min 38s ago)

Command Line: /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000 -n
Executable: /usr/lib/frr/zebra
Control Group: /system.slice/frr.service
Unit: frr.service
Slice: system.slice
Boot ID: 441257eb9d8c4cb9a6eb1e298479c2d9
Machine ID: 975b607fdf964e2aa0af9602a8bb0a9d
Hostname: amenonuhoko
Storage: /var/lib/systemd/coredump/core.zebra.111.441257eb9d8c4cb9a6eb1e298479c2d9.825488.1714144209000000.zst (present)
Size on Disk: 762.7K
Message: Process 825488 (zebra) of user 111 dumped core.

            Stack trace of thread 825490:
            #0  0x0000555c132e566c netlink_route_multipath_msg_encode (zebra + 0xa266c)
            #1  0x0000555c132e5f63 n/a (zebra + 0xa2f63)
            #2  0x0000555c132d8fc1 netlink_batch_add_msg (zebra + 0x95fc1)
            #3  0x0000555c132d9318 kernel_update_multi (zebra + 0x96318)
            #4  0x0000555c132fde18 n/a (zebra + 0xbae18)
            #5  0x0000555c132f7887 n/a (zebra + 0xb4887)
            #6  0x00007f5711f0e901 event_call (libfrr.so.0 + 0x10e901)
            #7  0x00007f5711eabe38 n/a (libfrr.so.0 + 0xabe38)
            #8  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #9  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825488:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eba49b frr_run (libfrr.so.0 + 0xba49b)
            #3  0x0000555c132c7576 main (zebra + 0x84576)
            #4  0x00007f5711b0c24a __libc_start_call_main (libc.so.6 + 0x2724a)
            #5  0x00007f5711b0c305 __libc_start_main_impl (libc.so.6 + 0x27305)
            #6  0x0000555c132c8521 _start (zebra + 0x85521)
            
            Stack trace of thread 825512:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825504:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825505:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825489:
            #0  0x00007f5711be6719 syscall (libc.so.6 + 0x101719)
            #1  0x00007f5711ef96e7 seqlock_wait (libfrr.so.0 + 0xf96e7)
            #2  0x00007f5711eab167 n/a (libfrr.so.0 + 0xab167)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825506:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825491:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825507:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825515:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            
            Stack trace of thread 825509:
            #0  0x00007f5711be1256 __ppoll (libc.so.6 + 0xfc256)
            #1  0x00007f5711f0de8e event_fetch (libfrr.so.0 + 0x10de8e)
            #2  0x00007f5711eabde1 n/a (libfrr.so.0 + 0xabde1)
            #3  0x00007f5711b6e134 start_thread (libc.so.6 + 0x89134)
            #4  0x00007f5711bee7dc __clone3 (libc.so.6 + 0x1097dc)
            ELF object binary architecture: AMD x86-64

Refusing to dump core to tty (use shell redirection or specify --output).


### Additional context

I think maybe this error is part of #15748

### Checklist

- [X] I have searched the open issues for this bug.
- [X] I have not included sensitive information in this report.
@DS819src DS819src added the triage Needs further investigation label Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage Needs further investigation
Projects
None yet
Development

No branches or pull requests

1 participant