Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel updated, pkt-gen cause kernel panic #937

Open
ke-ding opened this issue Aug 24, 2023 · 4 comments
Open

kernel updated, pkt-gen cause kernel panic #937

ke-ding opened this issue Aug 24, 2023 · 4 comments

Comments

@ke-ding
Copy link

ke-ding commented Aug 24, 2023

archlinux, kernel version updated to 6.4.11-arch2-1. rebuild netmap, pkt-gen cause kernel panic

[ 344.065162] detected buffer overflow in strcpy
[ 344.065172] ------------[ cut here ]------------
[ 344.065173] kernel BUG at lib/string_helpers.c:1027!
[ 344.065177] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 344.065180] CPU: 4 PID: 1451 Comm: pkt-gen Tainted: G OE 6.4.11-arch2-1 #1 97b2f722c7732577cb713428e1f14bfdbe1faa91
[ 344.065183] Hardware name: Intel(R) Client Systems NUC10i7FNK/NUC10i7FNB, BIOS FNCML357.0038.2020.0131.1422 01/31/2020
[ 344.065184] RIP: 0010:fortify_panic+0x13/0x20
[ 344.065189] Code: 41 5d c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 48 89 fe 48 c7 c7 b8 f7 26 ba e8 2d 50 b2 ff <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90
[ 344.065191] RSP: 0018:ffffade0029ff958 EFLAGS: 00010246
[ 344.065194] RAX: 0000000000000022 RBX: 0000000000000000 RCX: 0000000000000027
[ 344.065195] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8b2d20d216c0
[ 344.065196] RBP: ffffade0029ffaf0 R08: 0000000000000000 R09: ffffade0029ff800
[ 344.065197] R10: 0000000000000003 R11: ffffffffbaaca868 R12: ffffade0029ff99c
[ 344.065199] R13: ffff8b2584030000 R14: 00000000ffffffff R15: 0000000000000000
[ 344.065200] FS: 00007f16286dc740(0000) GS:ffff8b2d20d00000(0000) knlGS:0000000000000000
[ 344.065202] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 344.065203] CR2: 00007f1628531c70 CR3: 0000000141ad8003 CR4: 00000000003706e0
[ 344.065204] Call Trace:
[ 344.065206]
[ 344.065208] ? die+0x36/0x90
[ 344.065212] ? do_trap+0xda/0x100
[ 344.065213] ? fortify_panic+0x13/0x20
[ 344.065216] ? do_error_trap+0x6a/0x90
[ 344.065218] ? fortify_panic+0x13/0x20
[ 344.065220] ? exc_invalid_op+0x50/0x70
[ 344.065223] ? fortify_panic+0x13/0x20
[ 344.065226] ? asm_exc_invalid_op+0x1a/0x20
[ 344.065231] ? fortify_panic+0x13/0x20
[ 344.065233] ? fortify_panic+0x13/0x20
[ 344.065237] tc_configure+0x555/0x560 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065262] ? nm_os_catch_tx+0x21d/0x270 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065279] nm_os_catch_tx+0x21d/0x270 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065296] ? nm_os_catch_rx+0x54/0x120 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065312] generic_netmap_register+0x35a/0x710 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065331] netmap_do_regif+0x4ac/0x600 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065350] netmap_ioctl+0x1063/0x16d0 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065369] ? path_openat+0xdbc/0x1170
[ 344.065372] linux_netmap_ioctl+0xc3/0x170 [netmap 7f07a9bf7ff239ee81112568460636ed08deeb05]
[ 344.065391] __x64_sys_ioctl+0x91/0xd0
[ 344.065394] do_syscall_64+0x5d/0x90
[ 344.065396] ? syscall_exit_to_user_mode+0x1b/0x40
[ 344.065399] ? do_syscall_64+0x6c/0x90
[ 344.065401] ? ksys_write+0x6f/0xf0
[ 344.065403] ? syscall_exit_to_user_mode+0x1b/0x40
[ 344.065405] ? do_syscall_64+0x6c/0x90
[ 344.065407] ? exc_page_fault+0x7f/0x180
[ 344.065409] entry_SYSCALL_64_after_hwframe+0x77/0xe1
[ 344.065412] RIP: 0033:0x7f162850c9df
[ 344.065436] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 18 48 8b 44 24 18 64 48 2b 04 25 28 00 00
[ 344.065438] RSP: 002b:00007ffc6198fc10 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[ 344.065440] RAX: ffffffffffffffda RBX: 0000559f436392c0 RCX: 00007f162850c9df
[ 344.065441] RDX: 0000559f436392c0 RSI: 00000000c0586997 RDI: 0000000000000003
[ 344.065442] RBP: 0000000000000000 R08: 00007f16285873c0 R09: 0000559f436392c0
[ 344.065443] R10: 00007f1628416300 R11: 0000000000000246 R12: 0000559f43095c40
[ 344.065444] R13: 00007ffc61990047 R14: 0000559f43095280 R15: 0000559f436392c0
[ 344.065446]
[ 344.065447] Modules linked in: netmap(OE) xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink iptable_nat xt_addrtype iptable_filter br_netfilter bridge stp llc rpcrdma rdma_cm iw_cm ib_cm ib_core ccm rfcomm cmac algif_hash algif_skcipher af_alg snd_hda_codec_hdmi snd_ctl_led snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio overlay tps6598x typec roles snd_sof_pci_intel_cnl snd_sof_intel_hda_common soundwire_intel soundwire_cadence snd_sof_intel_hda_mlink snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils soundwire_generic_allocation soundwire_bus snd_soc_skl snd_soc_hdac_hda snd_hda_ext_core snd_soc_sst_ipc openvswitch snd_soc_sst_dsp intel_rapl_msr bnep nsh intel_rapl_common nf_conncount snd_soc_acpi_intel_match x86_pkg_temp_thermal nf_nat snd_soc_acpi intel_powerclamp coretemp nf_conntrack snd_soc_core kvm_intel nf_defrag_ipv6 snd_compress ac97_bus nf_defrag_ipv4 snd_pcm_dmaengine libcrc32c kvm snd_hda_intel mousedev irqbypass joydev iwlmvm btusb snd_intel_dspcfg crct10dif_pclmul
[ 344.065485] crc32_pclmul snd_intel_sdw_acpi i915 btrtl polyval_clmulni snd_hda_codec mac80211 polyval_generic gf128mul btbcm ghash_clmulni_intel btintel sha512_ssse3 btmtk drm_buddy libarc4 snd_hda_core aesni_intel mei_hdcp mei_pxp bluetooth crypto_simd iTCO_wdt i2c_algo_bit snd_hwdep cryptd iwlwifi ttm spi_nor intel_pmc_bxt rapl snd_pcm drm_display_helper ee1004 iTCO_vendor_support usbhid ecdh_generic intel_wmi_thunderbolt wmi_bmof intel_cstate mtd mei_me cfg80211 snd_timer cec i2c_i801 intel_uncore pcspkr intel_lpss_pci snd mei e1000e i2c_smbus intel_gtt intel_lpss intel_pch_thermal rfkill soundcore idma64 serial_multi_instantiate video wmi acpi_pad acpi_tad mac_hid nfsd auth_rpcgss nfs_acl lockd grace crypto_user sunrpc dm_mod fuse loop bpf_preload ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 sdhci_pci cqhci nvme sdhci nvme_core spi_intel_pci xhci_pci crc32c_intel mmc_core spi_intel nvme_common xhci_pci_renesas
[ 344.065531] ---[ end trace 0000000000000000 ]---

any suggestion?

@vmaffione
Copy link
Collaborator

Can you please share the pkt-gen command you used?

@ke-ding
Copy link
Author

ke-ding commented Aug 27, 2023

Can you please share the pkt-gen command you used?

i just use the demo parameters:
pkt-gen -i eth0 -f tx -l 60

actually, when this command is entered, it halted after print
337.846397 main [3036] interface is eth0
337.846456 main [3159] using default burst size: 512
337.846516 main [3167] running on 1 cpus (have 16)
337.846854 extract_ip_range [477] range is 10.0.0.1:1234 to 10.0.0.1:1234
337.846865 extract_ip_range [477] range is 10.1.0.1:1234 to 10.1.0.1:1234

system is still responsing... i got the kernel dump via dmesg command.

@omerdemirelA
Copy link

Do we have an estimated time frame for when the issue with the detected buffer overflow in strcpy will be addressed?

@brian90013
Copy link
Contributor

I am seeing the same fortify panic for a strcpy() overflow on Ubuntu 22.04.3 kernel version 6.5.0-14-generic using netmap 5bb6334. I have other systems running kernel version 5.15.0-91-generic and using netmap without issue. In both cases I am using the generic/emulated adapter driver. I believe the issue must be the only strcpy() call in tc_configure():

strcpy(((void *)attr_kind) + NLA_HDRLEN, qdisc_name);

I configured an interface for single-queue with ethtool -L enp33s0f0np0 combined 1 then ran pkt-gen -f tx -i enp33s0f0np0 and saw the buffer overflow. tc_configure() was passed qdisc_name=netmapemu.

Then I configured the interface for 2 queues ethtool -L enp33s0f0np0 combined 2, ran pkt-gen again and saw the same overflow. This time tc_configure() was passed qdisc_name=mq.

While looking for strcpy() in netmap_linux.c I found one other use in netmap_sink_init() I don't understand.

    strlcpy(netdev->name, "nmsink", sizeof(netdev->name)); 
    netdev->features = NETIF_F_HIGHDMA;
    strcpy(netdev->name, "nmsink%d");

Why do the strlcpy() followed by a strcpy()? Is %d desired to be part of the name or was it supposed to be used with snprintf()? I don't think it's involved in this overflow but it jumped out to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants