syzbot


INFO: rcu detected stall in br_handle_frame (5)

Status: upstream: reported syz repro on 2024/10/12 07:17
Subsystems: bridge
[Documentation on labels]
Reported-by: syzbot+c596faae21a68bf7afd0@syzkaller.appspotmail.com
Fix commit: netdevsim: use cond_resched() in nsim_dev_trap_report_work()
Patched on: [ci-upstream-net-this-kasan-gce], missing on: [ci-qemu-gce-upstream-auto ci-qemu-native-arm64-kvm ci-qemu-upstream ci-qemu-upstream-386 ci-qemu2-arm32 ci-qemu2-arm64 ci-qemu2-arm64-compat ci-qemu2-arm64-mte ci-qemu2-riscv64 ci-snapshot-upstream-root ci-upstream-bpf-kasan-gce ci-upstream-bpf-next-kasan-gce ci-upstream-gce-arm64 ci-upstream-gce-leak ci-upstream-kasan-badwrites-root ci-upstream-kasan-gce ci-upstream-kasan-gce-386 ci-upstream-kasan-gce-root ci-upstream-kasan-gce-selinux-root ci-upstream-kasan-gce-smack-root ci-upstream-kmsan-gce-386-root ci-upstream-kmsan-gce-root ci-upstream-linux-next-kasan-gce-root ci-upstream-net-kasan-gce ci2-upstream-fs ci2-upstream-kcsan-gce ci2-upstream-usb]
First crash: 144d, last: 15h50m
Discussions (2)
Title Replies (including bot) Last reply
[PATCH v2 net] netdevsim: use cond_resched() in nsim_dev_trap_report_work() 2 (2) 2024/10/15 17:10
[syzbot] [bridge?] INFO: rcu detected stall in br_handle_frame (5) 0 (1) 2024/10/12 07:17
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 INFO: rcu detected stall in br_handle_frame (3) 1 1476d 1476d 0/1 auto-closed as invalid on 2021/01/28 07:46
upstream INFO: rcu detected stall in br_handle_frame C done 341 1856d 1862d 13/28 fixed on 2019/10/09 10:54
upstream INFO: rcu detected stall in br_handle_frame (2) net C done 2 1761d 1757d 15/28 fixed on 2020/02/18 14:31
upstream INFO: rcu detected stall in br_handle_frame (3) bridge 1 1186d 1186d 0/28 auto-closed as invalid on 2021/10/15 13:41
linux-4.14 INFO: rcu detected stall in br_handle_frame (2) C done 1 1761d 1761d 1/1 fixed on 2020/01/19 15:05
linux-4.14 INFO: rcu detected stall in br_handle_frame C done 15 1854d 1865d 1/1 fixed on 2019/12/07 19:24
linux-4.19 INFO: rcu detected stall in br_handle_frame (2) C error 31 621d 1462d 0/1 upstream: reported C repro on 2020/10/14 18:56
linux-4.19 INFO: rcu detected stall in br_handle_frame C done 41 1853d 1866d 1/1 fixed on 2019/12/07 19:18
linux-5.15 INFO: rcu detected stall in br_handle_frame origin:lts-only C error 1 250d 250d 0/3 upstream: reported C repro on 2024/02/08 13:52
linux-6.1 INFO: rcu detected stall in br_handle_frame 2 50d 132d 0/3 upstream: reported on 2024/06/05 18:32
upstream INFO: rcu detected stall in br_handle_frame (4) kernel 1 1025d 1025d 0/28 closed as invalid on 2022/02/08 10:10
android-5-15 BUG: soft lockup in br_handle_frame 2 58d 64d 0/2 premoderation: reported on 2024/08/12 10:16
Cause bisection attempts (1)
Created Duration User Patch Repo Result
2024/10/12 13:06 5h58m bisect net-next error job log

Sample crash report:
bridge0: received packet on veth0_to_bridge with own address as source address (addr:6e:a5:51:5e:bc:50, vlan:0)
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	(detected by 1, t=10502 jiffies, g=8089, q=1074 ncpus=2)
rcu: All QSes seen, last rcu_preempt kthread activity 8424 (4294972116-4294963692), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_preempt kthread starved for 8425 jiffies! g8089 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25952 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5315 [inline]
 __schedule+0x1895/0x4b30 kernel/sched/core.c:6675
 __schedule_loop kernel/sched/core.c:6752 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6767
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2615
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2045
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2247
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 5350 Comm: kworker/0:3 Not tainted 6.12.0-rc1-syzkaller-00242-g1405981bbba0 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: events nsim_dev_trap_report_work
RIP: 0010:task_tick_fair+0x282/0x7c0 kernel/sched/fair.c:13046
Code: 89 e7 e8 01 d3 96 00 4d 39 34 24 75 60 48 81 c3 78 01 00 00 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 de d2 96 00 <48> 8b 3b 83 7c 24 08 00 74 07 e8 9f 6f fb ff eb 0c 48 81 c7 40 0d
RSP: 0018:ffffc900000067c8 EFLAGS: 00000046
RAX: 1ffff110170c7d97 RBX: ffff8880b863ecb8 RCX: 000000000008c5a2
RDX: 000000000008c5a2 RSI: dffffc0000000000 RDI: ffff8880b863eb54
RBP: ffff888071fc5b28 R08: ffffffff901ce76f R09: 1ffffffff2039ced
R10: dffffc0000000000 R11: ffffffff8167b790 R12: ffff888071fc5a80
R13: dffffc0000000000 R14: 0000000000000000 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8e57107ab8 CR3: 0000000077f18000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 sched_tick+0x219/0x610 kernel/sched/core.c:5593
 update_process_times+0x202/0x230 kernel/time/timer.c:2524
 tick_sched_handle kernel/time/tick-sched.c:276 [inline]
 tick_nohz_handler+0x37c/0x500 kernel/time/tick-sched.c:297
 __run_hrtimer kernel/time/hrtimer.c:1691 [inline]
 __hrtimer_run_queues+0x551/0xd50 kernel/time/hrtimer.c:1755
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1817
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1026 [inline]
 __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1043
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1037 [inline]
 sysvec_apic_timer_interrupt+0x52/0xc0 arch/x86/kernel/apic/apic.c:1037
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:trace_kmem_cache_alloc+0x18/0xc0 include/trace/events/kmem.h:12
Code: 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 90 65 8b 05 f3 54 11 7e 83 f8 08 73 23 89 c0 48 0f a3 05 80 64 2a 0e <73> 12 e8 21 41 88 ff 84 c0 75 09 f6 05 0a 22 14 0e 01 74 0b c3 cc
RSP: 0018:ffffc90000006c80 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8881a6f96d00 RCX: 0000000000000820
RDX: ffff888141ae13c0 RSI: ffff8881a6f96d00 RDI: ffffffff898d7336
RBP: 0000000000000000 R08: 00000000ffffffff R09: 0000000000000000
R10: ffff8881a6f96d00 R11: ffffffff81808f50 R12: 00000000000000b8
R13: ffff888141ae13c0 R14: 0000000000000820 R15: ffffffff898d7336
 kmem_cache_alloc_noprof+0x185/0x2a0 mm/slub.c:4145
 skb_ext_maybe_cow net/core/skbuff.c:6942 [inline]
 skb_ext_add+0x1d6/0x910 net/core/skbuff.c:7016
 nf_bridge_unshare net/bridge/br_netfilter_hooks.c:168 [inline]
 br_nf_forward_ip+0xd8/0x7b0 net/bridge/br_netfilter_hooks.c:710
 nf_hook_entry_hookfn include/linux/netfilter.h:154 [inline]
 nf_hook_slow+0xc3/0x220 net/netfilter/core.c:626
 nf_hook include/linux/netfilter.h:269 [inline]
 NF_HOOK+0x2a7/0x460 include/linux/netfilter.h:312
 __br_forward+0x489/0x660 net/bridge/br_forward.c:115
 deliver_clone net/bridge/br_forward.c:131 [inline]
 maybe_deliver+0xb3/0x150 net/bridge/br_forward.c:190
 br_flood+0x2e4/0x660 net/bridge/br_forward.c:236
 br_handle_frame_finish+0x18ba/0x1fe0 net/bridge/br_input.c:215
 br_nf_hook_thresh+0x472/0x590
 br_nf_pre_routing_finish_ipv6+0xaa0/0xdd0
 NF_HOOK include/linux/netfilter.h:314 [inline]
 br_nf_pre_routing_ipv6+0x379/0x770 net/bridge/br_netfilter_ipv6.c:184
 nf_hook_entry_hookfn include/linux/netfilter.h:154 [inline]
 nf_hook_bridge_pre net/bridge/br_input.c:277 [inline]
 br_handle_frame+0x9fd/0x1530 net/bridge/br_input.c:424
 __netif_receive_skb_core+0x13e8/0x4570 net/core/dev.c:5560
 __netif_receive_skb_one_core net/core/dev.c:5664 [inline]
 __netif_receive_skb+0x12f/0x650 net/core/dev.c:5779
 process_backlog+0x662/0x15b0 net/core/dev.c:6111
 __napi_poll+0xcb/0x490 net/core/dev.c:6775
 napi_poll net/core/dev.c:6844 [inline]
 net_rx_action+0x89b/0x1240 net/core/dev.c:6966
 handle_softirqs+0x2c5/0x980 kernel/softirq.c:554
 do_softirq+0x11b/0x1e0 kernel/softirq.c:455
 </IRQ>
 <TASK>
 __local_bh_enable_ip+0x1bb/0x200 kernel/softirq.c:382
 spin_unlock_bh include/linux/spinlock.h:396 [inline]
 nsim_dev_trap_report drivers/net/netdevsim/dev.c:820 [inline]
 nsim_dev_trap_report_work+0x75d/0xaa0 drivers/net/netdevsim/dev.c:850
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
net_ratelimit: 22358 callbacks suppressed
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:1b, vlan:0)
bridge0: received packet on veth0_to_bridge with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on veth0_to_bridge with own address as source address (addr:6e:a5:51:5e:bc:50, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:1b, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:1b, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on veth0_to_bridge with own address as source address (addr:ca:e3:c2:99:e2:22, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)

Crashes (16):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/10/08 07:11 net-next 1405981bbba0 402f1df0 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/15 08:30 upstream eca631b8fe80 b01b6661 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in br_handle_frame
2024/10/12 17:52 upstream 09f6b0c8904b 084d8178 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in br_handle_frame
2024/10/07 01:33 upstream 8cf0b93919e1 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in br_handle_frame
2024/09/28 04:17 upstream ad46e8f95e93 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in br_handle_frame
2024/09/26 15:33 upstream aa486552a110 0d19f247 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/07/12 21:11 upstream 43db1e03c086 eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in br_handle_frame
2024/06/27 07:46 upstream 24ca36a562d6 5c045c04 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in br_handle_frame
2024/10/12 06:32 net 8a6be4bd6fb3 084d8178 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/10 06:48 net 983e35ce2e1e 0278d004 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/13 20:57 net-next eae38f09cc0e 084d8178 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/06 16:01 net-next cf9545686230 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/04 13:05 net-next b63c755cb65d d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/10/03 18:21 net-next 7c2f1c2690a5 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/09/30 09:29 net-next c824deb1a897 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in br_handle_frame
2024/05/24 02:12 linux-next 124cfbcd6d18 8f98448e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in br_handle_frame
* Struck through repros no longer work on HEAD.