syzbot


INFO: rcu detected stall in br_handle_frame

Status: upstream: reported C repro on 2024/02/08 13:52
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+e89f73a86247175d216a@syzkaller.appspotmail.com
First crash: 82d, last: 82d
Bug presence (2)
Date Name Commit Repro Result
2024/02/08 linux-5.15.y (ToT) 6139f2a02fe0 C [report] INFO: rcu detected stall in sys_clone
2024/02/08 upstream (ToT) 047371968ffc C Didn't crash
Similar bugs (9)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 INFO: rcu detected stall in br_handle_frame (3) 1 1309d 1309d 0/1 auto-closed as invalid on 2021/01/28 07:46
upstream INFO: rcu detected stall in br_handle_frame C done 341 1689d 1694d 13/26 fixed on 2019/10/09 10:54
upstream INFO: rcu detected stall in br_handle_frame (2) net C done 2 1593d 1589d 15/26 fixed on 2020/02/18 14:31
upstream INFO: rcu detected stall in br_handle_frame (3) bridge 1 1018d 1018d 0/26 auto-closed as invalid on 2021/10/15 13:41
linux-4.14 INFO: rcu detected stall in br_handle_frame (2) C done 1 1594d 1594d 1/1 fixed on 2020/01/19 15:05
linux-4.14 INFO: rcu detected stall in br_handle_frame C done 15 1686d 1698d 1/1 fixed on 2019/12/07 19:24
linux-4.19 INFO: rcu detected stall in br_handle_frame (2) C error 31 453d 1294d 0/1 upstream: reported C repro on 2020/10/14 18:56
linux-4.19 INFO: rcu detected stall in br_handle_frame C done 41 1686d 1698d 1/1 fixed on 2019/12/07 19:18
upstream INFO: rcu detected stall in br_handle_frame (4) kernel 1 857d 857d 0/26 closed as invalid on 2022/02/08 10:10
Fix bisection attempts (3)
Created Duration User Patch Repo Result
2024/04/22 07:33 18m fix candidate upstream error job log (0)
2024/03/16 09:58 17m fix candidate upstream error job log (0)
2024/02/14 20:51 17m fix candidate upstream error job log (0)

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	0-....: (10369 ticks this GP) idle=fb1/1/0x4000000000000000 softirq=11014/11018 fqs=4370 
	(t=10500 jiffies g=15645 q=227)
NMI backtrace for cpu 0
CPU: 0 PID: 3629 Comm: syz-executor404 Not tainted 5.15.148-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
 rcu_dump_cpu_stacks+0x223/0x390 kernel/rcu/tree_stall.h:349
 print_cpu_stall+0x320/0x600 kernel/rcu/tree_stall.h:633
 check_cpu_stall kernel/rcu/tree_stall.h:727 [inline]
 rcu_pending kernel/rcu/tree.c:3932 [inline]
 rcu_sched_clock_irq+0x8d9/0x1150 kernel/rcu/tree.c:2619
 update_process_times+0x196/0x200 kernel/time/timer.c:1788
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x3e/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:orc_find arch/x86/kernel/unwind_orc.c:169 [inline]
RIP: 0010:unwind_next_frame+0x1fe/0x1fa0 arch/x86/kernel/unwind_orc.c:448
Code: 46 ff 39 e8 0f 86 a7 15 00 00 89 e8 48 8d 1c 85 dc 21 ec 8e 48 89 d8 48 c1 e8 03 0f b6 04 10 84 c0 0f 85 3d 17 00 00 44 8b 23 <89> e8 ff c0 48 8d 1c 85 dc 21 ec 8e 48 89 d8 48 c1 e8 03 0f b6 04
RSP: 0018:ffffc90000006328 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff8eed5754 RCX: 0000000000096084
RDX: dffffc0000000000 RSI: ffffffff814d5e05 RDI: 0000000000000001
RBP: 0000000000004d5e R08: 000000000000001d R09: ffffc900000064d0
R10: 0000000000000000 R11: dffffc0000000001 R12: 000000000000e806
R13: ffffffff814d5e04 R14: 0000000000096084 R15: ffffc900000063e0
 arch_stack_walk+0x10d/0x140 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x113/0x1c0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:38 [inline]
 kasan_set_track+0x4b/0x80 mm/kasan/common.c:46
 kasan_set_free_info+0x1f/0x40 mm/kasan/generic.c:360
 ____kasan_slab_free+0xd8/0x120 mm/kasan/common.c:366
 kasan_slab_free include/linux/kasan.h:230 [inline]
 slab_free_hook mm/slub.c:1705 [inline]
 slab_free_freelist_hook+0xdd/0x160 mm/slub.c:1731
 slab_free mm/slub.c:3499 [inline]
 kmem_cache_free+0x91/0x1f0 mm/slub.c:3515
 skb_ext_del include/linux/skbuff.h:4420 [inline]
 nf_bridge_info_free net/bridge/br_netfilter_hooks.c:147 [inline]
 br_nf_dev_queue_xmit+0x4f0/0x1aa0 net/bridge/br_netfilter_hooks.c:752
 NF_HOOK+0x697/0x760 include/linux/netfilter.h:302
 br_nf_post_routing+0x8f1/0xc80 net/bridge/br_netfilter_hooks.c:855
 nf_hook_entry_hookfn include/linux/netfilter.h:142 [inline]
 nf_hook_slow+0xae/0x1e0 net/netfilter/core.c:584
 nf_hook include/linux/netfilter.h:257 [inline]
 NF_HOOK+0x27c/0x420 include/linux/netfilter.h:300
 br_forward_finish+0x74/0x80 net/bridge/br_forward.c:66
 br_nf_forward_finish+0xa5b/0xe20 net/bridge/br_netfilter_hooks.c:566
 NF_HOOK+0x697/0x760 include/linux/netfilter.h:302
 br_nf_forward_ip+0xbe2/0x1010 net/bridge/br_netfilter_hooks.c:636
 nf_hook_entry_hookfn include/linux/netfilter.h:142 [inline]
 nf_hook_slow+0xae/0x1e0 net/netfilter/core.c:584
 nf_hook include/linux/netfilter.h:257 [inline]
 NF_HOOK+0x27c/0x420 include/linux/netfilter.h:300
 __br_forward+0x430/0x5f0 net/bridge/br_forward.c:115
 br_handle_frame_finish+0x12ea/0x1710 net/bridge/br_input.c:175
 br_nf_hook_thresh+0x46a/0x580 net/bridge/br_netfilter_hooks.c:1047
 br_nf_pre_routing_finish_ipv6+0xa08/0xcd0
 NF_HOOK include/linux/netfilter.h:302 [inline]
 br_nf_pre_routing_ipv6+0x35d/0x720 net/bridge/br_netfilter_ipv6.c:237
 nf_hook_entry_hookfn include/linux/netfilter.h:142 [inline]
 nf_hook_bridge_pre net/bridge/br_input.c:237 [inline]
 br_handle_frame+0x8a8/0x10b0 net/bridge/br_input.c:377
 __netif_receive_skb_core+0x13b9/0x3df0 net/core/dev.c:5379
 __netif_receive_skb_one_core net/core/dev.c:5483 [inline]
 __netif_receive_skb+0x11c/0x530 net/core/dev.c:5599
 process_backlog+0x363/0x7f0 net/core/dev.c:6476
 __napi_poll+0xc7/0x440 net/core/dev.c:7035
 napi_poll net/core/dev.c:7102 [inline]
 net_rx_action+0x617/0xda0 net/core/dev.c:7189
 __do_softirq+0x3b3/0x93a kernel/softirq.c:558
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:637
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1096
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:finish_lock_switch+0x91/0x100 kernel/sched/core.c:4785
Code: 45 31 c9 68 c7 65 59 81 e8 ac 1a 09 00 48 83 c4 08 4c 89 ff e8 60 da fe ff 66 90 4c 89 ff e8 d6 a1 cc 08 e8 f1 44 2d 00 fb 5b <41> 5c 41 5d 41 5e 41 5f c3 44 89 f1 80 e1 07 80 c1 03 38 c1 7c 87
RSP: 0018:ffffc90002fe7358 EFLAGS: 00000282
RAX: 60bb9165b5e9d500 RBX: ffff888018adbbb4 RCX: ffffffff8162ee28
RDX: dffffc0000000000 RSI: ffffffff8a8b0f80 RDI: ffffffff8ad87e40
RBP: ffffc90002fe73d0 R08: dffffc0000000000 R09: fffffbfff1f7a01a
R10: 0000000000000000 R11: dffffc0000000001 R12: dffffc0000000000
R13: 1ffff1101734748b R14: ffff8880b9a3a458 R15: ffff8880b9a39700
 finish_task_switch+0x134/0x630 kernel/sched/core.c:4902
 context_switch kernel/sched/core.c:5033 [inline]
 __schedule+0x12cc/0x45b0 kernel/sched/core.c:6376
 preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6780
 irqentry_exit+0x53/0x80 kernel/entry/common.c:426
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:inet_twsk_purge+0x97e/0xa20 net/ipv4/inet_timewait_sock.c:264
Code: 13 05 01 48 c7 c7 00 2e 83 8b be d6 02 00 00 48 c7 c2 a0 2e 83 8b e8 21 23 a6 f8 e8 6c e3 ae f8 e8 17 01 00 00 48 8b 74 24 58 <ff> c6 48 8b 54 24 40 48 89 d0 48 c1 e8 03 42 0f b6 04 38 84 c0 75
RSP: 0018:ffffc90002fe7740 EFLAGS: 00000246
RAX: 60bb9165b5e9d500 RBX: 0000000000000001 RCX: ffffc90002fe7603
RDX: 0000000000000001 RSI: 0000000000075099 RDI: ffffffff8ad87e40
RBP: ffffc90002fe7850 R08: dffffc0000000000 R09: fffffbfff1bc73d6
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000075099
R13: 1ffffffff1b77484 R14: ffffffff917bd8a0 R15: dffffc0000000000
 ops_exit_list net/core/net_namespace.c:174 [inline]
 setup_net+0x81c/0x9e0 net/core/net_namespace.c:352
 copy_net_ns+0x395/0x5d0 net/core/net_namespace.c:473
 create_new_namespaces+0x425/0x7a0 kernel/nsproxy.c:110
 copy_namespaces+0x365/0x3b0 kernel/nsproxy.c:178
 copy_process+0x1841/0x3ef0 kernel/fork.c:2209
 kernel_clone+0x210/0x960 kernel/fork.c:2604
 __do_sys_clone kernel/fork.c:2721 [inline]
 __se_sys_clone kernel/fork.c:2705 [inline]
 __x64_sys_clone+0x23f/0x290 kernel/fork.c:2705
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7f2a56f5e9a9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 a1 1d 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffebb768748 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f2a56f5e9a9
RDX: 0000000000000000 RSI: 0100000000000000 RDI: 0030626f7c2a0000
RBP: 00000000000f4240 R08: 0000000000000000 R09: 0000000000000003
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f2a56fe0b40
R13: 00007f2a56fe2d00 R14: 00007ffebb76882c R15: 00007ffebb7688a0
 </TASK>

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/02/08 13:52 linux-5.15.y 6139f2a02fe0 6404acf9 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in br_handle_frame
* Struck through repros no longer work on HEAD.