syzbot

 sign-in | mailing list | source | docs
🐞 Open [749]
🐞 Fixed [50]
🐞 Invalid [462]
⬇ Missing Backports [60]
Kernel Health Bugs/Month Bug Lifetimes Fuzzing Crashes
💬 Send us feedback

INFO: rcu detected stall in hsr_announce

Status: upstream: reported on 2024/08/15 19:00
Reported-by: syzbot+10d5aeaed47a1779bb0d@syzkaller.appspotmail.com
First crash: 43d, last: 14h57m
▶ ▼ Similar bugs (11)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in hsr_announce 2 6d09h 25d 0/3 upstream: reported on 2024/09/03 13:47
upstream INFO: rcu detected stall in hsr_announce (3) net 1 1760d 1760d 0/28 closed as invalid on 2019/12/04 14:04
upstream INFO: rcu detected stall in hsr_announce (5) net 4 1048d 1060d 0/28 closed as invalid on 2022/02/08 10:00
upstream INFO: rcu detected stall in hsr_announce (6) kvm 2 886d 886d 0/28 auto-closed as invalid on 2022/06/25 07:59
linux-4.14 INFO: rcu detected stall in hsr_announce C error 25 1548d 1746d 0/1 upstream: reported C repro on 2019/12/17 23:42
upstream INFO: rcu detected stall in hsr_announce (4) net 1 1386d 1386d 0/28 auto-closed as invalid on 2021/03/11 15:56
upstream INFO: rcu detected stall in hsr_announce net 1 1840d 1840d 0/28 auto-closed as invalid on 2019/11/13 18:24
upstream INFO: rcu detected stall in hsr_announce (2) net 3 1765d 1765d 0/28 closed as invalid on 2019/11/29 14:24
linux-4.19 INFO: rcu detected stall in hsr_announce 1 1460d 1460d 0/1 auto-closed as invalid on 2021/01/27 07:34
linux-4.19 BUG: soft lockup in hsr_announce (2) 7 711d 1165d 0/1 auto-obsoleted due to no activity on 2023/02/15 11:56
upstream BUG: soft lockup in hsr_announce (2) net 3 366d 375d 0/28 auto-obsoleted due to no activity on 2023/12/27 12:08

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (10500 ticks this GP) idle=b09/1/0x4000000000000000 softirq=69702/69702 fqs=0 
	(t=10500 jiffies g=111349 q=1540)
rcu: rcu_preempt kthread starved for 10500 jiffies! g111349 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:26144 pid:   15 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5027 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6373
 schedule+0x11b/0x1f0 kernel/sched/core.c:6456
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1914
 rcu_gp_fqs_loop+0x2bf/0x1080 kernel/rcu/tree.c:1972
 rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2145
 kthread+0x3f6/0x4f0 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 14327 Comm: syz.2.2970 Not tainted 5.15.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
 rcu_check_gp_kthread_starvation+0x1d2/0x240 kernel/rcu/tree_stall.h:487
 print_cpu_stall+0x31b/0x600 kernel/rcu/tree_stall.h:631
 check_cpu_stall kernel/rcu/tree_stall.h:727 [inline]
 rcu_pending kernel/rcu/tree.c:3932 [inline]
 rcu_sched_clock_irq+0x8d9/0x1150 kernel/rcu/tree.c:2619
 update_process_times+0x196/0x200 kernel/time/timer.c:1818
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x3e/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:list_empty include/linux/list.h:290 [inline]
RIP: 0010:dev_nit_active net/core/dev.c:2275 [inline]
RIP: 0010:xmit_one net/core/dev.c:3611 [inline]
RIP: 0010:dev_hard_start_xmit+0x11a/0x7a0 net/core/dev.c:3633
Code: a8 9e 05 48 c7 c1 30 c9 e9 8d 48 39 c8 74 07 e8 6c 39 35 f9 eb 26 48 8b 44 24 40 42 80 3c 38 00 74 08 48 89 ef e8 76 03 7f f9 <48> 8b 45 00 48 39 e8 0f 84 03 02 00 00 e8 44 39 35 f9 4c 89 f7 48
RSP: 0018:ffffc90000dd0658 EFLAGS: 00000246
RAX: 1ffff1100a306410 RBX: ffff888051832000 RCX: ffffffff8de9c930
RDX: 0000000000000100 RSI: ffff888051832000 RDI: ffff888019d94140
RBP: ffff888051832080 R08: ffffffff884b6e2c R09: ffffffff884b1b8e
R10: 0000000000000002 R11: ffff88803ea61dc0 R12: 1ffff110033b2828
R13: ffff888028d0a800 R14: ffff888019d94140 R15: dffffc0000000000
 __dev_queue_xmit+0x1cee/0x3230 net/core/dev.c:4248
 hsr_xmit net/hsr/hsr_forward.c:338 [inline]
 hsr_forward_do net/hsr/hsr_forward.c:429 [inline]
 hsr_forward_skb+0x133c/0x1b50 net/hsr/hsr_forward.c:577
 send_hsr_supervision_frame+0x540/0xad0 net/hsr/hsr_device.c:326
 hsr_announce+0x176/0x300 net/hsr/hsr_device.c:382
 call_timer_fn+0x16d/0x560 kernel/time/timer.c:1451
 expire_timers kernel/time/timer.c:1496 [inline]
 __run_timers+0x67c/0x890 kernel/time/timer.c:1767
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1780
 handle_softirqs+0x3a7/0x930 kernel/softirq.c:558
 __do_softirq kernel/softirq.c:592 [inline]
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x157/0x240 kernel/softirq.c:641
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:653
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1096
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:trace_lock_acquire include/trace/events/lock.h:13 [inline]
RIP: 0010:lock_acquire+0xd6/0x4f0 kernel/locking/lockdep.c:5594
Code: 83 fb 08 0f 83 e2 02 00 00 89 d8 c1 e8 06 48 8d 3c c5 a8 60 e9 8d be 08 00 00 00 e8 04 78 67 00 89 d8 48 0f a3 05 82 b2 86 0c <73> 0d e8 03 6b 08 00 84 c0 0f 84 b9 02 00 00 48 c7 c0 44 94 e9 8d
RSP: 0018:ffffc9000549f5a0 EFLAGS: 00000257
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffffffff8162ae1c
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8de960a8
RBP: ffffc9000549f700 R08: dffffc0000000000 R09: fffffbfff1bd2c16
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000a93ebc
R13: dffffc0000000000 R14: 0000000000000000 R15: 1ffff92000a93ef4
 rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:312
 rcu_read_lock include/linux/rcupdate.h:739 [inline]
 BPF_PROG_RUN_ARRAY include/linux/bpf.h:1333 [inline]
 trace_call_bpf+0x146/0x660 kernel/trace/bpf_trace.c:127
 perf_trace_run_bpf_submit+0x7b/0x1d0 kernel/events/core.c:9987
 perf_trace_lock+0x37f/0x440 include/trace/events/lock.h:39
 trace_lock_release include/trace/events/lock.h:58 [inline]
 lock_release+0x93c/0x9a0 kernel/locking/lockdep.c:5634
 might_alloc include/linux/sched/mm.h:206 [inline]
 slab_pre_alloc_hook+0x22/0xc0 mm/slab.h:492
 slab_alloc_node mm/slub.c:3134 [inline]
 slab_alloc mm/slub.c:3228 [inline]
 kmem_cache_alloc_trace+0x49/0x290 mm/slub.c:3245
 kmalloc include/linux/slab.h:591 [inline]
 kzalloc include/linux/slab.h:721 [inline]
 apparmor_sk_alloc_security+0x73/0x100 security/apparmor/lsm.c:785
 security_sk_alloc+0x6d/0xa0 security/security.c:2308
 sk_prot_alloc+0xfa/0x200 net/core/sock.c:1866
 sk_alloc+0x35/0x310 net/core/sock.c:1916
 tipc_sk_create+0x107/0x1c50 net/tipc/socket.c:488
 __sock_create+0x460/0x8d0 net/socket.c:1486
 sock_create net/socket.c:1537 [inline]
 __sys_socketpair+0x2c1/0x700 net/socket.c:1637
 __do_sys_socketpair net/socket.c:1690 [inline]
 __se_sys_socketpair net/socket.c:1687 [inline]
 __x64_sys_socketpair+0x97/0xb0 net/socket.c:1687
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7fd53e501ff9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fd53c97a038 EFLAGS: 00000246 ORIG_RAX: 0000000000000035
RAX: ffffffffffffffda RBX: 00007fd53e6b9f80 RCX: 00007fd53e501ff9
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 000000000000001e
RBP: 00007fd53e574296 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000020000040 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007fd53e6b9f80 R15: 00007ffed93fff88
 </TASK>
NMI backtrace for cpu 1
CPU: 1 PID: 14327 Comm: syz.2.2970 Not tainted 5.15.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
 rcu_dump_cpu_stacks+0x223/0x390 kernel/rcu/tree_stall.h:349
 print_cpu_stall+0x320/0x600 kernel/rcu/tree_stall.h:633
 check_cpu_stall kernel/rcu/tree_stall.h:727 [inline]
 rcu_pending kernel/rcu/tree.c:3932 [inline]
 rcu_sched_clock_irq+0x8d9/0x1150 kernel/rcu/tree.c:2619
 update_process_times+0x196/0x200 kernel/time/timer.c:1818
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x3e/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:list_empty include/linux/list.h:290 [inline]
RIP: 0010:dev_nit_active net/core/dev.c:2275 [inline]
RIP: 0010:xmit_one net/core/dev.c:3611 [inline]
RIP: 0010:dev_hard_start_xmit+0x11a/0x7a0 net/core/dev.c:3633
Code: a8 9e 05 48 c7 c1 30 c9 e9 8d 48 39 c8 74 07 e8 6c 39 35 f9 eb 26 48 8b 44 24 40 42 80 3c 38 00 74 08 48 89 ef e8 76 03 7f f9 <48> 8b 45 00 48 39 e8 0f 84 03 02 00 00 e8 44 39 35 f9 4c 89 f7 48
RSP: 0018:ffffc90000dd0658 EFLAGS: 00000246
RAX: 1ffff1100a306410 RBX: ffff888051832000 RCX: ffffffff8de9c930
RDX: 0000000000000100 RSI: ffff888051832000 RDI: ffff888019d94140
RBP: ffff888051832080 R08: ffffffff884b6e2c R09: ffffffff884b1b8e
R10: 0000000000000002 R11: ffff88803ea61dc0 R12: 1ffff110033b2828
R13: ffff888028d0a800 R14: ffff888019d94140 R15: dffffc0000000000
 __dev_queue_xmit+0x1cee/0x3230 net/core/dev.c:4248
 hsr_xmit net/hsr/hsr_forward.c:338 [inline]
 hsr_forward_do net/hsr/hsr_forward.c:429 [inline]
 hsr_forward_skb+0x133c/0x1b50 net/hsr/hsr_forward.c:577
 send_hsr_supervision_frame+0x540/0xad0 net/hsr/hsr_device.c:326
 hsr_announce+0x176/0x300 net/hsr/hsr_device.c:382
 call_timer_fn+0x16d/0x560 kernel/time/timer.c:1451
 expire_timers kernel/time/timer.c:1496 [inline]
 __run_timers+0x67c/0x890 kernel/time/timer.c:1767
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1780
 handle_softirqs+0x3a7/0x930 kernel/softirq.c:558
 __do_softirq kernel/softirq.c:592 [inline]
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x157/0x240 kernel/softirq.c:641
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:653
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1096
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:trace_lock_acquire include/trace/events/lock.h:13 [inline]
RIP: 0010:lock_acquire+0xd6/0x4f0 kernel/locking/lockdep.c:5594
Code: 83 fb 08 0f 83 e2 02 00 00 89 d8 c1 e8 06 48 8d 3c c5 a8 60 e9 8d be 08 00 00 00 e8 04 78 67 00 89 d8 48 0f a3 05 82 b2 86 0c <73> 0d e8 03 6b 08 00 84 c0 0f 84 b9 02 00 00 48 c7 c0 44 94 e9 8d
RSP: 0018:ffffc9000549f5a0 EFLAGS: 00000257
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffffffff8162ae1c
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8de960a8
RBP: ffffc9000549f700 R08: dffffc0000000000 R09: fffffbfff1bd2c16
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000a93ebc
R13: dffffc0000000000 R14: 0000000000000000 R15: 1ffff92000a93ef4
 rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:312
 rcu_read_lock include/linux/rcupdate.h:739 [inline]
 BPF_PROG_RUN_ARRAY include/linux/bpf.h:1333 [inline]
 trace_call_bpf+0x146/0x660 kernel/trace/bpf_trace.c:127
 perf_trace_run_bpf_submit+0x7b/0x1d0 kernel/events/core.c:9987
 perf_trace_lock+0x37f/0x440 include/trace/events/lock.h:39
 trace_lock_release include/trace/events/lock.h:58 [inline]
 lock_release+0x93c/0x9a0 kernel/locking/lockdep.c:5634
 might_alloc include/linux/sched/mm.h:206 [inline]
 slab_pre_alloc_hook+0x22/0xc0 mm/slab.h:492
 slab_alloc_node mm/slub.c:3134 [inline]
 slab_alloc mm/slub.c:3228 [inline]
 kmem_cache_alloc_trace+0x49/0x290 mm/slub.c:3245
 kmalloc include/linux/slab.h:591 [inline]
 kzalloc include/linux/slab.h:721 [inline]
 apparmor_sk_alloc_security+0x73/0x100 security/apparmor/lsm.c:785
 security_sk_alloc+0x6d/0xa0 security/security.c:2308
 sk_prot_alloc+0xfa/0x200 net/core/sock.c:1866
 sk_alloc+0x35/0x310 net/core/sock.c:1916
 tipc_sk_create+0x107/0x1c50 net/tipc/socket.c:488
 __sock_create+0x460/0x8d0 net/socket.c:1486
 sock_create net/socket.c:1537 [inline]
 __sys_socketpair+0x2c1/0x700 net/socket.c:1637
 __do_sys_socketpair net/socket.c:1690 [inline]
 __se_sys_socketpair net/socket.c:1687 [inline]
 __x64_sys_socketpair+0x97/0xb0 net/socket.c:1687
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7fd53e501ff9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fd53c97a038 EFLAGS: 00000246 ORIG_RAX: 0000000000000035
RAX: ffffffffffffffda RBX: 00007fd53e6b9f80 RCX: 00007fd53e501ff9
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 000000000000001e
RBP: 00007fd53e574296 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000020000040 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007fd53e6b9f80 R15: 00007ffed93fff88
 </TASK>

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/27 23:18 linux-5.15.y 3a5928702e71 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in hsr_announce
2024/09/02 05:45 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in hsr_announce
2024/08/30 09:34 linux-5.15.y fa93fa65db6e ee2602b8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in hsr_announce
2024/08/22 05:57 linux-5.15.y fa93fa65db6e ca02180f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in hsr_announce
2024/08/15 18:59 linux-5.15.y 7e89efd3ae1c e4bacdaf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in hsr_announce
* Struck through repros no longer work on HEAD.