syzbot
INFO: rcu detected stall in __run_timers
Status:
upstream: reported on 2026/06/10 07:15
Reported-by: syzbot+e29a1b5151f0452db037@syzkaller.appspotmail.com
First crash: 48d, last: 9d12h
Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 1-....: (1 GPs behind) idle=619c/1/0x4000000000000000 softirq=12919/12922 fqs=57
rcu: hardirqs softirqs csw/system
rcu: number: 221099 0 0
rcu: cputime: 4296 8062 13 ==> 12360(ms)
rcu: (t=10500 jiffies g=13745 q=621 ncpus=2)
rcu: rcu_preempt kthread starved for 1099 jiffies! g13745 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:25832 pid:17 ppid:2 flags:0x00004000
Call Trace:
<TASK>
context_switch kernel/sched/core.c:5381 [inline]
__schedule+0x15ae/0x4660 kernel/sched/core.c:6700
schedule+0xbd/0x170 kernel/sched/core.c:6774
schedule_timeout+0x188/0x2d0 kernel/time/timer.c:2168
rcu_gp_fqs_loop+0x30d/0x1590 kernel/rcu/tree.c:1667
rcu_gp_kthread+0x9d/0x3b0 kernel/rcu/tree.c:1866
kthread+0x2fa/0x390 kernel/kthread.c:388
ret_from_fork+0x48/0x80 arch/x86/kernel/process.c:152
ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:293
</TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 6427 Comm: syz.0.181 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/09/2026
RIP: 0010:check_preemption_disabled+0x18/0x110 lib/smp_processor_id.c:14
Code: 00 fa 1c 8b eb 0c 66 2e 0f 1f 84 00 00 00 00 00 66 90 55 41 57 41 56 53 48 83 ec 10 65 48 8b 04 25 28 00 00 00 48 89 44 24 08 <65> 8b 05 dd da 77 75 65 8b 0d d2 da 77 75 f7 c1 ff ff ff 7f 74 1f
RSP: 0018:ffffc900000078a8 EFLAGS: 00000082
RAX: 703b4dcf9aca2b00 RBX: 0000000000000000 RCX: ffffffff81681b46
RDX: 0000000000000000 RSI: ffffffff8acadce0 RDI: ffffffff8b1cfa20
RBP: ffffc900000079e8 R08: dffffc0000000000 R09: 1ffffffff223cea1
R10: dffffc0000000000 R11: fffffbfff223cea2 R12: 1ffff92000000f28
R13: ffff8880b8e2b758 R14: 0000000000000046 R15: dffffc0000000000
FS: 00007f0694b2a6c0(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0b95a4604d CR3: 000000002ef10000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
<IRQ>
lockdep_recursion_finish kernel/locking/lockdep.c:467 [inline]
lock_acquire+0x1ae/0x420 kernel/locking/lockdep.c:5756
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0xb4/0x100 kernel/locking/spinlock.c:162
hrtimer_interrupt+0xf8/0x980 kernel/time/hrtimer.c:1861
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1077 [inline]
__sysvec_apic_timer_interrupt+0xfb/0x3b0 arch/x86/kernel/apic/apic.c:1094
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1088 [inline]
sysvec_apic_timer_interrupt+0x51/0xc0 arch/x86/kernel/apic/apic.c:1088
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:687
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:160 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x29/0x50 kernel/locking/spinlock.c:202
Code: 00 f3 0f 1e fa 53 48 89 fb 48 83 c7 18 48 8b 74 24 08 e8 4a 09 d3 f6 48 89 df e8 f2 dc d3 f6 e8 cd d6 f7 f6 fb bf 01 00 00 00 <e8> 12 ed c6 f6 65 8b 05 e3 3b 6e 75 85 c0 74 02 5b c3 e8 90 20 6b
RSP: 0018:ffffc90000007c90 EFLAGS: 00000286
RAX: 703b4dcf9aca2b00 RBX: ffff8880b8e295c0 RCX: 703b4dcf9aca2b00
RDX: dffffc0000000000 RSI: ffffffff8acacb60 RDI: 0000000000000001
RBP: ffffc90000007e10 R08: ffffffff8e8c6b2f R09: 1ffffffff1d18d65
R10: dffffc0000000000 R11: fffffbfff1d18d66 R12: 1ffff110171c52c0
R13: ffff8880b8e295c0 R14: ffff888030b6d410 R15: ffffc90000007d60
expire_timers kernel/time/timer.c:1751 [inline]
__run_timers+0x560/0x810 kernel/time/timer.c:2023
run_timer_softirq+0x67/0xf0 kernel/time/timer.c:2036
handle_softirqs+0x27d/0x820 kernel/softirq.c:578
__do_softirq kernel/softirq.c:612 [inline]
invoke_softirq kernel/softirq.c:452 [inline]
__irq_exit_rcu+0xd3/0x190 kernel/softirq.c:661
irq_exit_rcu+0x9/0x20 kernel/softirq.c:673
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1088 [inline]
sysvec_apic_timer_interrupt+0xa4/0xc0 arch/x86/kernel/apic/apic.c:1088
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:687
RIP: 0010:finish_task_switch+0x267/0x8f0 kernel/sched/core.c:5255
Code: 2e 09 85 c0 0f 84 30 01 00 00 48 85 db 0f 85 4f 01 00 00 0f 1f 44 00 00 4c 89 e7 e8 43 32 38 09 e8 8e 0a 30 00 fb 4c 8b 65 b8 <49> 8d bc 24 f8 15 00 00 48 89 f8 48 c1 e8 03 42 0f b6 04 30 84 c0
RSP: 0018:ffffc900189e7980 EFLAGS: 00000282
RAX: 703b4dcf9aca2b00 RBX: 0000000000000000 RCX: 703b4dcf9aca2b00
RDX: dffffc0000000000 RSI: ffffffff8acacb60 RDI: ffffffff8b1cfa20
RBP: ffffc900189e79d0 R08: ffffffff8e8c6b2f R09: 1ffffffff1d18d65
R10: dffffc0000000000 R11: fffffbfff1d18d66 R12: ffff88802d103c00
R13: 1ffff110171c79a9 R14: dffffc0000000000 R15: ffff8880b8e3cd48
context_switch kernel/sched/core.c:5384 [inline]
__schedule+0x15b6/0x4660 kernel/sched/core.c:6700
preempt_schedule_common+0x82/0xc0 kernel/sched/core.c:6867
preempt_schedule+0xc0/0xd0 kernel/sched/core.c:6891
preempt_schedule_thunk+0x1a/0x30 arch/x86/entry/thunk_64.S:45
rcu_read_unlock_sched include/linux/rcupdate.h:901 [inline]
fd_install+0x397/0x4e0 fs/file.c:611
__do_sys_perf_event_open kernel/events/core.c:12970 [inline]
__se_sys_perf_event_open+0x19fe/0x1c60 kernel/events/core.c:12601
do_syscall_x64 arch/x86/entry/common.c:46 [inline]
do_syscall_64+0x55/0xb0 arch/x86/entry/common.c:76
entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f0693b9de99
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f0694b2a028 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007f0693e25fa0 RCX: 00007f0693b9de99
RDX: ffffffffffffffff RSI: 0000000000000000 RDI: 0000200000000680
RBP: 00007f0693c33eaf R08: 0000000000000000 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000000
R13: 00007f0693e26038 R14: 00007f0693e25fa0 R15: 00007ffcaf61ed48
</TASK>
CPU: 1 PID: 6420 Comm: syz.1.178 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/09/2026
RIP: 0010:on_stack arch/x86/include/asm/stacktrace.h:56 [inline]
RIP: 0010:unwind_next_frame+0x1caf/0x2990 arch/x86/kernel/unwind_orc.c:665
Code: f8 48 c1 e8 03 42 80 3c 38 00 74 05 e8 ea 1c a4 00 49 8b 5d 08 4d 8d 7d 10 4c 89 f8 48 c1 e8 03 48 b9 00 00 00 00 00 fc ff df <80> 3c 08 00 74 08 4c 89 ff e8 c3 1c a4 00 49 8b 07 4c 39 f3 0f 96
RSP: 0000:ffffc900001f04d8 EFLAGS: 00000a06
RAX: 1ffff9200003e0b7 RBX: ffffc90018798000 RCX: dffffc0000000000
RDX: ffff88801b7e5a00 RSI: 0000000000000001 RDI: ffffc900001f05b0
RBP: 0000000000000001 R08: ffff88801b7e5a00 R09: 0000000000000003
R10: 0000000000000004 R11: 0000000000000100 R12: ffffc9001879ff30
R13: ffffc900001f05a8 R14: ffffc9001879ff50 R15: ffffc900001f05b8
FS: 00007f2da59f66c0(0000) GS:ffff8880b8f00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000110c342afd CR3: 000000002ee5d000 CR4: 00000000003506e0
Call Trace:
<IRQ>
arch_stack_walk+0x144/0x190 arch/x86/kernel/stacktrace.c:25
stack_trace_save+0xaa/0x100 kernel/stacktrace.c:122
kasan_save_stack mm/kasan/common.c:46 [inline]
kasan_set_track+0x4e/0x70 mm/kasan/common.c:53
__kasan_slab_alloc+0x6c/0x80 mm/kasan/common.c:329
kasan_slab_alloc include/linux/kasan.h:188 [inline]
slab_post_alloc_hook+0x6a/0x4d0 mm/slab.h:767
slab_alloc_node mm/slub.c:3495 [inline]
kmem_cache_alloc_node+0x1af/0x310 mm/slub.c:3540
__alloc_skb+0x103/0x2c0 net/core/skbuff.c:643
alloc_skb include/linux/skbuff.h:1316 [inline]
new_skb drivers/block/aoe/aoecmd.c:66 [inline]
aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
aoecmd_cfg+0x2c5/0xa50 drivers/block/aoe/aoecmd.c:1374
call_timer_fn+0x189/0x540 kernel/time/timer.c:1701
expire_timers kernel/time/timer.c:1752 [inline]
__run_timers+0x570/0x810 kernel/time/timer.c:2023
run_timer_softirq+0x67/0xf0 kernel/time/timer.c:2036
handle_softirqs+0x27d/0x820 kernel/softirq.c:578
__do_softirq kernel/softirq.c:612 [inline]
invoke_softirq kernel/softirq.c:452 [inline]
__irq_exit_rcu+0xd3/0x190 kernel/softirq.c:661
irq_exit_rcu+0x9/0x20 kernel/softirq.c:673
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1088 [inline]
sysvec_apic_timer_interrupt+0xa4/0xc0 arch/x86/kernel/apic/apic.c:1088
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:687
RIP: 0010:preempt_schedule_irq+0xba/0x150 kernel/sched/core.c:7010
Code: 00 00 43 c6 44 37 04 f8 74 0b 0f 0b 48 f7 03 08 00 00 00 74 6f bf 01 00 00 00 e8 51 51 c8 f6 e8 9c 3c f9 f6 fb bf 01 00 00 00 <e8> 31 b4 ff ff 43 c6 44 37 08 00 48 c7 44 24 40 00 00 00 00 9c 8f
RSP: 0000:ffffc9001879fda0 EFLAGS: 00000286
RAX: 89435e2b1f905600 RBX: 0000000000000000 RCX: 89435e2b1f905600
RDX: dffffc0000000000 RSI: ffffffff8acacb60 RDI: 0000000000000001
RBP: ffffc9001879fe50 R08: ffffffff8e8c6b2f R09: 1ffffffff1d18d65
R10: dffffc0000000000 R11: fffffbfff1d18d66 R12: 0000000000000000
R13: 0000000000000000 R14: dffffc0000000000 R15: 1ffff920030f3fb4
irqentry_exit+0x67/0x70 kernel/entry/common.c:438
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:687
RIP: 0010:exit_to_user_mode_loop+0x40/0x110 kernel/entry/common.c:164
Code: fb eb 1d e8 02 39 18 09 65 48 8b 05 0a 14 90 7e 4c 8b 30 41 f7 c6 0e 30 02 00 0f 84 c4 00 00 00 e8 d5 ae 19 00 fb 41 f6 c6 08 <74> 05 e8 c9 6c 20 09 41 f7 c6 00 10 00 00 74 08 48 89 df e8 48 a4
RSP: 0000:ffffc9001879ff10 EFLAGS: 00000202
RAX: 89435e2b1f905600 RBX: ffffc9001879ff58 RCX: 89435e2b1f905600
RDX: dffffc0000000000 RSI: ffffffff8acacb60 RDI: ffffffff8b1cfa20
RBP: 0000000000000000 R08: ffffffff8e8c6b2f R09: 1ffffffff1d18d65
R10: dffffc0000000000 R11: fffffbfff1d18d66 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000008 R15: ffff88801b7e5a00
exit_to_user_mode_prepare+0xee/0x180 kernel/entry/common.c:210
irqentry_exit_to_user_mode+0x9/0x30 kernel/entry/common.c:315
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:687
RIP: 0033:0x7f2da779de99
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f2da59f6028 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 00007f2da7a25fa0 RCX: 00007f2da779de99
RDX: 0000000000000005 RSI: 0000000040042408 RDI: 0000000000000007
RBP: 00007f2da7833eaf R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f2da7a26038 R14: 00007f2da7a25fa0 R15: 00007ffc8fb20c98
</TASK>
* Struck through repros no longer work on HEAD.