syzbot


BUG: soft lockup in aoecmd_cfg (3)

Status: upstream: reported on 2025/04/30 11:13
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+5dfe55156cc098033526@syzkaller.appspotmail.com
First crash: 243d, last: 4d21h
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [block?] BUG: soft lockup in aoecmd_cfg (3) 0 (1) 2025/04/30 11:13
Similar bugs (6)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in aoecmd_cfg 1 2 473d 563d 0/3 auto-obsoleted due to no activity on 2024/12/07 10:13
upstream BUG: soft lockup in aoecmd_cfg (2) block 1 3 378d 410d 0/29 auto-obsoleted due to no activity on 2025/03/02 19:34
upstream BUG: soft lockup in aoecmd_cfg block 1 1 697d 693d 0/29 auto-obsoleted due to no activity on 2024/04/17 16:28
upstream INFO: rcu detected stall in aoecmd_cfg (2) usb block 1 C done 7 473d 586d 28/29 fixed on 2024/10/22 11:57
linux-5.15 INFO: rcu detected stall in aoecmd_cfg 1 1 120d 120d 0/3 auto-obsoleted due to no activity on 2025/11/25 17:18
linux-5.15 INFO: rcu detected stall in aoecmd_cfg (2) 1 1 19d 19d 0/3 upstream: reported on 2025/11/26 13:19

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 118s! [syz.1.4576:23321]
Modules linked in:
irq event stamp: 16050927
hardirqs last  enabled at (16050926): [<ffffffff8b494f9d>] irqentry_exit+0x5dd/0x660 kernel/entry/common.c:219
hardirqs last disabled at (16050927): [<ffffffff8b49380e>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1056
softirqs last  enabled at (15999716): [<ffffffff8185372a>] __do_softirq kernel/softirq.c:656 [inline]
softirqs last  enabled at (15999716): [<ffffffff8185372a>] invoke_softirq kernel/softirq.c:496 [inline]
softirqs last  enabled at (15999716): [<ffffffff8185372a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:723
softirqs last disabled at (15999719): [<ffffffff8185372a>] __do_softirq kernel/softirq.c:656 [inline]
softirqs last disabled at (15999719): [<ffffffff8185372a>] invoke_softirq kernel/softirq.c:496 [inline]
softirqs last disabled at (15999719): [<ffffffff8185372a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:723
CPU: 0 UID: 0 PID: 23321 Comm: syz.1.4576 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/25/2025
RIP: 0010:lockdep_enabled kernel/locking/lockdep.c:121 [inline]
RIP: 0010:lock_release+0x60/0x3b0 kernel/locking/lockdep.c:5881
Code: 87 02 00 00 89 c0 48 0f a3 05 ac ff e1 0d 73 0d e8 45 ea 08 00 84 c0 0f 84 a3 02 00 00 83 3d b6 2f e2 0d 00 0f 84 3e 02 00 00 <65> 8b 05 89 d9 d8 10 85 c0 0f 85 2f 02 00 00 65 4c 8b 3c 25 08 f0
RSP: 0018:ffffc90000007410 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffffffff8173ef25 RCX: 50af6e211b697e00
RDX: 0000000000000100 RSI: ffffffff8bbfc640 RDI: ffffffff8bbfc600
RBP: dffffc0000000000 R08: ffffc90000007097 R09: 0000000000000000
R10: ffffc90000007088 R11: fffff52000000e13 R12: ffffc90000007ca0
R13: ffffffff8173ef25 R14: ffffffff8df41cc0 R15: ffffffff8df41cc0
FS:  00007f245e7f86c0(0000) GS:ffff8881260b1000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000110c345c8b CR3: 0000000027872000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 rcu_lock_release include/linux/rcupdate.h:341 [inline]
 rcu_read_unlock include/linux/rcupdate.h:897 [inline]
 class_rcu_destructor include/linux/rcupdate.h:1195 [inline]
 unwind_next_frame+0x19a9/0x2390 arch/x86/kernel/unwind_orc.c:680
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x9c/0xe0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:56 [inline]
 kasan_save_track+0x3e/0x80 mm/kasan/common.c:77
 unpoison_slab_object mm/kasan/common.c:342 [inline]
 __kasan_slab_alloc+0x6c/0x80 mm/kasan/common.c:368
 kasan_slab_alloc include/linux/kasan.h:252 [inline]
 slab_post_alloc_hook mm/slub.c:4948 [inline]
 slab_alloc_node mm/slub.c:5258 [inline]
 kmem_cache_alloc_node_noprof+0x433/0x710 mm/slub.c:5310
 __alloc_skb+0x255/0x430 net/core/skbuff.c:679
 alloc_skb include/linux/skbuff.h:1383 [inline]
 new_skb+0x2f/0x2b0 drivers/block/aoe/aoecmd.c:66
 aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
 aoecmd_cfg+0x28b/0x7c0 drivers/block/aoe/aoecmd.c:1374
 call_timer_fn+0x16e/0x590 kernel/time/timer.c:1748
 expire_timers kernel/time/timer.c:1799 [inline]
 __run_timers kernel/time/timer.c:2373 [inline]
 __run_timer_base+0x61a/0x860 kernel/time/timer.c:2385
 run_timer_base kernel/time/timer.c:2394 [inline]
 run_timer_softirq+0xb7/0x180 kernel/time/timer.c:2404
 handle_softirqs+0x27d/0x850 kernel/softirq.c:622
 __do_softirq kernel/softirq.c:656 [inline]
 invoke_softirq kernel/softirq.c:496 [inline]
 __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:723
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:739
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1056 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1056
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:697
RIP: 0010:rcu_read_unlock_special+0x87/0x520 kernel/rcu/tree_plugin.h:770
Code: f1 f1 f1 00 f2 f2 f2 4a 89 04 2b 66 42 c7 44 2b 09 f3 f3 42 c6 44 2b 0b f3 65 44 8b 35 32 f0 ce 10 41 f7 c6 00 00 f0 00 74 49 <48> c7 44 24 20 0e 36 e0 45 4a c7 04 2b 00 00 00 00 66 42 c7 44 2b
RSP: 0018:ffffc9000bd7f2a0 EFLAGS: 00000206
RAX: 50af6e211b697e00 RBX: 1ffff920017afe58 RCX: 50af6e211b697e00
RDX: 0000000000000000 RSI: ffffffff8d76bd39 RDI: ffffffff8bbfc660
RBP: ffffc9000bd7f378 R08: ffffffff8f805a77 R09: 1ffffffff1f00b4e
R10: dffffc0000000000 R11: fffffbfff1f00b4f R12: 0000000000000246
R13: dffffc0000000000 R14: 0000000000000001 R15: ffff8880b883b700
 __rcu_read_unlock+0x84/0xe0 kernel/rcu/tree_plugin.h:438
 rcu_read_unlock include/linux/rcupdate.h:899 [inline]
 class_rcu_destructor include/linux/rcupdate.h:1195 [inline]
 unwind_next_frame+0x19ae/0x2390 arch/x86/kernel/unwind_orc.c:680
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x9c/0xe0 kernel/stacktrace.c:122
 kasan_save_stack+0x3e/0x60 mm/kasan/common.c:56
 kasan_record_aux_stack+0xbd/0xd0 mm/kasan/generic.c:559
 __call_rcu_common kernel/rcu/tree.c:3119 [inline]
 call_rcu+0x157/0x9c0 kernel/rcu/tree.c:3239
 context_switch kernel/sched/core.c:5259 [inline]
 __schedule+0x14c4/0x5000 kernel/sched/core.c:6863
 preempt_schedule_common+0x83/0xd0 kernel/sched/core.c:7047
 preempt_schedule+0xae/0xc0 kernel/sched/core.c:7071
 preempt_schedule_thunk+0x16/0x30 arch/x86/entry/thunk.S:12
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
 _raw_spin_unlock_irqrestore+0xfd/0x110 kernel/locking/spinlock.c:194
 debug_timer_init kernel/time/timer.c:788 [inline]
 debug_init kernel/time/timer.c:836 [inline]
 timer_init_key+0x41/0x2b0 kernel/time/timer.c:880
 sock_init_data_uid+0x70/0x980 net/core/sock.c:3701
 packet_create+0x1d4/0x780 net/packet/af_packet.c:3361
 __sock_create+0x4b3/0x9d0 net/socket.c:1596
 sock_create net/socket.c:1654 [inline]
 __sys_socket_create net/socket.c:1691 [inline]
 __sys_socket+0xd7/0x320 net/socket.c:1738
 __do_sys_socket net/socket.c:1752 [inline]
 __se_sys_socket net/socket.c:1750 [inline]
 __x64_sys_socket+0x7a/0x90 net/socket.c:1750
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f245d98f749
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f245e7f8038 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00007f245dbe5fa0 RCX: 00007f245d98f749
RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000011
RBP: 00007f245da13f91 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f245dbe6038 R14: 00007f245dbe5fa0 R15: 00007fff78a20fe8
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 284 Comm: kworker/u8:6 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/25/2025
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:342 [inline]
RIP: 0010:smp_call_function_many_cond+0xccf/0x12b0 kernel/smp.c:877
Code: 45 8b 2c 24 44 89 ee 83 e6 01 31 ff e8 4a 94 0b 00 41 83 e5 01 49 bd 00 00 00 00 00 fc ff df 75 07 e8 f5 8f 0b 00 eb 38 f3 90 <42> 0f b6 04 2b 84 c0 75 11 41 f7 04 24 01 00 00 00 74 1e e8 d9 8f
RSP: 0018:ffffc900030a7620 EFLAGS: 00000293
RAX: ffffffff81b5ba87 RBX: 1ffff11017108545 RCX: ffff888021b49e80
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc900030a77a0 R08: ffffffff8f805a77 R09: 1ffffffff1f00b4e
R10: dffffc0000000000 R11: fffffbfff1f00b4f R12: ffff8880b8842a28
R13: dffffc0000000000 R14: ffff8880b893b9c0 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8881261b1000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f281f117d60 CR3: 000000000dd3a000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000200000000300
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1043
 on_each_cpu include/linux/smp.h:71 [inline]
 smp_text_poke_sync_each_cpu arch/x86/kernel/alternative.c:2711 [inline]
 smp_text_poke_batch_finish+0x5f9/0x1130 arch/x86/kernel/alternative.c:2921
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x128/0x240 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 toggle_allocation_gate+0xad/0x240 mm/kfence/core.c:854
 process_one_work kernel/workqueue.c:3257 [inline]
 process_scheduled_works+0xad1/0x1770 kernel/workqueue.c:3340
 worker_thread+0x8a0/0xda0 kernel/workqueue.c:3421
 kthread+0x711/0x8a0 kernel/kthread.c:463
 ret_from_fork+0x599/0xb30 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:246
 </TASK>

Crashes (16):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/12/07 03:44 bpf 861111b69896 d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/11/22 13:26 bpf 22d70d400556 4fb8ef37 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/09/16 22:47 bpf f36caa7c14f4 e2beed91 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/23 11:45 bpf 7abc678e3084 e1dd4f22 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/06 17:25 bpf bf4807c89d8f 4f67c4ae .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/22 19:30 bpf d4adf1c9ee77 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/12/11 02:37 bpf-next 759377dab35e d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/02 15:20 bpf-next 212ec9229567 0cd59a8f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/23 01:21 bpf-next 99fe8af069a9 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/28 05:36 bpf-next db22b1382b96 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/27 14:44 bpf-next 079e5c56a5c4 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 11:12 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 10:52 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/24 13:25 bpf-next 60400cd2b9be 9c80ffa0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/19 00:22 bpf-next 8582d9ab3efd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/16 07:41 bpf-next 7d0b43b68d1c 23b969b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
* Struck through repros no longer work on HEAD.