syzbot


BUG: soft lockup in aoecmd_cfg (3)

Status: upstream: reported on 2025/04/30 11:13
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+5dfe55156cc098033526@syzkaller.appspotmail.com
First crash: 152d, last: 54d
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [block?] BUG: soft lockup in aoecmd_cfg (3) 0 (1) 2025/04/30 11:13
Similar bugs (5)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in aoecmd_cfg 1 2 382d 472d 0/3 auto-obsoleted due to no activity on 2024/12/07 10:13
upstream BUG: soft lockup in aoecmd_cfg (2) block 1 3 286d 318d 0/29 auto-obsoleted due to no activity on 2025/03/02 19:34
upstream BUG: soft lockup in aoecmd_cfg block 1 1 606d 602d 0/29 auto-obsoleted due to no activity on 2024/04/17 16:28
upstream INFO: rcu detected stall in aoecmd_cfg (2) usb block 1 C done 7 382d 494d 28/29 fixed on 2024/10/22 11:57
linux-5.15 INFO: rcu detected stall in aoecmd_cfg 1 1 29d 29d 0/3 upstream: reported on 2025/08/17 17:18

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 123s! [syz.3.34:6105]
Modules linked in:
irq event stamp: 10292293
hardirqs last  enabled at (10292292): [<ffffffff8b6d8424>] irqentry_exit+0x74/0x90 kernel/entry/common.c:310
hardirqs last disabled at (10292293): [<ffffffff8b6d6f6e>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1050
softirqs last  enabled at (9901654): [<ffffffff8185bdba>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last  enabled at (9901654): [<ffffffff8185bdba>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last  enabled at (9901654): [<ffffffff8185bdba>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
softirqs last disabled at (9901657): [<ffffffff8185bdba>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last disabled at (9901657): [<ffffffff8185bdba>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last disabled at (9901657): [<ffffffff8185bdba>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
CPU: 0 UID: 0 PID: 6105 Comm: syz.3.34 Not tainted 6.16.0-rc6-syzkaller-g7abc678e3084 #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
RIP: 0010:lock_is_held_type+0x137/0x190 kernel/locking/lockdep.c:5948
Code: 01 75 44 48 c7 04 24 00 00 00 00 9c 8f 04 24 f7 04 24 00 02 00 00 75 4c 41 f7 c4 00 02 00 00 74 01 fb 65 48 8b 05 f9 52 32 07 <48> 3b 44 24 08 75 43 89 d8 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f
RSP: 0018:ffffc90000007178 EFLAGS: 00000206
RAX: d0a6df39c8afcf00 RBX: 0000000000000001 RCX: d0a6df39c8afcf00
RDX: ffff888025963c00 RSI: ffffffff8db83c98 RDI: ffffffff8be28c00
RBP: 00000000ffffffff R08: 0000000000000000 R09: ffffffff81cb5957
R10: dffffc0000000000 R11: fffff91ffff96205 R12: 0000000000000246
R13: ffff888025963c00 R14: ffffffff8e13f0e0 R15: 0000000000000002
FS:  00007f7fc61a06c0(0000) GS:ffff888125c23000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b31411ff8 CR3: 000000006e60a000 CR4: 00000000003526f0
DR0: 0000200000000300 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 00000000000f0602
Call Trace:
 <IRQ>
 trace_call_bpf+0x734/0x850 kernel/trace/bpf_trace.c:124
 perf_trace_run_bpf_submit+0x78/0x170 kernel/events/core.c:10891
 do_perf_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 perf_trace_lock_acquire+0x335/0x410 include/trace/events/lock.h:24
 __do_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 trace_lock_acquire include/trace/events/lock.h:24 [inline]
 lock_acquire+0x311/0x360 kernel/locking/lockdep.c:5834
 rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
 rcu_read_lock include/linux/rcupdate.h:841 [inline]
 class_rcu_constructor include/linux/rcupdate.h:1155 [inline]
 unwind_next_frame+0xc2/0x2390 arch/x86/kernel/unwind_orc.c:479
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x9c/0xe0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3e/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:319 [inline]
 __kasan_slab_alloc+0x6c/0x80 mm/kasan/common.c:345
 kasan_slab_alloc include/linux/kasan.h:250 [inline]
 slab_post_alloc_hook mm/slub.c:4148 [inline]
 slab_alloc_node mm/slub.c:4197 [inline]
 kmem_cache_alloc_node_noprof+0x1bb/0x3c0 mm/slub.c:4249
 kmalloc_reserve+0xbd/0x290 net/core/skbuff.c:579
 __alloc_skb+0x142/0x2d0 net/core/skbuff.c:670
 alloc_skb include/linux/skbuff.h:1336 [inline]
 new_skb+0x2f/0x2b0 drivers/block/aoe/aoecmd.c:66
 aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
 aoecmd_cfg+0x28b/0x7c0 drivers/block/aoe/aoecmd.c:1374
 call_timer_fn+0x17e/0x5f0 kernel/time/timer.c:1747
 expire_timers kernel/time/timer.c:1798 [inline]
 __run_timers kernel/time/timer.c:2372 [inline]
 __run_timer_base+0x61a/0x860 kernel/time/timer.c:2384
 run_timer_base kernel/time/timer.c:2393 [inline]
 run_timer_softirq+0xb7/0x180 kernel/time/timer.c:2403
 handle_softirqs+0x286/0x870 kernel/softirq.c:579
 __do_softirq kernel/softirq.c:613 [inline]
 invoke_softirq kernel/softirq.c:453 [inline]
 __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:696
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1050 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1050
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__sanitizer_cov_trace_pc+0x46/0x70 kernel/kcov.c:222
Code: ff 00 74 11 81 fa 00 01 00 00 75 35 83 b9 3c 16 00 00 00 74 2c 8b 91 18 16 00 00 83 fa 02 75 21 48 8b 91 20 16 00 00 48 8b 32 <48> 8d 7e 01 8b 89 1c 16 00 00 48 39 cf 73 08 48 89 3a 48 89 44 f2
RSP: 0018:ffffc9000c17ee18 EFLAGS: 00000246
RAX: ffffffff81cb59a4 RBX: 0000000000000000 RCX: ffff888025963c00
RDX: ffffc9000ca79000 RSI: 00000000000193be RDI: ffffffff8be28ba0
RBP: ffffc9000c17ef18 R08: 0000000000000000 R09: ffffffff81cb5957
R10: dffffc0000000000 R11: fffff91ffff95e05 R12: 0000000000000001
R13: 1ffff9200182fdd0 R14: ffffffff81cb5957 R15: ffffffff8e00e120
 rcu_read_lock include/linux/rcupdate.h:842 [inline]
 trace_call_bpf+0x104/0x850 kernel/trace/bpf_trace.c:145
 perf_trace_run_bpf_submit+0x78/0x170 kernel/events/core.c:10891
 do_perf_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 perf_trace_lock_acquire+0x335/0x410 include/trace/events/lock.h:24
 __do_trace_lock_acquire include/trace/events/lock.h:24 [inline]
 trace_lock_acquire include/trace/events/lock.h:24 [inline]
 lock_acquire+0x311/0x360 kernel/locking/lockdep.c:5834
 rcu_lock_acquire include/linux/rcupdate.h:331 [inline]
 rcu_read_lock include/linux/rcupdate.h:841 [inline]
 tcp_metrics_nl_dump+0x177/0x800 net/ipv4/tcp_metrics.c:780
 genl_dumpit+0x10b/0x1b0 net/netlink/genetlink.c:1027
 netlink_dump+0x6de/0xe60 net/netlink/af_netlink.c:2327
 __netlink_dump_start+0x5cb/0x7e0 net/netlink/af_netlink.c:2442
 genl_family_rcv_msg_dumpit+0x1e7/0x2c0 net/netlink/genetlink.c:1076
 genl_family_rcv_msg net/netlink/genetlink.c:1192 [inline]
 genl_rcv_msg+0x5da/0x790 net/netlink/genetlink.c:1210
 netlink_rcv_skb+0x208/0x470 net/netlink/af_netlink.c:2552
 genl_rcv+0x28/0x40 net/netlink/genetlink.c:1219
 netlink_unicast_kernel net/netlink/af_netlink.c:1320 [inline]
 netlink_unicast+0x75c/0x8e0 net/netlink/af_netlink.c:1346
 netlink_sendmsg+0x805/0xb30 net/netlink/af_netlink.c:1896
 sock_sendmsg_nosec net/socket.c:712 [inline]
 __sock_sendmsg+0x21c/0x270 net/socket.c:727
 ____sys_sendmsg+0x505/0x830 net/socket.c:2566
 ___sys_sendmsg+0x21f/0x2a0 net/socket.c:2620
 __sys_sendmsg net/socket.c:2652 [inline]
 __do_sys_sendmsg net/socket.c:2657 [inline]
 __se_sys_sendmsg net/socket.c:2655 [inline]
 __x64_sys_sendmsg+0x19b/0x260 net/socket.c:2655
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0x3b0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f7fc538e9a9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f7fc61a0038 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007f7fc55b5fa0 RCX: 00007f7fc538e9a9
RDX: 0000000004000010 RSI: 0000200000000000 RDI: 0000000000000009
RBP: 00007f7fc5410d69 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f7fc55b5fa0 R15: 00007fff30280578
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 5839 Comm: syz-executor Not tainted 6.16.0-rc6-syzkaller-g7abc678e3084 #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
RIP: 0010:csd_lock_wait kernel/smp.c:340 [inline]
RIP: 0010:smp_call_function_many_cond+0xf69/0x12d0 kernel/smp.c:885
Code: 00 45 8b 2f 44 89 ee 83 e6 01 31 ff e8 a0 6e 0b 00 41 83 e5 01 49 bd 00 00 00 00 00 fc ff df 75 07 e8 4b 6a 0b 00 eb 37 f3 90 <43> 0f b6 04 2c 84 c0 75 10 41 f7 07 01 00 00 00 74 1e e8 30 6a 0b
RSP: 0000:ffffc9000422f360 EFLAGS: 00000293
RAX: ffffffff81b4bec0 RBX: ffff8880b873b1c0 RCX: ffff88802eb40000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc9000422f4c0 R08: ffffffff8fa1d6f7 R09: 1ffffffff1f43ade
R10: dffffc0000000000 R11: fffffbfff1f43adf R12: 1ffff110170c8385
R13: dffffc0000000000 R14: 0000000000000000 R15: ffff8880b8641c28
FS:  0000000000000000(0000) GS:ffff888125d23000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f101b282380 CR3: 000000000df38000 CR4: 00000000003526f0
DR0: 0000200000000300 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1052
 __flush_tlb_multi arch/x86/include/asm/paravirt.h:91 [inline]
 flush_tlb_multi arch/x86/mm/tlb.c:1361 [inline]
 flush_tlb_mm_range+0x6b1/0x12c0 arch/x86/mm/tlb.c:1451
 tlb_flush arch/x86/include/asm/tlb.h:23 [inline]
 tlb_flush_mmu_tlbonly include/asm-generic/tlb.h:490 [inline]
 tlb_flush_mmu+0x1a7/0x680 mm/mmu_gather.c:403
 tlb_finish_mmu+0xc3/0x1d0 mm/mmu_gather.c:497
 free_ldt_pgtables+0x17b/0x320 arch/x86/kernel/ldt.c:411
 arch_exit_mmap arch/x86/include/asm/mmu_context.h:234 [inline]
 exit_mmap+0x17c/0xb50 mm/mmap.c:1270
 __mmput+0x118/0x420 kernel/fork.c:1121
 exit_mm+0x1da/0x2c0 kernel/exit.c:581
 do_exit+0x648/0x22e0 kernel/exit.c:952
 do_group_exit+0x21c/0x2d0 kernel/exit.c:1105
 get_signal+0x1286/0x1340 kernel/signal.c:3034
 arch_do_signal_or_restart+0x9a/0x750 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop+0x75/0x110 kernel/entry/common.c:111
 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline]
 syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline]
 syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline]
 do_syscall_64+0x2bd/0x3b0 arch/x86/entry/syscall_64.c:100
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f3125f84bd3
Code: Unable to access opcode bytes at 0x7f3125f84ba9.
RSP: 002b:00007ffdea95fe98 EFLAGS: 00000202 ORIG_RAX: 000000000000003d
RAX: fffffffffffffe00 RBX: 00000000000016dd RCX: 00007f3125f84bd3
RDX: 0000000040000000 RSI: 00007ffdea95feac RDI: 00000000ffffffff
RBP: 00007ffdea95feac R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000008
R13: 0000000000000003 R14: 0000000000000009 R15: 0000000000000000
 </TASK>

Crashes (12):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/07/23 11:45 bpf 7abc678e3084 e1dd4f22 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/06 17:25 bpf bf4807c89d8f 4f67c4ae .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/22 19:30 bpf d4adf1c9ee77 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/02 15:20 bpf-next 212ec9229567 0cd59a8f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/23 01:21 bpf-next 99fe8af069a9 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/28 05:36 bpf-next db22b1382b96 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/27 14:44 bpf-next 079e5c56a5c4 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 11:12 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 10:52 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/24 13:25 bpf-next 60400cd2b9be 9c80ffa0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/19 00:22 bpf-next 8582d9ab3efd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/16 07:41 bpf-next 7d0b43b68d1c 23b969b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
* Struck through repros no longer work on HEAD.