syzbot


BUG: soft lockup in aoecmd_cfg (3)

Status: upstream: reported on 2025/04/30 11:13
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+5dfe55156cc098033526@syzkaller.appspotmail.com
First crash: 198d, last: 44d
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [block?] BUG: soft lockup in aoecmd_cfg (3) 0 (1) 2025/04/30 11:13
Similar bugs (5)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in aoecmd_cfg 1 2 427d 517d 0/3 auto-obsoleted due to no activity on 2024/12/07 10:13
upstream BUG: soft lockup in aoecmd_cfg (2) block 1 3 332d 364d 0/29 auto-obsoleted due to no activity on 2025/03/02 19:34
upstream BUG: soft lockup in aoecmd_cfg block 1 1 651d 647d 0/29 auto-obsoleted due to no activity on 2024/04/17 16:28
upstream INFO: rcu detected stall in aoecmd_cfg (2) usb block 1 C done 7 427d 540d 28/29 fixed on 2024/10/22 11:57
linux-5.15 INFO: rcu detected stall in aoecmd_cfg 1 1 74d 74d 0/3 upstream: reported on 2025/08/17 17:18

Sample crash report:
watchdog: BUG: soft lockup - CPU#1 stuck for 122s! [syz.7.3492:17609]
Modules linked in:
irq event stamp: 12937473
hardirqs last  enabled at (12937472): [<ffffffff8b7a4234>] irqentry_exit+0x74/0x90 kernel/entry/common.c:200
hardirqs last disabled at (12937473): [<ffffffff8b7a2e6e>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1050
softirqs last  enabled at (11423372): [<ffffffff8185050a>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last  enabled at (11423372): [<ffffffff8185050a>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last  enabled at (11423372): [<ffffffff8185050a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
softirqs last disabled at (11423375): [<ffffffff8185050a>] __do_softirq kernel/softirq.c:613 [inline]
softirqs last disabled at (11423375): [<ffffffff8185050a>] invoke_softirq kernel/softirq.c:453 [inline]
softirqs last disabled at (11423375): [<ffffffff8185050a>] __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
CPU: 1 UID: 0 PID: 17609 Comm: syz.7.3492 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/18/2025
RIP: 0010:__kernel_text_address+0x0/0x40 kernel/extable.c:78
Code: c1 03 38 c1 7c c1 48 c7 c7 80 62 a3 8f e8 38 a2 96 00 eb b3 66 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 <66> 0f 1f 00 53 48 89 fb e8 43 00 00 00 85 c0 0f 95 c0 48 81 fb 00
RSP: 0018:ffffc90000a08120 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffc90000a08190 RCX: fc13fd74f476d801
RDX: ffffc90000a081d8 RSI: dffffc0000000000 RDI: ffffffff81ac4aac
RBP: ffffc90000a081d0 R08: ffffc90000a081d0 R09: 0000000000000000
R10: ffffc90000a08198 R11: fffff52000141035 R12: ffff888078d91e00
R13: 1ffff92000141054 R14: dffffc0000000000 R15: 1ffff92000141032
FS:  00007f3a673f66c0(0000) GS:ffff888125d15000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5fd33056c0 CR3: 0000000049630000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 unwind_get_return_address+0x4d/0x90 arch/x86/kernel/unwind_orc.c:369
 arch_stack_walk+0xfc/0x150 arch/x86/kernel/stacktrace.c:26
 stack_trace_save+0x9c/0xe0 kernel/stacktrace.c:122
 save_stack+0xf5/0x1f0 mm/page_owner.c:156
 __set_page_owner+0x8d/0x4a0 mm/page_owner.c:329
 set_page_owner include/linux/page_owner.h:32 [inline]
 post_alloc_hook+0x240/0x2a0 mm/page_alloc.c:1851
 prep_new_page mm/page_alloc.c:1859 [inline]
 get_page_from_freelist+0x21e4/0x22c0 mm/page_alloc.c:3858
 __alloc_frozen_pages_noprof+0x181/0x370 mm/page_alloc.c:5148
 alloc_pages_mpol+0x232/0x4a0 mm/mempolicy.c:2416
 alloc_slab_page mm/slub.c:2492 [inline]
 allocate_slab+0x8a/0x370 mm/slub.c:2660
 new_slab mm/slub.c:2714 [inline]
 ___slab_alloc+0xbeb/0x1420 mm/slub.c:3901
 __slab_alloc mm/slub.c:3992 [inline]
 __slab_alloc_node mm/slub.c:4067 [inline]
 slab_alloc_node mm/slub.c:4228 [inline]
 kmem_cache_alloc_node_noprof+0x280/0x3c0 mm/slub.c:4292
 __alloc_skb+0x112/0x2d0 net/core/skbuff.c:659
 alloc_skb include/linux/skbuff.h:1336 [inline]
 new_skb+0x2f/0x2b0 drivers/block/aoe/aoecmd.c:66
 aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
 aoecmd_cfg+0x28b/0x7c0 drivers/block/aoe/aoecmd.c:1374
 call_timer_fn+0x17b/0x5f0 kernel/time/timer.c:1747
 expire_timers kernel/time/timer.c:1798 [inline]
 __run_timers kernel/time/timer.c:2372 [inline]
 __run_timer_base+0x61a/0x860 kernel/time/timer.c:2384
 run_timer_base kernel/time/timer.c:2393 [inline]
 run_timer_softirq+0xb7/0x180 kernel/time/timer.c:2403
 handle_softirqs+0x283/0x870 kernel/softirq.c:579
 __do_softirq kernel/softirq.c:613 [inline]
 invoke_softirq kernel/softirq.c:453 [inline]
 __irq_exit_rcu+0xca/0x1f0 kernel/softirq.c:680
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:696
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1050 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1050
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:preempt_schedule_irq+0xb0/0x150 kernel/sched/core.c:7288
Code: 24 20 f6 44 24 21 02 74 0c 90 0f 0b 48 f7 03 08 00 00 00 74 64 bf 01 00 00 00 e8 ab a8 14 f6 e8 16 05 4c f6 fb bf 01 00 00 00 <e8> 5b ab ff ff 48 c7 44 24 40 00 00 00 00 9c 8f 44 24 40 8b 44 24
RSP: 0018:ffffc9000aee7ba0 EFLAGS: 00000286
RAX: fc13fd74f476d800 RBX: ffff888078d91e00 RCX: fc13fd74f476d800
RDX: 0000000000000006 RSI: ffffffff8d9b9eb1 RDI: 0000000000000001
RBP: ffffc9000aee7c40 R08: ffffffff8fa3a437 R09: 1ffffffff1f47486
R10: dffffc0000000000 R11: fffffbfff1f47487 R12: 0000000000000000
R13: 0000000000000000 R14: dffffc0000000000 R15: 1ffff920015dcf74
 irqentry_exit+0x6f/0x90 kernel/entry/common.c:197
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__do_sys_perf_event_open kernel/events/core.c:13719 [inline]
RIP: 0010:__se_sys_perf_event_open+0x193a/0x1d70 kernel/events/core.c:13360
Code: 00 00 00 fc ff df 0f b6 04 08 84 c0 0f 85 1a 03 00 00 48 8b 74 24 10 8b 96 0c 03 00 00 48 8b 5c 24 28 48 89 df e8 26 49 00 00 <48> 89 df e8 3e 22 ff ff 48 8b 7c 24 60 e8 94 ce 8b 09 80 7c 24 20
RSP: 0018:ffffc9000aee7d00 EFLAGS: 00000246
RAX: fc13fd74f476d800 RBX: ffff88805528f400 RCX: 0000000000080000
RDX: ffffc9001ffe8000 RSI: 0000000000002c4a RDI: 0000000000002c4b
RBP: ffffc9000aee7ee0 R08: ffffffff8fa3a437 R09: 1ffffffff1f47486
R10: dffffc0000000000 R11: fffffbfff1f47487 R12: ffff88805528f550
R13: ffff888034d35dc0 R14: ffff888078d91e00 R15: 1ffff920015dcfb0
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0x3b0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f3a6918eba9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f3a673f6038 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007f3a693d5fa0 RCX: 00007f3a6918eba9
RDX: ffffffffffffffff RSI: 0000000000000000 RDI: 00002000000000c0
RBP: 00007f3a69211e19 R08: 0000000000000008 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000000
R13: 00007f3a693d6038 R14: 00007f3a693d5fa0 R15: 00007fff9f0f55e8
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 12 Comm: kworker/u8:0 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/18/2025
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:342 [inline]
RIP: 0010:smp_call_function_many_cond+0xd3c/0x12d0 kernel/smp.c:877
Code: 01 31 ff e8 56 63 0b 00 41 83 e5 01 49 bd 00 00 00 00 00 fc ff df 75 07 e8 01 5f 0b 00 eb 38 f3 90 42 0f b6 04 2b 84 c0 75 11 <41> f7 04 24 01 00 00 00 74 1e e8 e5 5e 0b 00 eb e4 44 89 e1 80 e1
RSP: 0000:ffffc90000117660 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 1ffff110170e7f61 RCX: ffff88801cea5a00
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc900001177e0 R08: ffffffff8fa3a437 R09: 1ffffffff1f47486
R10: dffffc0000000000 R11: fffffbfff1f47487 R12: ffff8880b873fb08
R13: dffffc0000000000 R14: ffff8880b863b1c0 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff888125c15000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa42c200218 CR3: 000000000df36000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1044
 on_each_cpu include/linux/smp.h:71 [inline]
 smp_text_poke_sync_each_cpu arch/x86/kernel/alternative.c:2653 [inline]
 smp_text_poke_batch_finish+0x5f9/0x1130 arch/x86/kernel/alternative.c:2863
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x128/0x250 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 toggle_allocation_gate+0xad/0x240 mm/kfence/core.c:850
 process_one_work kernel/workqueue.c:3236 [inline]
 process_scheduled_works+0xae1/0x17b0 kernel/workqueue.c:3319
 worker_thread+0x8a0/0xda0 kernel/workqueue.c:3400
 kthread+0x70e/0x8a0 kernel/kthread.c:463
 ret_from_fork+0x436/0x7d0 arch/x86/kernel/process.c:148
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>

Crashes (13):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/09/16 22:47 bpf f36caa7c14f4 e2beed91 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/23 11:45 bpf 7abc678e3084 e1dd4f22 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/06 17:25 bpf bf4807c89d8f 4f67c4ae .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/22 19:30 bpf d4adf1c9ee77 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/07/02 15:20 bpf-next 212ec9229567 0cd59a8f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/06/23 01:21 bpf-next 99fe8af069a9 d6cdfb8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/28 05:36 bpf-next db22b1382b96 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/05/27 14:44 bpf-next 079e5c56a5c4 874a1386 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 11:12 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/30 10:52 bpf-next 38d976c32d85 85a5a23f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/24 13:25 bpf-next 60400cd2b9be 9c80ffa0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/19 00:22 bpf-next 8582d9ab3efd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2025/04/16 07:41 bpf-next 7d0b43b68d1c 23b969b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
* Struck through repros no longer work on HEAD.