syzbot


BUG: soft lockup in sys_perf_event_open

Status: moderation: reported on 2024/06/04 02:01
Subsystems: kvm
[Documentation on labels]
Reported-by: syzbot+11102fe1a02ecad3c929@syzkaller.appspotmail.com
First crash: 18d, last: 18d
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-5-15 BUG: soft lockup in sys_perf_event_open 1 43d 43d 0/2 premoderation: reported on 2024/05/06 07:00
linux-5.15 INFO: rcu detected stall in sys_perf_event_open (2) origin:lts-only C error 1 162d 160d 0/3 upstream: reported C repro on 2024/01/09 18:19
linux-6.1 INFO: rcu detected stall in sys_perf_event_open 1 7d10h 7d10h 0/3 upstream: reported on 2024/06/11 01:45
linux-5.15 INFO: rcu detected stall in sys_perf_event_open 1 426d 426d 0/3 auto-obsoleted due to no activity on 2023/08/17 04:33

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 143s! [syz-executor.3:7017]
CPU#0 Utilization every 22s during lockup:
	#1:  98% system,	  3% softirq,	  1% hardirq,	  0% idle
	#2:  98% system,	  3% softirq,	  1% hardirq,	  0% idle
	#3:  97% system,	  3% softirq,	  1% hardirq,	  0% idle
	#4:  98% system,	  3% softirq,	  1% hardirq,	  0% idle
	#5:  98% system,	  3% softirq,	  1% hardirq,	  0% idle
Modules linked in:
irq event stamp: 741138
hardirqs last  enabled at (741137): [<ffffffff8140372b>] kvm_wait+0x23b/0x2c0
hardirqs last disabled at (741138): [<ffffffff8b85d2ae>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1043
softirqs last  enabled at (741122): [<ffffffff8159fb84>] __do_softirq kernel/softirq.c:588 [inline]
softirqs last  enabled at (741122): [<ffffffff8159fb84>] invoke_softirq kernel/softirq.c:428 [inline]
softirqs last  enabled at (741122): [<ffffffff8159fb84>] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
softirqs last disabled at (741083): [<ffffffff8159fb84>] __do_softirq kernel/softirq.c:588 [inline]
softirqs last disabled at (741083): [<ffffffff8159fb84>] invoke_softirq kernel/softirq.c:428 [inline]
softirqs last disabled at (741083): [<ffffffff8159fb84>] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
CPU: 0 PID: 7017 Comm: syz-executor.3 Not tainted 6.9.0-syzkaller-12111-gfbe3e8473f39 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:native_safe_halt arch/x86/include/asm/irqflags.h:48 [inline]
RIP: 0010:arch_safe_halt arch/x86/include/asm/irqflags.h:86 [inline]
RIP: 0010:kvm_wait+0x250/0x2c0 arch/x86/kernel/kvm.c:1060
Code: 3b 45 0f b6 f6 44 89 ff 44 89 f6 e8 da dd 55 00 e8 35 4a 5d 00 45 38 f7 75 15 66 90 e8 49 dd 55 00 0f 00 2d 62 e5 87 0a fb f4 <e9> 50 fe ff ff e8 36 dd 55 00 fb e9 45 fe ff ff 89 d9 80 e1 07 38
RSP: 0018:ffffc9001382f860 EFLAGS: 00000246
RAX: ffffffff81403737 RBX: ffff88801d45d4c8 RCX: 0000000000040000
RDX: ffffc900094ea000 RSI: 000000000003ffff RDI: 0000000000040000
RBP: ffffc9001382f930 R08: ffffffff92fa65bf R09: 1ffffffff25f4cb7
R10: dffffc0000000000 R11: fffffbfff25f4cb8 R12: 1ffff92002705f10
R13: dffffc0000000000 R14: 0000000000000003 R15: 0000000000000003
FS:  00007fe37ee546c0(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000140 CR3: 000000007ca1e000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 pv_wait arch/x86/include/asm/paravirt.h:596 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:466 [inline]
 __pv_queued_spin_lock_slowpath+0x8d0/0xdc0 kernel/locking/qspinlock.c:508
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:584 [inline]
 queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x272/0x370 kernel/locking/spinlock_debug.c:116
 spin_lock include/linux/spinlock.h:351 [inline]
 d_instantiate+0x64/0xa0 fs/dcache.c:1879
 alloc_path_pseudo fs/file_table.c:334 [inline]
 alloc_file_pseudo+0x19e/0x290 fs/file_table.c:346
 __anon_inode_getfile fs/anon_inodes.c:109 [inline]
 anon_inode_getfile+0xc8/0x180 fs/anon_inodes.c:147
 __do_sys_perf_event_open kernel/events/core.c:12719 [inline]
 __se_sys_perf_event_open+0x25e5/0x38d0 kernel/events/core.c:12431
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fe37e07cee9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe37ee540c8 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007fe37e1b3fa0 RCX: 00007fe37e07cee9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000020000140
RBP: 00007fe37e0c947f R08: 0000000000000000 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fe37e1b3fa0 R15: 00007fffd62204b8
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 6999 Comm: syz-executor.1 Not tainted 6.9.0-syzkaller-12111-gfbe3e8473f39 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:write_comp_data kernel/kcov.c:236 [inline]
RIP: 0010:__sanitizer_cov_trace_cmp4+0x2d/0x90 kernel/kcov.c:278
Code: fa 4c 8b 04 24 65 48 8b 14 25 00 d5 03 00 65 8b 05 b0 be 6d 7e a9 00 01 ff 00 74 10 a9 00 01 00 00 74 5b 83 ba 1c 16 00 00 00 <74> 52 8b 82 f8 15 00 00 83 f8 03 75 47 48 8b 8a 00 16 00 00 44 8b
RSP: 0018:ffffc90000a18998 EFLAGS: 00000046
RAX: 0000000000000104 RBX: 0000000000000001 RCX: ffffffff9479e603
RDX: ffff888024591e00 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffff8880b953f6f0 R08: ffffffff81a473c3 R09: 1ffffffff1f5a6bd
R10: dffffc0000000000 R11: fffffbfff1f5a6be R12: 0000000000000000
R13: 0000000000000000 R14: ffff8880b953f600 R15: 0000000000000001
FS:  0000555581912480(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b32d3b000 CR3: 000000005e7de000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 irq_work_queue_on+0x133/0x260 kernel/irq_work.c:153
 rcu_read_unlock_special+0x3cf/0x550 kernel/rcu/tree_plugin.h:675
 __rcu_read_unlock+0xa1/0x110 kernel/rcu/tree_plugin.h:426
 rcu_read_unlock include/linux/rcupdate.h:814 [inline]
 trace_call_bpf+0x607/0x8a0 kernel/trace/bpf_trace.c:149
 perf_trace_run_bpf_submit+0x7c/0x1d0 kernel/events/core.c:10269
 perf_trace_preemptirq_template+0x2d2/0x3f0 include/trace/events/preemptirq.h:14
 trace_irq_enable+0xf1/0x120 include/trace/events/preemptirq.h:40
 trace_hardirqs_on+0x18/0x40 kernel/trace/trace_preemptirq.c:56
 irqentry_exit+0x63/0x90 kernel/entry/common.c:357
 asm_sysvec_irq_work+0x1a/0x20 arch/x86/include/asm/idtentry.h:738
RIP: 0010:variable_ffs arch/x86/include/asm/bitops.h:321 [inline]
RIP: 0010:handle_softirqs+0x1df/0x970 kernel/softirq.c:542
Code: 89 7c 24 70 0f b7 db 48 c7 c7 40 ad c9 8b e8 48 14 2c 0a 65 66 c7 05 96 e3 a9 7e 00 00 e8 c9 8f 43 00 fb 49 c7 c4 c0 a0 00 8e <b8> ff ff ff ff 0f bc c3 41 89 c7 41 ff c7 0f 84 e6 03 00 00 89 5c
RSP: 0018:ffffc90000a18e40 EFLAGS: 00000286
RAX: 7a6ce7dd3ef43300 RBX: 0000000000000200 RCX: ffffffff8172c72a
RDX: dffffc0000000000 RSI: ffffffff8bcab880 RDI: ffffffff8c1fe200
RBP: ffffc90000a18f50 R08: ffffffff92fa65bf R09: 1ffffffff25f4cb7
R10: dffffc0000000000 R11: fffffbfff25f4cb8 R12: ffffffff8e00a0c0
R13: 0000000000000000 R14: dffffc0000000000 R15: 1ffff920001431dc
 __do_softirq kernel/softirq.c:588 [inline]
 invoke_softirq kernel/softirq.c:428 [inline]
 __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:649
 instr_sysvec_irq_work arch/x86/kernel/irq_work.c:17 [inline]
 sysvec_irq_work+0xa3/0xc0 arch/x86/kernel/irq_work.c:17
 </IRQ>
 <TASK>
 asm_sysvec_irq_work+0x1a/0x20 arch/x86/include/asm/idtentry.h:738
RIP: 0010:rcu_read_unlock_special+0x88/0x550 kernel/rcu/tree_plugin.h:682
Code: f1 f1 f1 00 f2 f2 f2 49 89 04 17 66 41 c7 44 17 09 f3 f3 41 c6 44 17 0b f3 65 44 8b 25 f9 b5 86 7e 41 f7 c4 00 00 f0 00 74 49 <48> c7 44 24 20 0e 36 e0 45 4a c7 04 3a 00 00 00 00 66 42 c7 44 3a
RSP: 0018:ffffc900146ffd20 EFLAGS: 00000206
RAX: 7a6ce7dd3ef43300 RBX: 1ffff920028dffac RCX: ffffffff8172c72a
RDX: dffffc0000000000 RSI: ffffffff8bcab880 RDI: ffffffff8c1fe200
RBP: ffffc900146ffdf0 R08: ffffffff92fa65bf R09: 1ffffffff25f4cb7
R10: dffffc0000000000 R11: fffffbfff25f4cb8 R12: 0000000000000000
R13: ffff888024592260 R14: ffffc900146ffd60 R15: 1ffff920028dffa8
 __rcu_read_unlock+0xa1/0x110 kernel/rcu/tree_plugin.h:426
 dput+0x197/0x2b0 fs/dcache.c:844
 __fput+0x68c/0x8b0 fs/file_table.c:430
 __do_sys_close fs/open.c:1555 [inline]
 __se_sys_close fs/open.c:1540 [inline]
 __x64_sys_close+0x7f/0x110 fs/open.c:1540
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fbda5c7bdda
Code: 48 3d 00 f0 ff ff 77 48 c3 0f 1f 80 00 00 00 00 48 83 ec 18 89 7c 24 0c e8 03 7f 02 00 8b 7c 24 0c 89 c2 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 36 89 d7 89 44 24 0c e8 63 7f 02 00 8b 44 24
RSP: 002b:00007fff20f9ee50 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000006 RCX: 00007fbda5c7bdda
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000005
RBP: 00007fbda5db59a0 R08: 0000001b33320000 R09: 0000000000000003
R10: 0000000000000353 R11: 0000000000000293 R12: 000000000001b481
R13: 00007fbda5db3fac R14: 0000000000000032 R15: 00007fbda5db59a0
 </TASK>

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/31 01:52 bpf-next fbe3e8473f39 34889ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in sys_perf_event_open
* Struck through repros no longer work on HEAD.