syzbot


INFO: rcu detected stall in sys_perf_event_open (2)

Status: upstream: reported C repro on 2024/09/23 05:55
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+5284a86a0b0a31ab266a@syzkaller.appspotmail.com
First crash: 383d, last: 5d13h
Fix commit to backport (bisect log) :
tree: upstream
commit f4bf3ca2e5cba655824b6e0893a98dfb33ed24e5
Author: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Date: Fri Apr 7 23:05:26 2023 +0000

  softirq: Add trace points for tasklet entry/exit

  
Discussions (2)
Title Replies (including bot) Last reply
Re: [PATCH 6.1] softirq: Add trace points for tasklet entry/exit 3 (3) 2025/08/21 22:05
[PATCH 6.1] softirq: Add trace points for tasklet entry/exit 1 (1) 2025/08/21 18:40
Bug presence (2)
Date Name Commit Repro Result
2025/09/18 linux-6.1.y (ToT) 3db754f56897 C [report] INFO: rcu detected stall in smp_call_function
2025/09/18 upstream (ToT) 86cc796e5e9b C Didn't crash
Similar bugs (15)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in sys_perf_event_open (2) origin:lts-only 1 C error 20 5d15h 641d 0/3 upstream: reported C repro on 2024/01/09 18:19
upstream INFO: rcu detected stall in sys_perf_event_open (3) perf 1 1 2103d 2103d 0/29 closed as invalid on 2020/01/08 05:23
upstream INFO: rcu detected stall in sys_perf_event_open (4) perf 1 1 2103d 2103d 0/29 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in sys_perf_event_open (2) perf 1 8 2138d 2139d 0/29 closed as invalid on 2019/12/04 14:14
linux-6.1 INFO: rcu detected stall in sys_perf_event_open 1 1 487d 487d 0/3 auto-obsoleted due to no activity on 2024/09/19 01:45
linux-6.6 INFO: rcu detected stall in perf_event_open 1 C done 4 44d 58d 2/2 fixed on 2025/10/06 10:09
linux-5.15 INFO: rcu detected stall in sys_perf_event_open 1 1 906d 906d 0/3 auto-obsoleted due to no activity on 2023/08/17 04:33
upstream INFO: rcu detected stall in sys_perf_event_open perf 1 1 2583d 2581d 0/29 auto-closed as invalid on 2019/03/14 17:19
android-5-15 BUG: soft lockup in sys_perf_event_open 1 1 523d 523d 0/2 auto-obsoleted due to no activity on 2024/08/04 07:07
upstream BUG: soft lockup in sys_perf_event_open perf 1 1 498d 494d 0/29 auto-obsoleted due to no activity on 2024/08/29 01:52
upstream BUG: soft lockup in perf_event_open trace bpf 1 2 108d 104d 0/29 auto-obsoleted due to no activity on 2025/09/23 09:29
upstream BUG: soft lockup in sys_perf_event_open (2) kvm fs 1 6 266d 373d 0/29 auto-obsoleted due to no activity on 2025/04/18 04:31
upstream INFO: rcu detected stall in perf_event_open (2) perf 1 1 1475d 1475d 0/29 auto-closed as invalid on 2021/11/26 20:49
upstream INFO: rcu detected stall in perf_event_open kernel 1 1 1571d 1571d 0/29 auto-closed as invalid on 2021/09/21 01:14
upstream INFO: rcu detected stall in __se_sys_perf_event_open perf 1 1 1925d 1925d 0/29 auto-closed as invalid on 2020/10/02 02:08
Last patch testing requests (1)
Created Duration User Patch Repo Result
2025/08/12 15:47 2h08m sumanth.gavini@yahoo.com patch linux-6.1.y OK log
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/12/03 12:42 7h31m fix candidate upstream OK (1) job log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=a4ac/1/0x4000000000000002 softirq=13223/13224 fqs=2
	(detected by 1, t=10502 jiffies, g=14009, q=269 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5337 Comm: syz.1.369 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
RIP: 0010:lockdep_enabled kernel/locking/lockdep.c:116 [inline]
RIP: 0010:lock_release+0xe3/0x910 kernel/locking/lockdep.c:5675
Code: 1f 8e be 08 00 00 00 e8 fb 79 6d 00 4c 0f a3 35 4b fe bb 0c 73 0d e8 ec 99 07 00 84 c0 0f 84 29 05 00 00 48 c7 c0 24 a7 1f 8e <48> c1 e8 03 42 0f b6 04 28 84 c0 0f 85 31 05 00 00 83 3d d9 32 bc
RSP: 0018:ffffc90000006da0 EFLAGS: 00000002
RAX: ffffffff8e1fa724 RBX: 0000000000000000 RCX: 899b20c7e713cd00
RDX: 0000000000000000 RSI: ffffffff8adefac0 RDI: ffffffff8adefa80
RBP: ffffc90000006eb0 R08: dffffc0000000000 R09: fffffbfff1c3ee4e
R10: fffffbfff1c3ee4e R11: 1ffffffff1c3ee4d R12: dffffc0000000000
R13: dffffc0000000000 R14: 0000000000000000 R15: 1ffff92000000dc0
FS:  00007f9cced276c0(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007feee41fbd58 CR3: 0000000073933000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 rcu_lock_release include/linux/rcupdate.h:355 [inline]
 rcu_read_unlock_sched include/linux/rcupdate.h:904 [inline]
 pfn_valid include/linux/mmzone.h:1867 [inline]
 __virt_addr_valid+0x435/0x540 arch/x86/mm/physaddr.c:65
 kasan_addr_to_slab+0x9/0xc0 mm/kasan/common.c:35
 __kasan_record_aux_stack+0xf/0xc0 mm/kasan/generic.c:471
 irq_work_queue_on+0x10f/0x260 kernel/irq_work.c:140
 rcu_read_unlock_special+0x3c7/0x500 kernel/rcu/tree_plugin.h:678
 __rcu_read_unlock+0x78/0xd0 kernel/rcu/tree_plugin.h:426
 rcu_read_unlock include/linux/rcupdate.h:823 [inline]
 __perf_event_output kernel/events/core.c:7741 [inline]
 perf_event_output_forward+0x237/0x2a0 kernel/events/core.c:7750
 __perf_event_overflow+0x448/0x610 kernel/events/core.c:9496
 perf_swevent_overflow kernel/events/core.c:9572 [inline]
 perf_swevent_event+0x4ad/0x530 kernel/events/core.c:9610
 perf_tp_event+0x335/0xb60 kernel/events/core.c:10029
 perf_trace_run_bpf_submit+0xf3/0x1c0 kernel/events/core.c:10002
 perf_trace_preemptirq_template+0x287/0x330 include/trace/events/preemptirq.h:14
 trace_irq_enable_rcuidle+0xd3/0x140 include/trace/events/preemptirq.h:40
 trace_hardirqs_on+0x24/0x40 kernel/trace/trace_preemptirq.c:44
 asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:728
RIP: 0010:rcu_read_unlock_special+0x84/0x500 kernel/rcu/tree_plugin.h:685
Code: f1 f1 f1 00 f2 f2 f2 4a 89 04 2b 66 42 c7 44 2b 09 f3 f3 42 c6 44 2b 0b f3 65 44 8b 35 5d 97 96 7e 41 f7 c6 00 00 f0 00 74 45 <48> c7 44 24 40 0e 36 e0 45 4a c7 04 2b 00 00 00 00 66 42 c7 44 2b
RSP: 0018:ffffc90000007720 EFLAGS: 00000206
RAX: 899b20c7e713cd00 RBX: 1ffff92000000eec RCX: 899b20c7e713cd00
RDX: dffffc0000000000 RSI: ffffffff8a8c0300 RDI: ffffffff8adefae0
RBP: ffffc90000007818 R08: dffffc0000000000 R09: fffffbfff215c460
R10: fffffbfff215c460 R11: 1ffffffff215c45f R12: 0000000000000246
R13: dffffc0000000000 R14: ffff8880b8e3b900 R15: 0000000000000001
 __rcu_read_unlock+0x78/0xd0 kernel/rcu/tree_plugin.h:426
 rcu_read_unlock include/linux/rcupdate.h:823 [inline]
 __ip_queue_xmit+0x10b1/0x19b0 net/ipv4/ip_output.c:533
 __tcp_transmit_skb+0x1c6c/0x3160 net/ipv4/tcp_output.c:1405
 tcp_delack_timer_handler+0x2a9/0x440 net/ipv4/tcp_timer.c:322
 tcp_delack_timer+0x188/0x2e0 net/ipv4/tcp_timer.c:345
 call_timer_fn+0x1a0/0x670 kernel/time/timer.c:1504
 expire_timers kernel/time/timer.c:1549 [inline]
 __run_timers+0x525/0x7c0 kernel/time/timer.c:1820
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1833
 handle_softirqs+0x2a1/0x920 kernel/softirq.c:596
 __do_softirq kernel/softirq.c:630 [inline]
 invoke_softirq kernel/softirq.c:470 [inline]
 __irq_exit_rcu+0x12f/0x220 kernel/softirq.c:679
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:691
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1118 [inline]
 sysvec_apic_timer_interrupt+0xa0/0xc0 arch/x86/kernel/apic/apic.c:1118
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:fd_install+0x1a8/0x4e0 fs/file.c:609
Code: ef 03 41 80 3c 1f 00 74 08 48 89 ef e8 91 78 ea ff 45 89 ec 49 c1 e4 03 4c 8b 6d 00 4d 01 e5 4c 89 e8 48 c1 e8 03 80 3c 18 00 <74> 08 4c 89 ef e8 6e 78 ea ff 49 83 7d 00 00 0f 85 0a 03 00 00 41
RSP: 0018:ffffc90005ee7d58 EFLAGS: 00000246
RAX: 1ffff11005032024 RBX: dffffc0000000000 RCX: ffff88805a479dc0
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffff888026e04788 R08: dffffc0000000000 R09: fffffbfff215c449
R10: fffffbfff215c449 R11: 1ffffffff215c448 R12: 0000000000000120
R13: ffff888028190120 R14: ffffffff81e672bc R15: 1ffff11004dc08f1
 __do_sys_perf_event_open kernel/events/core.c:12807 [inline]
 __se_sys_perf_event_open+0x1c2f/0x1e60 kernel/events/core.c:12378
 do_syscall_x64 arch/x86/entry/common.c:51 [inline]
 do_syscall_64+0x4c/0xa0 arch/x86/entry/common.c:81
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f9ccdf8ebe9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f9cced27038 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007f9cce1c5fa0 RCX: 00007f9ccdf8ebe9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000200000000480
RBP: 00007f9cce011e19 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000000
R13: 00007f9cce1c6038 R14: 00007f9cce1c5fa0 R15: 00007ffd52ec9508
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10496 jiffies! g14009 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=1 timer-softirq=6821
rcu: rcu_preempt kthread starved for 10497 jiffies! g14009 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:27720 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5244 [inline]
 __schedule+0x10ec/0x40b0 kernel/sched/core.c:6561
 schedule+0xb9/0x180 kernel/sched/core.c:6637
 schedule_timeout+0x15c/0x280 kernel/time/timer.c:1965
 rcu_gp_fqs_loop+0x2f2/0x1310 kernel/rcu/tree.c:1706
 rcu_gp_kthread+0x95/0x380 kernel/rcu/tree.c:1905
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 4271 Comm: syz-executor Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/12/2025
RIP: 0010:native_safe_halt arch/x86/include/asm/irqflags.h:51 [inline]
RIP: 0010:arch_safe_halt arch/x86/include/asm/irqflags.h:89 [inline]
RIP: 0010:kvm_wait+0x141/0x190 arch/x86/kernel/kvm.c:1064
Code: 89 df 48 89 f8 48 c1 e8 03 42 0f b6 04 30 84 c0 75 47 0f b6 1f e8 df 3a 4f 00 44 38 e3 75 10 66 90 0f 00 2d d1 71 51 09 fb f4 <e9> 33 ff ff ff fb e9 2d ff ff ff e8 1f 61 df 08 89 f9 80 e1 07 38
RSP: 0018:ffffc90003ba78c0 EFLAGS: 00000246
RAX: c6eb9b5dcee2d800 RBX: 0000000000000003 RCX: c6eb9b5dcee2d800
RDX: dffffc0000000000 RSI: ffffffff8a8c0300 RDI: ffffffff8adefae0
RBP: ffffc90003ba7970 R08: dffffc0000000000 R09: fffffbfff215c460
R10: fffffbfff215c460 R11: 1ffffffff215c45f R12: 0000000000000003
R13: ffff8880b8f3b8d4 R14: dffffc0000000000 R15: 1ffff92000774f18
FS:  000055555b269500(0000) GS:ffff8880b8f00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4a677b0201 CR3: 000000002f71f000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 pv_wait arch/x86/include/asm/paravirt.h:603 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline]
 __pv_queued_spin_lock_slowpath+0x616/0x9c0 kernel/locking/qspinlock.c:511
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:591 [inline]
 queued_spin_lock_slowpath+0x43/0x50 arch/x86/include/asm/qspinlock.h:51
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x217/0x280 kernel/locking/spinlock_debug.c:115
 spin_lock_bh include/linux/spinlock.h:356 [inline]
 lock_sock_nested+0x66/0x100 net/core/sock.c:3491
 lock_sock include/net/sock.h:1756 [inline]
 tcp_sendmsg+0x1d/0x40 net/ipv4/tcp.c:1489
 sock_sendmsg_nosec net/socket.c:718 [inline]
 __sock_sendmsg net/socket.c:730 [inline]
 sock_write_iter+0x2b5/0x3a0 net/socket.c:1143
 call_write_iter include/linux/fs.h:2265 [inline]
 new_sync_write fs/read_write.c:491 [inline]
 vfs_write+0x44c/0x960 fs/read_write.c:584
 ksys_write+0x143/0x240 fs/read_write.c:637
 do_syscall_x64 arch/x86/entry/common.c:51 [inline]
 do_syscall_64+0x4c/0xa0 arch/x86/entry/common.c:81
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f8945b8d660
Code: 40 00 48 c7 c2 a8 ff ff ff f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 80 3d 01 af 20 00 00 74 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83 ec 28 48 89
RSP: 002b:00007fff96f46da8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00000000088601c0 RCX: 00007f8945b8d660
RDX: 00000000000000c8 RSI: 00007f893ec3ff38 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000007 R09: 000000000003fde8
R10: 3600a90ff5f76110 R11: 0000000000000202 R12: 00000000000000c8
R13: 000055555b27dcc0 R14: 00007fff96f47270 R15: 00007f893ec3ff38
 </TASK>

Crashes (17):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/09/09 04:26 linux-6.1.y 28c695c365e1 d291dd2d .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/02/28 09:23 linux-6.1.y 3a8358583626 6a8fcbc4 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in sys_perf_event_open
2024/11/22 03:43 linux-6.1.y b67dc5c9ade9 4b25d554 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in sys_perf_event_open
2025/10/06 09:44 linux-6.1.y 882efbdd9d34 49379ee0 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/27 10:08 linux-6.1.y 7b34dc04e4ff 001c9061 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/14 01:11 linux-6.1.y 3db754f56897 e2beed91 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/08/16 16:05 linux-6.1.y 0bc96de781b4 1804e95e .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/08/10 03:31 linux-6.1.y 3594f306da12 32a0e5ed .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/07/11 06:25 linux-6.1.y dfc486ec9cce 3cda49cf .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/01/12 02:40 linux-6.1.y c63962be84ef 6dbc6a9b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in sys_perf_event_open
2024/09/23 05:54 linux-6.1.y e526b12bf916 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in sys_perf_event_open
2025/10/03 15:05 linux-6.1.y 882efbdd9d34 49379ee0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/22 09:04 linux-6.1.y 363a599da6d9 67c37560 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/21 00:25 linux-6.1.y 363a599da6d9 67c37560 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/19 22:03 linux-6.1.y 363a599da6d9 67c37560 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/09/02 12:41 linux-6.1.y f89b6e15694c 807a3b61 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
2025/08/10 01:37 linux-6.1.y 3594f306da12 32a0e5ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in perf_event_open
* Struck through repros no longer work on HEAD.