syzbot


INFO: rcu detected stall in syscall_enter_from_user_mode (3)

Status: upstream: reported C repro on 2025/02/27 13:22
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+d27661ce10e475e27efe@syzkaller.appspotmail.com
First crash: 302d, last: 28d
Fix commit to backport (bisect log) :
tree: upstream
commit f4bf3ca2e5cba655824b6e0893a98dfb33ed24e5
Author: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Date: Fri Apr 7 23:05:26 2023 +0000

  softirq: Add trace points for tasklet entry/exit

  
Fix bisection: the issue occurs on the latest tested release (bisect log)
Crash: no output from test machine (log)
Repro: syz .config
  
Bug presence (2)
Date Name Commit Repro Result
2025/08/27 linux-6.1.y (ToT) 0bc96de781b4 C [report] INFO: rcu detected stall in perf_event_open
2025/08/27 upstream (ToT) 39f90c196721 C Didn't crash
Similar bugs (8)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in syscall_enter_from_user_mode (2) 1 1 509d 509d 0/3 auto-obsoleted due to no activity on 2024/11/13 07:03
linux-6.1 INFO: rcu detected stall in syscall_enter_from_user_mode 1 1 906d 906d 0/3 auto-obsoleted due to no activity on 2023/10/13 06:46
upstream INFO: rcu detected stall in syscall_enter_from_user_mode (2) kernel 1 11 1081d 1370d 0/29 auto-obsoleted due to no activity on 2023/04/22 11:15
upstream INFO: rcu detected stall in syscall_enter_from_user_mode (3) kernel 1 5 745d 911d 0/29 auto-obsoleted due to no activity on 2024/03/11 14:48
linux-5.15 INFO: rcu detected stall in syscall_enter_from_user_mode missing-backport origin:upstream 1 C done 123 12d 745d 0/3 upstream: reported C repro on 2023/12/13 03:22
linux-6.6 INFO: rcu detected stall in syscall_enter_from_user_mode 1 1 45d 45d 0/2 upstream: reported on 2025/11/11 17:17
upstream INFO: rcu detected stall in syscall_enter_from_user_mode kernel 1 2 1428d 1441d 0/29 closed as invalid on 2022/02/08 10:10
android-5-15 BUG: soft lockup in syscall_enter_from_user_mode 1 1 589d 589d 0/2 auto-obsoleted due to no activity on 2024/08/14 14:14
Fix bisection attempts (4)
Created Duration User Patch Repo Result
2025/11/13 01:20 11h02m fix candidate upstream OK (1) job log
2025/06/27 04:15 2h09m fix candidate upstream OK (0) job log
2025/06/06 18:47 2h16m bisect fix linux-6.1.y OK (0) job log log
2025/04/07 14:34 2h31m bisect fix linux-6.1.y OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (1 GPs behind) idle=385c/1/0x4000000000000000 softirq=12313/12348 fqs=2
	(t=10500 jiffies g=12481 q=124 ncpus=2)
rcu: rcu_preempt kthread starved for 10496 jiffies! g12481 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:27840 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5244 [inline]
 __schedule+0x10ec/0x40b0 kernel/sched/core.c:6561
 schedule+0xb9/0x180 kernel/sched/core.c:6637
 schedule_timeout+0x15c/0x280 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x2f2/0x1310 kernel/rcu/tree.c:1706
 rcu_gp_kthread+0x95/0x380 kernel/rcu/tree.c:1905
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 4915 Comm: syz.0.17 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025
RIP: 0010:unwind_next_frame+0x688/0x20b0 arch/x86/kernel/unwind_orc.c:475
Code: 49 0f bf 04 24 48 01 c5 e9 fa 00 00 00 4c 8d 7b 40 4c 89 f8 48 c1 e8 03 42 80 3c 28 00 74 08 4c 89 ff e8 2b 9a 98 00 49 8b 17 <4d> 8d 7c 24 01 4c 89 e0 48 c1 e8 03 42 0f b6 04 28 84 c0 0f 85 cc
RSP: 0018:ffffc90000006308 EFLAGS: 00000046
RAX: 1ffff92000000c90 RBX: ffffc90000006440 RCX: ffffffff8e7f8040
RDX: ffffc90000007750 RSI: ffffffff8e7f803c RDI: 0000000000000001
RBP: ffffffff8f177c00 R08: 0000000000000001 R09: ffffc90000006490
R10: fffff52000000c94 R11: 1ffff92000000c92 R12: ffffffff8f177bfc
R13: dffffc0000000000 R14: ffffffff8e7f803c R15: ffffc90000006480
FS:  000055557841e500(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000029236000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 perf_callchain_kernel+0x461/0x5c0 arch/x86/events/core.c:2805
 get_perf_callchain+0x376/0x480 kernel/events/callchain.c:200
 perf_callchain kernel/events/core.c:7512 [inline]
 perf_prepare_sample+0x399/0x2040 kernel/events/core.c:7545
 __perf_event_output kernel/events/core.c:7730 [inline]
 perf_event_output_forward+0x163/0x2a0 kernel/events/core.c:7750
 __perf_event_overflow+0x448/0x610 kernel/events/core.c:9496
 perf_tp_event+0x335/0xb60 kernel/events/core.c:10029
 perf_trace_run_bpf_submit+0xf3/0x1c0 kernel/events/core.c:10002
 perf_trace_preemptirq_template+0x287/0x330 include/trace/events/preemptirq.h:14
 trace_irq_disable include/trace/events/preemptirq.h:36 [inline]
 trace_hardirqs_off_finish+0x13f/0x180 kernel/trace/trace_preemptirq.c:67
 irqentry_enter+0x33/0x50 kernel/entry/common.c:382
 sysvec_irq_work+0xa/0xc0 arch/x86/kernel/irq_work.c:17
 asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:728
RIP: 0010:seqcount_lockdep_reader_access+0x17b/0x1c0 include/linux/seqlock.h:105
Code: 00 4d 85 e4 75 16 e8 04 8f 0f 00 eb 15 e8 fd 8e 0f 00 e8 98 8a a6 08 4d 85 e4 74 ea e8 ee 8e 0f 00 fb 48 c7 04 24 0e 36 e0 45 <4b> c7 04 3e 00 00 00 00 66 43 c7 44 3e 09 00 00 43 c6 44 3e 0b 00
RSP: 0018:ffffc90000006d40 EFLAGS: 00000246
RAX: ffffffff81714772 RBX: 0000000000000000 RCX: ffff8880245c3b80
RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90000006df8 R08: dffffc0000000000 R09: fffffbfff215c468
R10: fffffbfff215c468 R11: 1ffffffff215c467 R12: 0000000000000200
R13: 000001fb63109b96 R14: 1ffff92000000da8 R15: dffffc0000000000
 timekeeping_get_delta kernel/time/timekeeping.c:254 [inline]
 timekeeping_get_ns kernel/time/timekeeping.c:388 [inline]
 ktime_get_with_offset+0xff/0x320 kernel/time/timekeeping.c:891
 ktime_get_real include/linux/timekeeping.h:79 [inline]
 netif_rx_internal+0x398/0x560 net/core/dev.c:5023
 __netif_rx+0x74/0xb0 net/core/dev.c:5066
 veth_forward_skb drivers/net/veth.c:290 [inline]
 veth_xmit+0x650/0xb00 drivers/net/veth.c:343
 __netdev_start_xmit include/linux/netdevice.h:4896 [inline]
 netdev_start_xmit include/linux/netdevice.h:4910 [inline]
 xmit_one net/core/dev.c:3670 [inline]
 dev_hard_start_xmit+0x262/0x870 net/core/dev.c:3686
 __dev_queue_xmit+0x1bf1/0x3760 net/core/dev.c:4346
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 hsr_xmit net/hsr/hsr_forward.c:380 [inline]
 hsr_forward_do net/hsr/hsr_forward.c:471 [inline]
 hsr_forward_skb+0x12f4/0x20d0 net/hsr/hsr_forward.c:624
 hsr_dev_xmit+0x1e3/0x340 net/hsr/hsr_device.c:238
 __netdev_start_xmit include/linux/netdevice.h:4896 [inline]
 netdev_start_xmit include/linux/netdevice.h:4910 [inline]
 xmit_one net/core/dev.c:3670 [inline]
 dev_hard_start_xmit+0x262/0x870 net/core/dev.c:3686
 __dev_queue_xmit+0x1bf1/0x3760 net/core/dev.c:4346
 neigh_output include/net/neighbour.h:545 [inline]
 ip6_finish_output2+0xda2/0x1590 net/ipv6/ip6_output.c:138
 dst_output include/net/dst.h:453 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 ndisc_send_skb+0xbfb/0x1510 net/ipv6/ndisc.c:513
 addrconf_rs_timer+0x2b8/0x600 net/ipv6/addrconf.c:3979
 call_timer_fn+0x1a0/0x670 kernel/time/timer.c:1700
 expire_timers kernel/time/timer.c:1751 [inline]
 __run_timers+0x529/0x7d0 kernel/time/timer.c:2022
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:2035
 handle_softirqs+0x2a1/0x920 kernel/softirq.c:596
 __do_softirq kernel/softirq.c:630 [inline]
 invoke_softirq kernel/softirq.c:470 [inline]
 __irq_exit_rcu+0x12f/0x220 kernel/softirq.c:679
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:691
 instr_sysvec_irq_work arch/x86/kernel/irq_work.c:17 [inline]
 sysvec_irq_work+0x9d/0xc0 arch/x86/kernel/irq_work.c:17
 </IRQ>
 <TASK>
 asm_sysvec_irq_work+0x16/0x20 arch/x86/include/asm/idtentry.h:728
RIP: 0010:preempt_schedule_irq+0xac/0x150 kernel/sched/core.c:6873
Code: 44 24 20 f6 44 24 21 02 74 0b 0f 0b 48 f7 03 08 00 00 00 74 70 bf 01 00 00 00 e8 0f 42 37 f7 e8 4a be 66 f7 fb bf 01 00 00 00 <e8> df b9 ff ff 48 c7 44 24 40 00 00 00 00 9c 8f 44 24 40 8b 44 24
RSP: 0018:ffffc90003c67da0 EFLAGS: 00000286
RAX: f28bc581c6172800 RBX: 0000000000000000 RCX: f28bc581c6172800
RDX: dffffc0000000000 RSI: ffffffff8a8c0580 RDI: 0000000000000001
RBP: ffffc90003c67e40 R08: ffffffff8f9f7008 R09: ffffffff8f9f7018
R10: ffffffff8f9f7010 R11: ffffffff8f9f7003 R12: 0000000000000000
R13: 0000000000000000 R14: dffffc0000000000 R15: 1ffff9200078cfb4
 irqentry_exit+0x63/0x70 kernel/entry/common.c:439
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:__syscall_enter_from_user_work kernel/entry/common.c:97 [inline]
RIP: 0010:syscall_enter_from_user_mode+0x2a/0x80 kernel/entry/common.c:118
Code: 41 56 53 48 89 f3 49 89 fe 48 8b 7c 24 10 e8 ed fa ff ff 66 90 66 90 e8 74 5b 6f f7 e8 ef 59 6f f7 fb 65 48 8b 05 b6 96 ea 75 <48> 8b 70 08 40 f6 c6 3f 74 0b 4c 89 f7 5b 41 5e e9 21 03 56 f7 48
RSP: 0018:ffffc90003c67f08 EFLAGS: 00000286
RAX: ffff8880245c3b80 RBX: 00000000000000ca RCX: f28bc581c6172800
RDX: dffffc0000000000 RSI: ffffffff8a8c0580 RDI: ffffffff8adef760
RBP: ffffc90003c67f48 R08: dffffc0000000000 R09: fffffbfff1c3ed56
R10: fffffbfff1c3ed56 R11: 1ffffffff1c3ed55 R12: 0000000000000000
R13: 0000000000000000 R14: ffffc90003c67f58 R15: 0000000000000000
 do_syscall_64+0x24/0xa0 arch/x86/entry/common.c:77
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7fac1478f749
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd08748688 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: ffffffffffffffda RBX: 00007fac149e5fa8 RCX: 00007fac1478f749
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00007fac149e5fac
RBP: 0000000000000000 R08: 001520b2f3500e44 R09: 000000050874897f
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fac149e5fac
R13: 00007fac149e5fa0 R14: 0000000000000848 R15: 0000000000000003
 </TASK>
CPU: 1 PID: 4792 Comm: syz.3.20 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/02/2025
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa5/0x100 kernel/locking/spinlock.c:194
Code: 74 05 e8 ce 48 65 f7 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4b f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> f6 cd 35 f7 65 8b 05 77 85 e0 75 85 c0 74 3c 48 c7 04 24 0e 36
RSP: 0018:ffffc900001e05c0 EFLAGS: 00000206
RAX: 27d144ae9c4f9a00 RBX: 0000000000000a06 RCX: 27d144ae9c4f9a00
RDX: dffffc0000000000 RSI: ffffffff8a8c0580 RDI: 0000000000000001
RBP: ffffc900001e0650 R08: dffffc0000000000 R09: fffffbfff215c46a
R10: fffffbfff215c46a R11: 1ffffffff215c469 R12: dffffc0000000000
R13: 1ffff9200003c0d0 R14: ffff8880753705b8 R15: 1ffff9200003c0b8
FS:  000055558683e500(0000) GS:ffff8880b8f00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2ed63fff CR3: 000000002fa04000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 spin_unlock_irqrestore include/linux/spinlock.h:406 [inline]
 ref_tracker_alloc+0x2a4/0x450 lib/ref_tracker.c:96
 __netdev_tracker_alloc include/linux/netdevice.h:4051 [inline]
 netdev_hold include/linux/netdevice.h:4080 [inline]
 dst_init+0xda/0x410 net/core/dst.c:52
 dst_alloc+0x12a/0x160 net/core/dst.c:92
 ip6_dst_alloc net/ipv6/route.c:345 [inline]
 icmp6_dst_alloc+0x7f/0x460 net/ipv6/route.c:3308
 ndisc_send_skb+0x3ff/0x1510 net/ipv6/ndisc.c:493
 addrconf_rs_timer+0x2b8/0x600 net/ipv6/addrconf.c:3979
 call_timer_fn+0x1a0/0x670 kernel/time/timer.c:1700
 expire_timers kernel/time/timer.c:1751 [inline]
 __run_timers+0x529/0x7d0 kernel/time/timer.c:2022
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:2035
 handle_softirqs+0x2a1/0x920 kernel/softirq.c:596
 __do_softirq kernel/softirq.c:630 [inline]
 invoke_softirq kernel/softirq.c:470 [inline]
 __irq_exit_rcu+0x12f/0x220 kernel/softirq.c:679
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:691
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1118 [inline]
 sysvec_apic_timer_interrupt+0xa0/0xc0 arch/x86/kernel/apic/apic.c:1118
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:finish_task_switch+0x26a/0x8f0 kernel/sched/core.c:5123
Code: 0f 84 3b 01 00 00 48 85 db 0f 85 5a 01 00 00 0f 1f 44 00 00 4c 8b 75 d0 4c 89 e7 e8 e0 6d c9 08 e8 6b b7 2e 00 fb 4c 8b 65 c0 <49> 8d bc 24 f8 15 00 00 48 89 f8 48 c1 e8 03 42 0f b6 04 28 84 c0
RSP: 0018:ffffc90003527598 EFLAGS: 00000282
RAX: 27d144ae9c4f9a00 RBX: 0000000000000000 RCX: 27d144ae9c4f9a00
RDX: dffffc0000000000 RSI: ffffffff8a8c0580 RDI: ffffffff8adef760
RBP: ffffc900035275f0 R08: dffffc0000000000 R09: fffffbfff215c44f
R10: fffffbfff215c44f R11: 1ffffffff215c44e R12: ffff88802a3ed940
R13: dffffc0000000000 R14: ffff8880797c1dc0 R15: ffff8880b8f3b590
 context_switch kernel/sched/core.c:5247 [inline]
 __schedule+0x10f4/0x40b0 kernel/sched/core.c:6561
 preempt_schedule_common+0x82/0xd0 kernel/sched/core.c:6730
 preempt_schedule+0xa7/0xb0 kernel/sched/core.c:6754
 preempt_schedule_thunk+0x16/0x18 arch/x86/entry/thunk_64.S:34
 __down_read_common kernel/locking/rwsem.c:1268 [inline]
 __down_read kernel/locking/rwsem.c:1274 [inline]
 down_read+0x1d8/0x2d0 kernel/locking/rwsem.c:1522
 mmap_read_lock include/linux/mmap_lock.h:117 [inline]
 acct_collect+0x154/0x730 kernel/acct.c:598
 do_exit+0x845/0x2400 kernel/exit.c:850
 do_group_exit+0x217/0x2d0 kernel/exit.c:1022
 get_signal+0x1272/0x1350 kernel/signal.c:2871
 arch_do_signal_or_restart+0xb7/0x1240 arch/x86/kernel/signal.c:871
 exit_to_user_mode_loop+0x70/0x110 kernel/entry/common.c:174
 exit_to_user_mode_prepare+0xee/0x180 kernel/entry/common.c:210
 irqentry_exit_to_user_mode+0x5/0x30 kernel/entry/common.c:316
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0033:0x7fc23478f749
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffc9bccc7f8 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 00007fc2349e5fa0 RCX: 00007fc23478f749
RDX: 00002000000006c0 RSI: 0000000000005452 RDI: 0000000000000004
RBP: 00007fc234813f91 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007fc2349e5fa0 R14: 00007fc2349e5fa0 R15: 0000000000000003
 </TASK>

Crashes (8):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/11/29 05:36 linux-6.1.y f6e38ae624cf d6526ea3 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/08/20 21:50 linux-6.1.y 0bc96de781b4 0b9605c8 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/10/16 18:59 linux-6.1.y c2fda4b3f577 19568248 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/02/27 13:21 linux-6.1.y 3a8358583626 6a8fcbc4 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/11/15 11:47 linux-6.1.y f6e38ae624cf f7988ea4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/11/11 14:28 linux-6.1.y f6e38ae624cf 4e1406b4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/09/21 21:32 linux-6.1.y 363a599da6d9 67c37560 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in syscall_enter_from_user_mode
2025/08/15 07:02 linux-6.1.y 3594f306da12 1804e95e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in syscall_enter_from_user_mode
* Struck through repros no longer work on HEAD.