INFO: rcu detected stall in pcpu_balance_workfn

Status: auto-obsoleted due to no activity on 2025/02/20 11:26
Subsystems: kernel
[Documentation on labels]
First crash: 630d, last: 630d
▶ ▼ Similar bugs (3)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in pcpu_balance_workfn (2) mm 1 1 391d 391d 0/29 auto-obsoleted due to no activity on 2025/10/17 19:49
linux-6.1 INFO: rcu detected stall in pcpu_balance_workfn 1 1 183d 183d 0/3 auto-obsoleted due to no activity on 2026/05/23 10:37
android-6-1 BUG: soft lockup in pcpu_balance_workfn 1 1 396d 396d 0/2 auto-obsoleted due to no activity on 2025/10/12 11:08

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=0eec/1/0x4000000000000000 softirq=117455/117456 fqs=2
rcu: 	(detected by 0, t=10502 jiffies, g=110305, q=135 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 8990 Comm: kworker/1:0 Not tainted 6.12.0-syzkaller-07749-g28eb75e178d3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/30/2024
Workqueue: events pcpu_balance_workfn
RIP: 0010:rb_erase+0x268/0x1360 lib/rbtree.c:446
Code: 3c 01 00 0f 85 50 10 00 00 48 b8 00 00 00 00 00 fc ff df 48 89 2a 48 89 da 48 c1 ea 03 80 3c 02 00 0f 85 47 10 00 00 4c 89 23 <48> 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f c3 cc cc cc cc 48 b8 00
RSP: 0018:ffffc90000a18d80 EFLAGS: 00000046
RAX: dffffc0000000000 RBX: 0000000000000001 RCX: ffff8880b872cb50
RDX: 1ffff110170e596a RSI: ffff8880b872cb50 RDI: ffff88805b4de350
RBP: ffff8880b872cb50 R08: 0000000000000006 R09: 0000000000000001
R10: ffff88805b4de340 R11: 0000000000000004 R12: ffff8880b872cb58
R13: ffff88805b4de340 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd82aec7f98 CR3: 000000005b648000 CR4: 00000000003526f0
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 rb_erase_cached include/linux/rbtree.h:126 [inline]
 timerqueue_del+0x83/0x150 lib/timerqueue.c:57
 __remove_hrtimer+0x99/0x290 kernel/time/hrtimer.c:1109
 __run_hrtimer kernel/time/hrtimer.c:1719 [inline]
 __hrtimer_run_queues+0x50b/0xae0 kernel/time/hrtimer.c:1803
 hrtimer_interrupt+0x392/0x8e0 kernel/time/hrtimer.c:1865
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1038 [inline]
 __sysvec_apic_timer_interrupt+0x10f/0x400 arch/x86/kernel/apic/apic.c:1055
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1049 [inline]
 sysvec_apic_timer_interrupt+0x9f/0xc0 arch/x86/kernel/apic/apic.c:1049
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0x31/0x80 kernel/locking/spinlock.c:194
Code: f5 53 48 8b 74 24 10 48 89 fb 48 83 c7 18 e8 06 3a 44 f6 48 89 df e8 7e b9 44 f6 f7 c5 00 02 00 00 75 23 9c 58 f6 c4 02 75 37 <bf> 01 00 00 00 e8 15 b0 35 f6 65 8b 05 e6 28 dd 74 85 c0 74 16 5b
RSP: 0018:ffffc9000d0979b8 EFLAGS: 00000246
RAX: 0000000000000006 RBX: ffffffff9aa554a0 RCX: 1ffffffff2dc8442
RDX: 0000000000000000 RSI: ffffffff8b6cdaa0 RDI: ffffffff8bd0dc80
RBP: 0000000000000286 R08: 0000000000000001 R09: fffffbfff2dc7f98
R10: ffffffff96e3fcc7 R11: 0000000000000003 R12: ffffe8ffff400000
R13: 00000000000001bb R14: 0000000000000004 R15: ffffe8ffff400000
 __debug_check_no_obj_freed lib/debugobjects.c:1108 [inline]
 debug_check_no_obj_freed+0x327/0x600 lib/debugobjects.c:1129
 remove_vm_area+0x1ae/0x3f0 mm/vmalloc.c:3220
 free_vm_area mm/vmalloc.c:4510 [inline]
 pcpu_free_vm_areas+0x91/0xd0 mm/vmalloc.c:4859
 pcpu_destroy_chunk mm/percpu-vm.c:367 [inline]
 pcpu_balance_free+0x5d9/0xb20 mm/percpu.c:1989
 pcpu_balance_workfn+0x9d4/0xd00 mm/percpu.c:2213
 process_one_work+0x9c5/0x1ba0 kernel/workqueue.c:3229
 process_scheduled_works kernel/workqueue.c:3310 [inline]
 worker_thread+0x6c8/0xf00 kernel/workqueue.c:3391
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: rcu_preempt kthread starved for 10498 jiffies! g110305 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:27616 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5369 [inline]
 __schedule+0xe5a/0x5ae0 kernel/sched/core.c:6756
 __schedule_loop kernel/sched/core.c:6833 [inline]
 schedule+0xe7/0x350 kernel/sched/core.c:6848
 schedule_timeout+0x124/0x280 kernel/time/sleep_timeout.c:99
 rcu_gp_fqs_loop+0x1eb/0xb00 kernel/rcu/tree.c:2045
 rcu_gp_kthread+0x271/0x380 kernel/rcu/tree.c:2247
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 UID: 0 PID: 6555 Comm: kworker/u8:10 Not tainted 6.12.0-syzkaller-07749-g28eb75e178d3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/30/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:__sanitizer_cov_trace_pc+0x41/0x70 kernel/kcov.c:217
Code: 00 48 8b 34 24 74 1d f6 c4 01 74 43 a9 00 00 0f 00 75 3c a9 00 00 f0 00 75 35 8b 82 1c 16 00 00 85 c0 74 2b 8b 82 f8 15 00 00 <83> f8 02 75 20 48 8b 8a 00 16 00 00 8b 92 fc 15 00 00 48 8b 01 48
RSP: 0018:ffffc9000c51f990 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8880b8744bc0 RCX: ffffffff8182221c
RDX: ffff888027e9a440 RSI: ffffffff818221f6 RDI: 0000000000000005
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000006 R12: ffffed10170e8979
R13: 0000000000000001 R14: ffff8880b8744bc8 R15: ffff8880b863ffc0
FS:  0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b3041eff8 CR3: 000000000df7e000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 rep_nop arch/x86/include/asm/vdso/processor.h:13 [inline]
 cpu_relax arch/x86/include/asm/vdso/processor.h:18 [inline]
 csd_lock_wait kernel/smp.c:340 [inline]
 smp_call_function_many_cond+0x456/0x1300 kernel/smp.c:884
 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1051
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2085 [inline]
 text_poke_bp_batch+0x22b/0x760 arch/x86/kernel/alternative.c:2295
 text_poke_flush arch/x86/kernel/alternative.c:2486 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:2483 [inline]
 text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2493
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x1d7/0x400 kernel/jump_label.c:920
 static_key_enable_cpuslocked+0x1b7/0x270 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 toggle_allocation_gate mm/kfence/core.c:849 [inline]
 toggle_allocation_gate+0xfc/0x260 mm/kfence/core.c:841
 process_one_work+0x9c5/0x1ba0 kernel/workqueue.c:3229
 process_scheduled_works kernel/workqueue.c:3310 [inline]
 worker_thread+0x6c8/0xf00 kernel/workqueue.c:3391
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/22 11:19 upstream 28eb75e178d3 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in pcpu_balance_workfn
* Struck through repros no longer work on HEAD.