syzbot


INFO: rcu detected stall in task_work_run

Status: auto-obsoleted due to no activity on 2023/11/07 12:15
Subsystems: kernel
[Documentation on labels]
First crash: 454d, last: 404d
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in task_work_run 1 174d 174d 0/3 auto-obsoleted due to no activity on 2024/07/04 00:01

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=e80c/1/0x4000000000000000 softirq=64244/64245 fqs=0
rcu: 	(detected by 0, t=10505 jiffies, g=93817, q=87 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 4468 Comm: udevd Not tainted 6.5.0-rc5-syzkaller-00053-g13b937206866 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
RIP: 0010:mark_lock+0x13f/0x1950 kernel/locking/lockdep.c:4663
Code: 01 00 00 48 69 d2 d0 00 00 00 48 81 c2 a0 42 84 91 48 8d 7a 60 48 b8 00 00 00 00 00 fc ff df 48 89 f9 48 c1 e9 03 80 3c 01 00 <0f> 85 0d 14 00 00 4c 89 e0 48 23 42 60 0f 84 d5 07 00 00 ba 01 00
RSP: 0018:ffffc900003e89a0 EFLAGS: 00000046
RAX: dffffc0000000000 RBX: ffff88807b2b4bc8 RCX: 1ffffffff231d8c0
RDX: ffffffff918ec5a0 RSI: 0000000000000008 RDI: ffffffff918ec600
RBP: ffffc900003e8b10 R08: 0000000000000000 R09: fffffbfff2308803
R10: ffffffff9184401f R11: 0000000000000000 R12: 0000000000000100
R13: 0000000000000008 R14: 1ffff9200007d13c R15: ffff88807b2b4be8
FS:  00007f9e636cac80(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f77386d81e9 CR3: 000000007bfcc000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 mark_usage kernel/locking/lockdep.c:4606 [inline]
 __lock_acquire+0x975/0x5de0 kernel/locking/lockdep.c:5098
 lock_acquire kernel/locking/lockdep.c:5761 [inline]
 lock_acquire+0x1ae/0x510 kernel/locking/lockdep.c:5726
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:351 [inline]
 advance_sched+0xd5/0xc60 net/sched/sch_taprio.c:935
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x203/0xc10 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x31b/0x800 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1098 [inline]
 __sysvec_apic_timer_interrupt+0x14a/0x430 arch/x86/kernel/apic/apic.c:1115
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1109
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:160 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x29/0x50 kernel/locking/spinlock.c:202
Code: 90 f3 0f 1e fa 53 48 8b 74 24 08 48 89 fb 48 83 c7 18 e8 6a a0 31 f7 48 89 df e8 42 1f 32 f7 e8 6d 4a 57 f7 fb bf 01 00 00 00 <e8> a2 c1 23 f7 65 8b 05 a3 38 cf 75 85 c0 74 02 5b c3 e8 c0 00 cc
RSP: 0018:ffffc9000348fe58 EFLAGS: 00000202
RAX: 0000000000ec4657 RBX: ffff88807b2b4ae8 RCX: 1ffffffff1d55691
RDX: 0000000000000000 RSI: ffffffff8a6c7140 RDI: 0000000000000001
RBP: ffff88807b2b4ae8 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff8eaae617 R11: 0000000000000000 R12: ffff88807b2b4a50
R13: ffff88807b2b4100 R14: ffffed100f65694a R15: 0000000000000001
 task_work_run+0x127/0x240 kernel/task_work.c:175
 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:171 [inline]
 exit_to_user_mode_prepare+0x210/0x240 kernel/entry/common.c:204
 __syscall_exit_to_user_mode_work kernel/entry/common.c:286 [inline]
 syscall_exit_to_user_mode+0x1d/0x50 kernel/entry/common.c:297
 do_syscall_64+0x44/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f9e6331a37f
Code: af 6a 0d 00 f7 d8 64 89 02 48 83 c8 ff 44 89 c7 48 89 04 24 e8 e1 fc f9 ff 48 8b 04 24 48 83 c4 28 c3 c3 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 76 10 48 8b 15 7a 6a 0d 00 f7 d8 64 89 02 48 83
RSP: 002b:00007ffdb83e8cd8 EFLAGS: 00000206 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000561c99ff2830 RCX: 00007f9e6331a37f
RDX: 00007f9e633f1b00 RSI: 0000561c99fbd010 RDI: 000000000000000e
RBP: 0000561c9a02f740 R08: 0000000000000000 R09: 28a09045a0baa66c
R10: 0000000000000812 R11: 0000000000000206 R12: 0000561c99fe7160
R13: 00000000000000fe R14: 0000561c99f5b1c4 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10505 jiffies! g93817 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:27984 pid:15    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5381 [inline]
 __schedule+0xee1/0x59f0 kernel/sched/core.c:6710
 schedule+0xe7/0x1b0 kernel/sched/core.c:6786
 schedule_timeout+0x157/0x2c0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x1ec/0xa50 kernel/rcu/tree.c:1609
 rcu_gp_kthread+0x249/0x380 kernel/rcu/tree.c:1808
 kthread+0x33a/0x430 kernel/kthread.c:389
 ret_from_fork+0x2c/0x70 arch/x86/kernel/process.c:145
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 9 Comm: kworker/u4:0 Not tainted 6.5.0-rc5-syzkaller-00053-g13b937206866 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:296 [inline]
RIP: 0010:smp_call_function_many_cond+0x4d6/0x1570 kernel/smp.c:835
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 fc 4c 89 fd 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 dc 5e 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 3c 0e 00 00 8b 43 08 31
RSP: 0018:ffffc900002ff938 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9941860 RCX: 0000000000000000
RDX: ffff88801925e200 RSI: ffffffff8179c1e4 RDI: 0000000000000005
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffffed101732830d
R13: 0000000000000001 R14: ffff8880b983d5c0 R15: ffff8880b9941868
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb5cfd7a1f8 CR3: 000000000c776000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1003
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2001 [inline]
 text_poke_bp_batch+0x250/0x780 arch/x86/kernel/alternative.c:2211
 text_poke_flush arch/x86/kernel/alternative.c:2402 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:2399 [inline]
 text_poke_finish+0x1a/0x30 arch/x86/kernel/alternative.c:2409
 arch_jump_label_transform_apply+0x17/0x30 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x32e/0x410 kernel/jump_label.c:829
 static_key_enable_cpuslocked+0x1b5/0x270 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate mm/kfence/core.c:831 [inline]
 toggle_allocation_gate+0xf4/0x250 mm/kfence/core.c:823
 process_one_work+0xaa2/0x16f0 kernel/workqueue.c:2600
 worker_thread+0x687/0x1110 kernel/workqueue.c:2751
 kthread+0x33a/0x430 kernel/kthread.c:389
 ret_from_fork+0x2c/0x70 arch/x86/kernel/process.c:145
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/08/09 12:08 upstream 13b937206866 3c27dfcd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in task_work_run
2023/06/20 11:56 linux-next f7efed9f38f8 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in task_work_run
* Struck through repros no longer work on HEAD.