syzbot

 sign-in | mailing list | source | docs
🐞 Open [1009] ≡ Subsystems 🐞 Fixed [5247] 🐞 Invalid [12544] ⬇ Missing Backports [83] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in pwq_unbound_release_workfn (4)

Status: auto-obsoleted due to no activity on 2023/10/08 15:59
Subsystems: kernel
[Documentation on labels]
First crash: 344d, last: 295d
▶ ▼ Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in pwq_unbound_release_workfn kernel 1 883d 883d 0/26 closed as invalid on 2022/02/08 10:10
upstream INFO: rcu detected stall in pwq_unbound_release_workfn (2) kernel 1 662d 662d 0/26 auto-closed as invalid on 2022/09/06 17:05
upstream INFO: rcu detected stall in pwq_unbound_release_workfn (3) kernel 1 461d 461d 0/26 auto-obsoleted due to no activity on 2023/04/25 19:12

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=5f0c/1/0x4000000000000000 softirq=150809/150809 fqs=0
rcu: 	(detected by 1, t=10502 jiffies, g=215777, q=126 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 11784 Comm: kworker/0:3 Not tainted 6.4.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Workqueue: events pwq_unbound_release_workfn
RIP: 0010:check_preemption_disabled+0x19/0x110 lib/smp_processor_id.c:14
Code: 42 38 8b eb 0c 66 2e 0f 1f 84 00 00 00 00 00 66 90 41 57 41 56 41 54 53 48 83 ec 10 65 48 8b 04 25 28 00 00 00 48 89 44 24 08 <65> 8b 1d 8c 9f 54 75 65 8b 05 81 9f 54 75 a9 ff ff ff 7f 74 22 65
RSP: 0018:ffffc90000007c10 EFLAGS: 00000082
RAX: 47306a525616b600 RBX: 0000000000000002 RCX: ffff88807e0fbb80
RDX: ffff88807e0fbb80 RSI: ffffffff8aea9e00 RDI: ffffffff8b384240
RBP: 0000000000000001 R08: ffffffff88d2403e R09: 0000000000000003
R10: ffffffffffffffff R11: dffffc0000000001 R12: 0000000000000046
R13: ffff88807e0fbb80 R14: 00000000ffffffff R15: ffff888028f24300
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555555df3708 CR3: 000000002a169000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 000000000000003b DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 lockdep_recursion_finish kernel/locking/lockdep.c:467 [inline]
 lock_is_held_type+0x101/0x190 kernel/locking/lockdep.c:5763
 lock_is_held include/linux/lockdep.h:288 [inline]
 advance_sched+0xd0/0xc80 net/sched/sch_taprio.c:930
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x59f/0xd10 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x396/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1095 [inline]
 __sysvec_apic_timer_interrupt+0x13f/0x480 arch/x86/kernel/apic/apic.c:1112
 sysvec_apic_timer_interrupt+0x90/0xb0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:synchronize_rcu+0x0/0x3e0 kernel/rcu/tree.c:3489
Code: bb 6e 00 e9 84 fe ff ff 44 89 f1 80 e1 07 80 c1 03 38 c1 0f 8c 9b fe ff ff 4c 89 f7 e8 29 bb 6e 00 e9 8e fe ff ff 0f 1f 40 00 <f3> 0f 1e fa 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 e4 e0 48
RSP: 0018:ffffc9000360fb38 EFLAGS: 00000206
RAX: dffffc0000000000 RBX: 1ffff920006c1f70 RCX: ffffffff91b20103
RDX: 0000000000000001 RSI: ffffffff8aea9960 RDI: ffffffff8b384240
RBP: ffffc9000360fc08 R08: dffffc0000000000 R09: fffffbfff2361cb3
R10: 0000000000000000 R11: dffffc0000000001 R12: ffffffff91b0b918
R13: 1ffff920006c1f6c R14: 0000000000000a07 R15: ffffc9000360fb80
 lockdep_unregister_key+0x4e8/0x5b0 kernel/locking/lockdep.c:6412
 wq_unregister_lockdep kernel/workqueue.c:3615 [inline]
 pwq_unbound_release_workfn+0x241/0x290 kernel/workqueue.c:3863
 process_one_work+0x8a0/0x10e0 kernel/workqueue.c:2405
 worker_thread+0xa63/0x1210 kernel/workqueue.c:2552
 kthread+0x2b8/0x350 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10501 jiffies! g215777 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=0 timer-softirq=167355
rcu: rcu_preempt kthread starved for 10502 jiffies! g215777 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:25208 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5343 [inline]
 __schedule+0x187b/0x4900 kernel/sched/core.c:6669
 schedule+0xc3/0x180 kernel/sched/core.c:6745
 schedule_timeout+0x1bd/0x310 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x2c6/0x1010 kernel/rcu/tree.c:1609
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1808
 kthread+0x2b8/0x350 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 11784 Comm: kworker/0:3 Not tainted 6.4.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Workqueue: events pwq_unbound_release_workfn
RIP: 0010:kasan_check_range+0x7/0x290 mm/kasan/generic.c:186
Code: 48 89 c7 e8 cb 89 cc 08 31 c0 c3 0f 0b b8 ea ff ff ff c3 0f 0b b8 ea ff ff ff c3 0f 1f 84 00 00 00 00 00 66 0f 1f 00 55 41 57 <41> 56 53 b0 01 48 85 f6 0f 84 9a 01 00 00 48 89 fd 48 01 f5 0f 82
RSP: 0018:ffffc90000007c48 EFLAGS: 00000046
RAX: 0000000000000000 RBX: dffffc0000000000 RCX: ffffffff816c7858
RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff8880b982b740
RBP: ffffc90000007d50 R08: dffffc0000000000 R09: fffffbfff1cab8b6
R10: 0000000000000000 R11: dffffc0000000001 R12: dffffc0000000000
R13: 1ffff92000000f98 R14: ffffc90000007ce0 R15: ffff8880b982b740
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555555df3708 CR3: 000000002a169000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 000000000000003b DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 instrument_atomic_read include/linux/instrumented.h:68 [inline]
 atomic_read include/linux/atomic/atomic-instrumented.h:27 [inline]
 queued_spin_is_locked include/asm-generic/qspinlock.h:57 [inline]
 debug_spin_unlock kernel/locking/spinlock_debug.c:100 [inline]
 do_raw_spin_unlock+0x58/0x8b0 kernel/locking/spinlock_debug.c:140
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:150 [inline]
 _raw_spin_unlock_irqrestore+0x81/0x140 kernel/locking/spinlock.c:194
 __run_hrtimer kernel/time/hrtimer.c:1681 [inline]
 __hrtimer_run_queues+0x490/0xd10 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x396/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1095 [inline]
 __sysvec_apic_timer_interrupt+0x13f/0x480 arch/x86/kernel/apic/apic.c:1112
 sysvec_apic_timer_interrupt+0x90/0xb0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:synchronize_rcu+0x0/0x3e0 kernel/rcu/tree.c:3489
Code: bb 6e 00 e9 84 fe ff ff 44 89 f1 80 e1 07 80 c1 03 38 c1 0f 8c 9b fe ff ff 4c 89 f7 e8 29 bb 6e 00 e9 8e fe ff ff 0f 1f 40 00 <f3> 0f 1e fa 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83 e4 e0 48
RSP: 0018:ffffc9000360fb38 EFLAGS: 00000206
RAX: dffffc0000000000 RBX: 1ffff920006c1f70 RCX: ffffffff91b20103
RDX: 0000000000000001 RSI: ffffffff8aea9960 RDI: ffffffff8b384240
RBP: ffffc9000360fc08 R08: dffffc0000000000 R09: fffffbfff2361cb3
R10: 0000000000000000 R11: dffffc0000000001 R12: ffffffff91b0b918
R13: 1ffff920006c1f6c R14: 0000000000000a07 R15: ffffc9000360fb80
 lockdep_unregister_key+0x4e8/0x5b0 kernel/locking/lockdep.c:6412
 wq_unregister_lockdep kernel/workqueue.c:3615 [inline]
 pwq_unbound_release_workfn+0x241/0x290 kernel/workqueue.c:3863
 process_one_work+0x8a0/0x10e0 kernel/workqueue.c:2405
 worker_thread+0xa63/0x1210 kernel/workqueue.c:2552
 kthread+0x2b8/0x350 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/05/22 15:52 upstream 44c026a73be8 4bce1a3e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in pwq_unbound_release_workfn
2023/07/10 15:58 linux-next fe57d0d86f03 52ae002a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in pwq_unbound_release_workfn
* Struck through repros no longer work on HEAD.