syzbot

 sign-in | mailing list | source | docs
🐞 Open [743]
🐞 Fixed [120]
🐞 Invalid [765]
⬇ Missing Backports [75]
≡ Crashes
Kernel Health Bugs/Month Bug Lifetimes Fuzzing
💬 Send us feedback

BUG: soft lockup in wb_workfn

Status: upstream: reported C repro on 2024/10/24 21:18
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+48cac2bbba146c43df3d@syzkaller.appspotmail.com
First crash: 226d, last: 3d20h
▶ ▼ Bug presence (1)
Date Name Commit Repro Result
2025/05/31 upstream (ToT) dee264c16a63 C [report] INFO: rcu detected stall in worker_thread
▶ ▼ Similar bugs (8)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in wb_workfn origin:lts-only syz error 4 241d 313d 0/3 auto-obsoleted due to no activity on 2025/01/21 00:06
linux-5.15 INFO: rcu detected stall in wb_workfn (2) origin:lts-only C error 10 1d06h 114d 0/3 upstream: reported C repro on 2025/02/13 22:15
upstream BUG: soft lockup in wb_workfn kernel 1 2125d 2121d 0/28 auto-closed as invalid on 2019/11/11 12:45
upstream INFO: rcu detected stall in wb_workfn (2) fs 1 1076d 1076d 0/28 auto-closed as invalid on 2022/09/25 22:53
upstream INFO: rcu detected stall in wb_workfn mm 2 1319d 1326d 0/28 auto-closed as invalid on 2022/01/25 21:50
linux-4.19 INFO: rcu detected stall in wb_workfn 1 1024d 1024d 0/1 auto-obsoleted due to no activity on 2022/12/16 09:39
upstream INFO: rcu detected stall in wb_workfn (3) hfs ext4 block 3 657d 739d 0/28 auto-obsoleted due to no activity on 2023/11/18 21:40
linux-4.14 INFO: rcu detected stall in wb_workfn 1 2057d 2057d 0/1 auto-closed as invalid on 2020/02/17 08:22
▶ ▼ Fix bisection attempts (1)
Created Duration User Patch Repo Result
2025/02/23 21:19 2h43m bisect fix linux-6.1.y OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=61dc/1/0x4000000000000000 softirq=8478/8478 fqs=0
	(detected by 1, t=10502 jiffies, g=7577, q=387 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 9 Comm: kworker/u4:0 Not tainted 6.1.140-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/07/2025
Workqueue: writeback wb_workfn (flush-8:0)
RIP: 0010:__lock_release kernel/locking/lockdep.c:5348 [inline]
RIP: 0010:lock_release+0x25d/0x910 kernel/locking/lockdep.c:5682
Code: e8 03 42 0f b6 04 28 84 c0 0f 85 af 04 00 00 41 83 3e 00 0f 85 c0 03 00 00 4d 8d 74 24 20 4c 89 f3 48 c1 eb 03 42 0f b6 04 2b <84> c0 4c 8b 7c 24 18 0f 85 ab 04 00 00 41 8b 06 3d 00 00 10 00 72
RSP: 0018:ffffc90000007ae0 EFLAGS: 00000802
RAX: 0000000000000000 RBX: 1ffff11027fc7183 RCX: 0000000000000001
RDX: 0000000000000007 RSI: ffffffff96c729e8 RDI: ffff88813fe38bf8
RBP: ffffc90000007bf0 R08: dffffc0000000000 R09: fffffbfff1bfd006
R10: fffffbfff1bfd006 R11: 1ffffffff1bfd005 R12: ffff88813fe38bf8
R13: dffffc0000000000 R14: ffff88813fe38c18 R15: 0000000000000007
FS:  0000000000000000(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd0f65bf19 CR3: 000000007474a000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:149 [inline]
 _raw_spin_unlock_irqrestore+0x6d/0x100 kernel/locking/spinlock.c:194
 debug_object_activate+0x2d7/0x490 lib/debugobjects.c:716
 debug_hrtimer_activate kernel/time/hrtimer.c:411 [inline]
 debug_activate kernel/time/hrtimer.c:466 [inline]
 enqueue_hrtimer+0x30/0x3f0 kernel/time/hrtimer.c:1075
 __run_hrtimer kernel/time/hrtimer.c:1708 [inline]
 __hrtimer_run_queues+0x642/0xc80 kernel/time/hrtimer.c:1755
 hrtimer_interrupt+0x3bb/0x8d0 kernel/time/hrtimer.c:1817
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1107 [inline]
 __sysvec_apic_timer_interrupt+0x153/0x5a0 arch/x86/kernel/apic/apic.c:1124
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1118 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1118
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa5/0x100 kernel/locking/spinlock.c:194
Code: 74 05 e8 2e f7 6d f7 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4b f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> b6 71 3e f7 65 8b 05 57 33 e9 75 85 c0 74 3c 48 c7 04 24 0e 36
RSP: 0018:ffffc900000e6c60 EFLAGS: 00000206
RAX: b77252b9441a0700 RBX: 0000000000000a02 RCX: b77252b9441a0700
RDX: dffffc0000000000 RSI: ffffffff8a6c0000 RDI: 0000000000000001
RBP: ffffc900000e6cf0 R08: dffffc0000000000 R09: fffffbfff211705d
R10: fffffbfff211705d R11: 1ffffffff211705c R12: dffffc0000000000
R13: 0000000000000000 R14: ffff88807423e930 R15: 1ffff9200001cd8c
 spin_unlock_irqrestore include/linux/spinlock.h:406 [inline]
 __folio_start_writeback+0x751/0xf80 mm/page-writeback.c:3022
 ext4_bio_write_page+0x30d/0x2ae0 fs/ext4/page-io.c:453
 mpage_submit_page+0x17a/0x210 fs/ext4/inode.c:2142
 mpage_map_and_submit_buffers fs/ext4/inode.c:2387 [inline]
 mpage_map_and_submit_extent fs/ext4/inode.c:2526 [inline]
 ext4_writepages+0x1a84/0x2e50 fs/ext4/inode.c:2855
 do_writepages+0x3b7/0x610 mm/page-writeback.c:2491
 __writeback_single_inode+0x156/0x1160 fs/fs-writeback.c:1612
 writeback_sb_inodes+0xad8/0x17d0 fs/fs-writeback.c:1903
 __writeback_inodes_wb+0x12a/0x3f0 fs/fs-writeback.c:1974
 wb_writeback+0x47a/0xd00 fs/fs-writeback.c:2079
 wb_check_old_data_flush fs/fs-writeback.c:2179 [inline]
 wb_do_writeback fs/fs-writeback.c:2232 [inline]
 wb_workfn+0xb66/0xec0 fs/fs-writeback.c:2260
 process_one_work+0x898/0x1160 kernel/workqueue.c:2292
 worker_thread+0xaa2/0x1250 kernel/workqueue.c:2439
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10501 jiffies! g7577 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=0 timer-softirq=3805
rcu: rcu_preempt kthread starved for 10502 jiffies! g7577 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:27464 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5244 [inline]
 __schedule+0x10e9/0x40d0 kernel/sched/core.c:6561
 schedule+0xb9/0x180 kernel/sched/core.c:6637
 schedule_timeout+0x15c/0x280 kernel/time/timer.c:1965
 rcu_gp_fqs_loop+0x2f2/0x1310 kernel/rcu/tree.c:1706
 rcu_gp_kthread+0x95/0x380 kernel/rcu/tree.c:1905
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 9 Comm: kworker/u4:0 Not tainted 6.1.140-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/07/2025
Workqueue: writeback wb_workfn (flush-8:0)
RIP: 0010:debug_spin_unlock kernel/locking/spinlock_debug.c:101 [inline]
RIP: 0010:do_raw_spin_unlock+0x66/0x230 kernel/locking/spinlock_debug.c:140
Code: 48 89 df be 04 00 00 00 e8 d7 1d 6e 00 48 89 d8 48 c1 e8 03 42 0f b6 04 20 84 c0 0f 85 6e 01 00 00 83 3b 00 0f 84 e4 00 00 00 <4c> 8d 73 10 4d 89 f5 49 c1 ed 03 43 80 7c 25 00 00 74 08 4c 89 f7
RSP: 0018:ffffc90000007c28 EFLAGS: 00000002
RAX: 0000000000000000 RBX: ffffffff96c729d0 RCX: ffffffff816430c9
RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffffffff96c729d0
RBP: ffffc90000007ce0 R08: dffffc0000000000 R09: fffffbfff2d8e53b
R10: fffffbfff2d8e53b R11: 1ffffffff2d8e53a R12: dffffc0000000000
R13: dffffc0000000000 R14: ffffffff96c729d4 R15: 1ffff92000000f8c
FS:  0000000000000000(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd0f65bf19 CR3: 000000007474a000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:150 [inline]
 _raw_spin_unlock_irqrestore+0x75/0x100 kernel/locking/spinlock.c:194
 debug_hrtimer_deactivate kernel/time/hrtimer.c:416 [inline]
 debug_deactivate+0x29/0x240 kernel/time/hrtimer.c:472
 __run_hrtimer kernel/time/hrtimer.c:1659 [inline]
 __hrtimer_run_queues+0x2d0/0xc80 kernel/time/hrtimer.c:1755
 hrtimer_interrupt+0x3bb/0x8d0 kernel/time/hrtimer.c:1817
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1107 [inline]
 __sysvec_apic_timer_interrupt+0x153/0x5a0 arch/x86/kernel/apic/apic.c:1124
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1118 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1118
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xa5/0x100 kernel/locking/spinlock.c:194
Code: 74 05 e8 2e f7 6d f7 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4b f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> b6 71 3e f7 65 8b 05 57 33 e9 75 85 c0 74 3c 48 c7 04 24 0e 36
RSP: 0018:ffffc900000e6c60 EFLAGS: 00000206
RAX: b77252b9441a0700 RBX: 0000000000000a02 RCX: b77252b9441a0700
RDX: dffffc0000000000 RSI: ffffffff8a6c0000 RDI: 0000000000000001
RBP: ffffc900000e6cf0 R08: dffffc0000000000 R09: fffffbfff211705d
R10: fffffbfff211705d R11: 1ffffffff211705c R12: dffffc0000000000
R13: 0000000000000000 R14: ffff88807423e930 R15: 1ffff9200001cd8c
 spin_unlock_irqrestore include/linux/spinlock.h:406 [inline]
 __folio_start_writeback+0x751/0xf80 mm/page-writeback.c:3022
 ext4_bio_write_page+0x30d/0x2ae0 fs/ext4/page-io.c:453
 mpage_submit_page+0x17a/0x210 fs/ext4/inode.c:2142
 mpage_map_and_submit_buffers fs/ext4/inode.c:2387 [inline]
 mpage_map_and_submit_extent fs/ext4/inode.c:2526 [inline]
 ext4_writepages+0x1a84/0x2e50 fs/ext4/inode.c:2855
 do_writepages+0x3b7/0x610 mm/page-writeback.c:2491
 __writeback_single_inode+0x156/0x1160 fs/fs-writeback.c:1612
 writeback_sb_inodes+0xad8/0x17d0 fs/fs-writeback.c:1903
 __writeback_inodes_wb+0x12a/0x3f0 fs/fs-writeback.c:1974
 wb_writeback+0x47a/0xd00 fs/fs-writeback.c:2079
 wb_check_old_data_flush fs/fs-writeback.c:2179 [inline]
 wb_do_writeback fs/fs-writeback.c:2232 [inline]
 wb_workfn+0xb66/0xec0 fs/fs-writeback.c:2260
 process_one_work+0x898/0x1160 kernel/workqueue.c:2292
 worker_thread+0xaa2/0x1250 kernel/workqueue.c:2439
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/05/30 16:26 linux-6.1.y da3c5173c55f 3d2f584d .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in wb_workfn
2025/06/04 07:07 linux-6.1.y da3c5173c55f a30356b7 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in wb_workfn
2024/11/09 15:06 linux-6.1.y d7039b844a1c 6b856513 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in wb_workfn
2024/10/24 21:17 linux-6.1.y 7ec6f9fa3d97 0d144d1a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 BUG: soft lockup in wb_workfn
2025/03/22 00:03 linux-6.1.y 344a09659766 c6512ef7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in wb_workfn
* Struck through repros no longer work on HEAD.