syzbot


INFO: task hung in __writeback_inodes_sb_nr (2)

Status: upstream: reported C repro on 2023/12/30 04:30
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+0a0f74ced3c329bb0c35@syzkaller.appspotmail.com
First crash: 384d, last: 5d05h
Fix bisection: failed (error log, bisect log)
  
Bug presence (2)
Date Name Commit Repro Result
2024/05/15 linux-6.1.y (ToT) 909ba1f1b414 C [report] INFO: task hung in __writeback_inodes_sb_nr
2024/05/15 upstream (ToT) 1b294a1f3561 C Didn't crash
Similar bugs (13)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in __writeback_inodes_sb_nr fs 53 2195d 2457d 0/28 closed as dup on 2018/09/08 15:33
android-49 INFO: task hung in __writeback_inodes_sb_nr 26 2291d 2434d 0/3 auto-closed as invalid on 2019/04/08 13:33
android-414 INFO: task hung in __writeback_inodes_sb_nr 1 2229d 2106d 0/1 auto-closed as invalid on 2019/06/09 16:55
upstream INFO: task hung in __writeback_inodes_sb_nr (5) ext4 C done 48 700d 1070d 22/28 fixed on 2023/02/24 13:50
linux-5.15 INFO: task hung in __writeback_inodes_sb_nr origin:lts-only C done 1496 8d06h 680d 0/3 upstream: reported C repro on 2023/03/09 17:32
linux-4.19 INFO: task hung in __writeback_inodes_sb_nr ext4 4 694d 842d 0/1 upstream: reported on 2022/09/28 21:19
linux-6.1 INFO: task hung in __writeback_inodes_sb_nr 1 518d 518d 0/3 auto-obsoleted due to no activity on 2023/11/26 06:37
upstream INFO: task hung in __writeback_inodes_sb_nr (6) bcachefs C done 3792 1h00m 581d 1/28 upstream: reported C repro on 2023/06/17 00:33
upstream INFO: task hung in __writeback_inodes_sb_nr (4) ext4 4 1078d 1132d 0/28 closed as invalid on 2022/02/08 10:10
upstream INFO: task hung in __writeback_inodes_sb_nr (3) ext4 2 1345d 1442d 0/28 auto-closed as invalid on 2021/08/11 17:49
linux-4.14 INFO: task hung in __writeback_inodes_sb_nr (2) 2 810d 811d 0/1 auto-obsoleted due to no activity on 2023/02/27 15:30
upstream INFO: task hung in __writeback_inodes_sb_nr (2) ext4 1 1626d 1626d 0/28 auto-closed as invalid on 2020/11/03 06:13
linux-4.14 INFO: task hung in __writeback_inodes_sb_nr 1 1693d 1693d 0/1 auto-closed as invalid on 2020/09/27 19:31
Fix bisection attempts (4)
Created Duration User Patch Repo Result
2024/10/14 22:00 0m bisect fix linux-6.1.y error job log
2024/08/19 07:33 1h53m bisect fix linux-6.1.y OK (0) job log log
2024/07/07 09:22 1h04m bisect fix linux-6.1.y OK (0) job log log
2024/05/28 16:14 1h29m fix candidate upstream OK (0) job log log

Sample crash report:
INFO: task syz-executor151:4261 blocked for more than 143 seconds.
      Not tainted 6.1.124-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor151 state:D stack:23880 pid:4261  ppid:4259   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x143f/0x4570 kernel/sched/core.c:6558
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 wb_wait_for_completion+0x162/0x290 fs/fs-writeback.c:191
 __writeback_inodes_sb_nr+0x2ce/0x370 fs/fs-writeback.c:2653
 sync_filesystem+0xa0/0x220 fs/sync.c:54
 generic_shutdown_super+0x6b/0x340 fs/super.c:474
 kill_block_super+0x7a/0xe0 fs/super.c:1470
 deactivate_locked_super+0xa0/0x110 fs/super.c:332
 cleanup_mnt+0x490/0x520 fs/namespace.c:1186
 task_work_run+0x246/0x300 kernel/task_work.c:203
 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
 exit_to_user_mode_loop+0xde/0x100 kernel/entry/common.c:177
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:210
 __syscall_exit_to_user_mode_work kernel/entry/common.c:292 [inline]
 syscall_exit_to_user_mode+0x60/0x270 kernel/entry/common.c:303
 do_syscall_64+0x47/0xb0 arch/x86/entry/common.c:87
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f6c2a50c447
RSP: 002b:00007ffc6f065648 EFLAGS: 00000206 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f6c2a50c447
RDX: 0000000000000000 RSI: 0000000000000009 RDI: 00007ffc6f065700
RBP: 00007ffc6f065700 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000206 R12: 00007ffc6f066770
R13: 000055555a51d6c0 R14: 431bde82d7b634db R15: 00007ffc6f066790
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8d32b290 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:517
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8d32ba90 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:517
1 lock held by khungtaskd/27:
 #0: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #0: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #0: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x51/0x290 kernel/locking/lockdep.c:6510
2 locks held by kworker/u4:3/46:
2 locks held by getty/4009:
 #0: ffff88814d285098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc9000325e2f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2198
1 lock held by syz-executor151/4261:
 #0: ffff888019eca0e0 (&type->s_umount_key#42){+.+.}-{3:3}, at: deactivate_super+0xa9/0xe0 fs/super.c:362

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 6.1.124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1ae/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf88/0xfd0 kernel/hung_task.c:377
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 46 Comm: kworker/u4:3 Not tainted 6.1.124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: writeback wb_workfn (flush-7:0)
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:29 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:check_preemption_disabled+0x58/0x110 lib/smp_processor_id.c:19
Code: 65 48 8b 04 25 28 00 00 00 48 3b 44 24 08 0f 85 c7 00 00 00 89 d8 48 83 c4 10 5b 41 5c 41 5e 41 5f c3 48 c7 04 24 00 00 00 00 <9c> 8f 04 24 f7 04 24 00 02 00 00 74 c9 49 89 f6 49 89 ff 65 4c 8b
RSP: 0018:ffffc90000b77068 EFLAGS: 00000046
RAX: 0000000080000000 RBX: 0000000000000001 RCX: ffff88801dee8000
RDX: ffff88801dee8000 RSI: ffffffff8b0c1020 RDI: ffffffff8b5e6840
RBP: 00000000ffffffff R08: ffffffff8aaec659 R09: fffffbfff2249c4d
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000246
R13: ffff88801dee8000 R14: 00000000ffffffff R15: ffff888140c8ca18
FS:  0000000000000000(0000) GS:ffff8880b8f00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000564215fcc600 CR3: 000000007f817000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 lockdep_recursion_inc kernel/locking/lockdep.c:462 [inline]
 lock_is_held_type+0x76/0x180 kernel/locking/lockdep.c:5704
 lock_is_held include/linux/lockdep.h:283 [inline]
 xa_entry include/linux/xarray.h:1198 [inline]
 xas_descend+0xfe/0x470 lib/xarray.c:204
 xas_load+0xba/0x140 lib/xarray.c:242
 mapping_get_entry mm/filemap.c:1881 [inline]
 __filemap_get_folio+0x22c/0xe80 mm/filemap.c:1943
 pagecache_get_page+0x28/0x250 mm/folio-compat.c:110
 find_or_create_page include/linux/pagemap.h:646 [inline]
 grow_dev_page fs/buffer.c:946 [inline]
 grow_buffers fs/buffer.c:1011 [inline]
 __getblk_slow fs/buffer.c:1038 [inline]
 __getblk_gfp+0x211/0xa20 fs/buffer.c:1333
 __bread_gfp+0x2a/0x370 fs/buffer.c:1367
 sb_bread include/linux/buffer_head.h:338 [inline]
 __fat_write_inode+0x281/0x9d0 fs/fat/inode.c:870
 write_inode fs/fs-writeback.c:1460 [inline]
 __writeback_single_inode+0x67d/0x11e0 fs/fs-writeback.c:1677
 writeback_sb_inodes+0xc2b/0x1b20 fs/fs-writeback.c:1903
 wb_writeback+0x49d/0xe10 fs/fs-writeback.c:2077
 wb_do_writeback fs/fs-writeback.c:2220 [inline]
 wb_workfn+0x427/0x1020 fs/fs-writeback.c:2260
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.571 msecs

Crashes (21):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/01/12 19:45 linux-6.1.y c63962be84ef 6dbc6a9b .config console log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2025/01/12 13:41 linux-6.1.y c63962be84ef 6dbc6a9b .config console log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2025/01/11 23:50 linux-6.1.y c63962be84ef 6dbc6a9b .config console log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2025/01/11 16:16 linux-6.1.y c63962be84ef 6dbc6a9b .config console log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/05/15 04:59 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/12/30 16:59 linux-6.1.y 563edd786f0a d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/12/27 21:53 linux-6.1.y 563edd786f0a d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/12/07 05:02 linux-6.1.y e4d90d63d385 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/12/05 17:34 linux-6.1.y e4d90d63d385 29f61fce .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/12/02 22:19 linux-6.1.y e4d90d63d385 b499ea68 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/11/29 13:03 linux-6.1.y e4d90d63d385 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/11/29 05:05 linux-6.1.y e4d90d63d385 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/11/29 04:08 linux-6.1.y e4d90d63d385 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/10/23 01:41 linux-6.1.y 7ec6f9fa3d97 15fa2979 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/08/28 11:38 linux-6.1.y ee5e09825b81 6c853ff9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/04/16 12:05 linux-6.1.y cd5d98c0556c 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/04/07 11:17 linux-6.1.y 347385861c50 ca620dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2024/03/10 00:10 linux-6.1.y 61adba85cc40 6ee49f2e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in __writeback_inodes_sb_nr
2023/12/30 04:29 linux-6.1.y 4aa6747d9352 fb427a07 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: task hung in __writeback_inodes_sb_nr
2025/01/07 06:00 linux-6.1.y 7dc732d24ff7 f3558dbf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in __writeback_inodes_sb_nr
2024/04/29 11:10 linux-6.1.y f2295faba5e8 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in __writeback_inodes_sb_nr
* Struck through repros no longer work on HEAD.