syzbot


INFO: task hung in sync_inodes_sb

Status: upstream: reported C repro on 2023/03/10 02:07
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+9a20df095d11804348bc@syzkaller.appspotmail.com
First crash: 560d, last: 8h02m
Bug presence (3)
Date Name Commit Repro Result
2023/09/14 linux-6.1.y (ToT) 09045dae0d90 C [report] INFO: task hung in sync_inodes_sb
2023/05/02 upstream (ToT) 865fdb08197e C [report] INFO: task hung in sync_inodes_sb
2023/09/14 upstream (ToT) 9fdfb15a3dbf C Didn't crash
Similar bugs (11)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: task hung in sync_inodes_sb origin:upstream missing-backport C error 207 16h03m 560d 0/3 upstream: reported C repro on 2023/03/10 02:13
upstream INFO: task hung in sync_inodes_sb (5) ext4 C error inconclusive 1399 3h12m 430d 0/28 upstream: reported C repro on 2023/07/18 04:00
upstream INFO: task hung in sync_inodes_sb (2) fs 4 1978d 1996d 0/28 auto-closed as invalid on 2019/10/19 16:22
upstream INFO: task hung in sync_inodes_sb (3) fs mm C done 6 1721d 1730d 15/28 fixed on 2020/02/14 01:19
upstream INFO: task hung in sync_inodes_sb fs 58 2076d 2314d 0/28 closed as dup on 2018/09/08 15:37
linux-4.14 INFO: task hung in sync_inodes_sb 1 1691d 1691d 0/1 auto-closed as invalid on 2020/06/02 17:26
android-49 INFO: task hung in sync_inodes_sb 11 2215d 2294d 0/3 auto-closed as invalid on 2019/02/24 06:19
android-49 INFO: task hung in sync_inodes_sb (2) 2 1804d 1966d 0/3 auto-closed as invalid on 2020/02/10 00:14
upstream INFO: task hung in sync_inodes_sb (4) nilfs C done inconclusive 345 444d 1399d 23/28 fixed on 2023/07/04 09:17
linux-4.14 INFO: task hung in sync_inodes_sb (2) vfs C 11 584d 1352d 0/1 upstream: reported C repro on 2021/01/07 19:48
linux-4.19 INFO: task hung in sync_inodes_sb xfs C error 13 602d 1442d 0/1 upstream: reported C repro on 2020/10/09 07:19
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2023/12/16 04:55 1h44m bisect fix linux-6.1.y OK (0) job log log
2023/10/18 13:31 1h27m fix candidate upstream OK (0) job log log

Sample crash report:
INFO: task syz-executor257:3588 blocked for more than 143 seconds.
      Not tainted 6.1.75-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor257 state:D stack:23880 pid:3588  ppid:3586   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5245 [inline]
 __schedule+0x142d/0x4550 kernel/sched/core.c:6558
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 wb_wait_for_completion+0x162/0x290 fs/fs-writeback.c:191
 sync_inodes_sb+0x28f/0xb10 fs/fs-writeback.c:2739
 sync_filesystem+0x16b/0x220 fs/sync.c:64
 generic_shutdown_super+0x6b/0x340 fs/super.c:474
 kill_block_super+0x7a/0xe0 fs/super.c:1459
 deactivate_locked_super+0xa0/0x110 fs/super.c:332
 cleanup_mnt+0x490/0x520 fs/namespace.c:1186
 task_work_run+0x246/0x300 kernel/task_work.c:179
 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
 exit_to_user_mode_loop+0xde/0x100 kernel/entry/common.c:171
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:204
 __syscall_exit_to_user_mode_work kernel/entry/common.c:286 [inline]
 syscall_exit_to_user_mode+0x60/0x270 kernel/entry/common.c:297
 do_syscall_64+0x49/0xb0 arch/x86/entry/common.c:87
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f11ad651547
RSP: 002b:00007fffbcff0fb8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f11ad651547
RDX: 0000000000000000 RSI: 000000000000000a RDI: 00007fffbcff1070
RBP: 00007fffbcff1070 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000206 R12: 00007fffbcff20e0
R13: 0000555555bd66c0 R14: 431bde82d7b634db R15: 00007fffbcff2100
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8d12a490 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8d12ac90 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by khungtaskd/28:
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:318 [inline]
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:759 [inline]
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x51/0x290 kernel/locking/lockdep.c:6494
2 locks held by getty/3305:
 #0: ffff88814b2c6098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc900031262f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2188
2 locks held by syz-executor257/3588:
 #0: ffff888047b840e0 (&type->s_umount_key#31){++++}-{3:3}, at: deactivate_super+0xa9/0xe0 fs/super.c:362
 #1: ffff8881433087d0 (&bdi->wb_switch_rwsem){+.+.}-{3:3}, at: bdi_down_write_wb_switch_rwsem fs/fs-writeback.c:362 [inline]
 #1: ffff8881433087d0 (&bdi->wb_switch_rwsem){+.+.}-{3:3}, at: sync_inodes_sb+0x270/0xb10 fs/fs-writeback.c:2737
3 locks held by kworker/u4:2/3594:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 28 Comm: khungtaskd Not tainted 6.1.75-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b0/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf88/0xfd0 kernel/hung_task.c:377
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 102 Comm: kworker/u4:4 Not tainted 6.1.75-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:dequeue_task_fair+0xafb/0x1460
Code: 24 70 0f b6 04 10 84 c0 0f 85 48 05 00 00 48 8b 44 24 18 8b 18 48 8b 44 24 78 0f b6 04 10 84 c0 0f 85 5c 05 00 00 85 db 74 0d <48> 8b 44 24 50 3b 18 0f 84 5e 03 00 00 f6 44 24 24 01 48 8b 5c 24
RSP: 0018:ffffc900015b7700 EFLAGS: 00000002
RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff888013fb5940
RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffff8880b9939edc
RBP: ffffc900015b7850 R08: ffffffff813f109f R09: fffffbfff1ce6a56
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000000
R13: 0000000000000001 R14: ffff8880b9939ec0 R15: 0000000000000009
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000560b7c6b5600 CR3: 000000000ce8e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 deactivate_task kernel/sched/core.c:2097 [inline]
 __schedule+0x5f9/0x4550 kernel/sched/core.c:6512
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1935
 toggle_allocation_gate+0x23b/0x480 mm/kfence/core.c:811
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.450 msecs

Crashes (428):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/01/28 22:51 linux-6.1.y 883d1a956208 cc4a4020 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2023/09/22 15:13 linux-6.1.y a356197db198 0b6a67ac .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2023/09/15 08:09 linux-6.1.y 09045dae0d90 0b6a67ac .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2023/03/26 19:08 linux-6.1.y e3a87a10f259 fbf0499a .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/03/01 03:57 linux-6.1.y 81e1dc2f7001 352ab904 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in sync_inodes_sb
2023/03/11 00:33 linux-6.1.y 8a923980a190 5205ef30 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in sync_inodes_sb
2023/06/17 13:18 linux-6.1.y ca87e77a2ef8 f3921d4d .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2023/05/15 05:41 linux-6.1.y bf4ad6fa4e53 2b9ba477 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/20 15:55 linux-6.1.y e526b12bf916 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/20 01:25 linux-6.1.y e526b12bf916 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/19 08:18 linux-6.1.y e526b12bf916 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/19 01:45 linux-6.1.y e526b12bf916 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/17 09:26 linux-6.1.y 5f55cad62cc9 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/16 21:15 linux-6.1.y 5f55cad62cc9 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/15 11:57 linux-6.1.y 5f55cad62cc9 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/15 01:42 linux-6.1.y 5f55cad62cc9 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/14 18:39 linux-6.1.y 5f55cad62cc9 c7e35043 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/14 14:24 linux-6.1.y 5f55cad62cc9 c7e35043 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/13 20:58 linux-6.1.y 5f55cad62cc9 b58f933c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/13 07:54 linux-6.1.y 5f55cad62cc9 73e8a465 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/12 17:01 linux-6.1.y 5f55cad62cc9 60e1a995 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/12 16:49 linux-6.1.y 5f55cad62cc9 60e1a995 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/12 05:06 linux-6.1.y 5ca5b389fddf d94c83d8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/12 01:52 linux-6.1.y 5ca5b389fddf d94c83d8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/11 10:09 linux-6.1.y 5ca5b389fddf 8ab55d0e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/10 21:07 linux-6.1.y 5ca5b389fddf 86aa7bd7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/10 07:32 linux-6.1.y 5ca5b389fddf 784df80e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/09 19:34 linux-6.1.y 5ca5b389fddf 073f8be2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/09 19:32 linux-6.1.y 5ca5b389fddf 073f8be2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/08 13:38 linux-6.1.y 5ca5b389fddf 9750182a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/07 01:23 linux-6.1.y 699506173494 9750182a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/05 19:50 linux-6.1.y 699506173494 464ac2ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/05 15:28 linux-6.1.y 699506173494 464ac2ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/05 05:53 linux-6.1.y 699506173494 dfbe2ed4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/05 01:39 linux-6.1.y 699506173494 dfbe2ed4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/04 12:14 linux-6.1.y 311d8503ef9f 9d47f20a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/04 12:13 linux-6.1.y 311d8503ef9f 9d47f20a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/03 15:51 linux-6.1.y 311d8503ef9f 326f9c5a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/09/03 08:26 linux-6.1.y 311d8503ef9f 8045124c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/31 01:55 linux-6.1.y 311d8503ef9f 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/30 11:38 linux-6.1.y 311d8503ef9f ee2602b8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/29 23:55 linux-6.1.y ee5e09825b81 9bd464fc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/29 19:44 linux-6.1.y ee5e09825b81 9bd464fc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/29 09:38 linux-6.1.y ee5e09825b81 ef3de9e8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/28 20:09 linux-6.1.y ee5e09825b81 ef3de9e8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/27 22:50 linux-6.1.y ee5e09825b81 6c853ff9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/27 02:01 linux-6.1.y ee5e09825b81 9aee4e0b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/26 21:23 linux-6.1.y ee5e09825b81 9aee4e0b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/23 21:34 linux-6.1.y ee5e09825b81 d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/23 18:01 linux-6.1.y ee5e09825b81 d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/23 04:50 linux-6.1.y ee5e09825b81 ce8a9099 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/22 07:39 linux-6.1.y ee5e09825b81 ca02180f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/21 15:11 linux-6.1.y ee5e09825b81 db5852f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
2024/08/15 02:51 linux-6.1.y 117ac406ba90 e6b88e20 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in sync_inodes_sb
2023/03/10 02:07 linux-6.1.y 42616e0f09fb f08b59ac .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in sync_inodes_sb
* Struck through repros no longer work on HEAD.