bisecting fixing commit since 4707d8e5727387e36ea99c74d5ff0ad227700fd0 building syzkaller on e6b89e4e5adde15c0dc0a241e03dc215f2e249b3 testing commit 4707d8e5727387e36ea99c74d5ff0ad227700fd0 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 8f51d96437a49c8a9cdfc00194e664e1a201454d599b881bde8ae45018a41268 run #0: crashed: INFO: task hung in vfs_unlink run #1: crashed: INFO: task hung in pipe_write run #2: crashed: INFO: task hung in pipe_read run #3: crashed: INFO: task hung in pipe_write run #4: crashed: INFO: task hung in pipe_write run #5: crashed: INFO: task hung in pipe_write run #6: crashed: INFO: task hung in pipe_write run #7: crashed: INFO: task hung in pipe_write run #8: crashed: INFO: task hung in pipe_write run #9: crashed: INFO: task hung in pipe_write run #10: crashed: INFO: task hung in pipe_write run #11: crashed: INFO: task hung in pipe_read run #12: crashed: INFO: task hung in pipe_write run #13: crashed: INFO: task hung in pipe_write run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK testing current HEAD 4938296e03bd227e5020d63d418956fe52baf97c testing commit 4938296e03bd227e5020d63d418956fe52baf97c compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: acd1288a520fc6d8119a332fc3acc20985cfe4a7176d533fea44837b5eefa004 run #0: crashed: INFO: task hung in pipe_write run #1: crashed: INFO: task hung in pipe_write run #2: crashed: INFO: task hung in vfs_unlink run #3: crashed: INFO: task hung in pipe_write run #4: crashed: INFO: task hung in vfs_unlink run #5: crashed: INFO: task hung in pipe_write run #6: crashed: INFO: task hung in vfs_unlink run #7: OK run #8: OK run #9: OK revisions tested: 2, total time: 34m42.41995922s (build: 16m49.722463632s, test: 17m18.980665458s) the crash still happens on HEAD commit msg: Linux 4.19.198 crash: INFO: task hung in vfs_unlink INFO: task syz-executor.3:5846 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.3 D23640 5846 1 0x00000004 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 __rwsem_down_write_failed_common kernel/locking/rwsem-xadd.c:589 [inline] rwsem_down_write_failed+0x3bb/0x780 kernel/locking/rwsem-xadd.c:618 call_rwsem_down_write_failed+0x17/0x30 arch/x86/lib/rwsem.S:117 __down_write arch/x86/include/asm/rwsem.h:142 [inline] down_write+0x53/0x90 kernel/locking/rwsem.c:72 inode_lock include/linux/fs.h:748 [inline] vfs_unlink+0xb3/0x430 fs/namei.c:3993 do_unlinkat+0x422/0x650 fs/namei.c:4065 __do_sys_unlink fs/namei.c:4112 [inline] __se_sys_unlink fs/namei.c:4110 [inline] __x64_sys_unlink+0x39/0x50 fs/namei.c:4110 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x463cf7 Code: Bad RIP value. RSP: 002b:00007ffcdafb76b8 EFLAGS: 00000206 ORIG_RAX: 0000000000000057 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000463cf7 RDX: 00007ffcdafb76f0 RSI: 00007ffcdafb76f0 RDI: 00007ffcdafb7780 RBP: 00007ffcdafb7780 R08: 0000000000000001 R09: 00007ffcdafb7550 R10: 00000000024dc7c3 R11: 0000000000000206 R12: 00000000004ad490 R13: 00007ffcdafb8830 R14: 00000000024dc6f0 R15: 0000000000000009 INFO: task syz-executor.5:16232 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.5 D29448 16232 5853 0x00000000 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3619 __mutex_lock_common kernel/locking/mutex.c:1015 [inline] __mutex_lock+0x4ba/0x1190 kernel/locking/mutex.c:1083 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1098 __pipe_lock fs/pipe.c:83 [inline] pipe_write+0xa6/0xd00 fs/pipe.c:380 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464319 Code: Bad RIP value. RSP: 002b:00007f95f21d9198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000464319 RDX: 000000000208e24b RSI: 0000000020000040 RDI: 0000000000000000 RBP: 0000000000526220 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007f95f21d96bc R14: 00000000ffffffff R15: 0000000000000003 INFO: task syz-executor.1:16835 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.1 D29448 16835 5850 0x00000000 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3619 __mutex_lock_common kernel/locking/mutex.c:1015 [inline] __mutex_lock+0x4ba/0x1190 kernel/locking/mutex.c:1083 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1098 __pipe_lock fs/pipe.c:83 [inline] pipe_write+0xa6/0xd00 fs/pipe.c:380 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464319 Code: Bad RIP value. RSP: 002b:00007f3024da8198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000464319 RDX: 000000000208e24b RSI: 0000000020000040 RDI: 0000000000000000 RBP: 0000000000526220 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007f3024da86bc R14: 00000000ffffffff R15: 0000000000000003 INFO: task syz-executor.5:16943 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.5 D29784 16943 5853 0x00000000 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3619 __mutex_lock_common kernel/locking/mutex.c:1015 [inline] __mutex_lock+0x4ba/0x1190 kernel/locking/mutex.c:1083 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1098 __pipe_lock fs/pipe.c:83 [inline] pipe_write+0xa6/0xd00 fs/pipe.c:380 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464319 Code: Bad RIP value. RSP: 002b:00007f95f21d9198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000464319 RDX: 000000000208e24b RSI: 0000000020000040 RDI: 0000000000000000 RBP: 0000000000526220 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007f95f21d96bc R14: 00000000ffffffff R15: 0000000000000003 INFO: task syz-executor.5:17998 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.5 D29784 17998 5853 0x00000000 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3619 __mutex_lock_common kernel/locking/mutex.c:1015 [inline] __mutex_lock+0x4ba/0x1190 kernel/locking/mutex.c:1083 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1098 __pipe_lock fs/pipe.c:83 [inline] pipe_write+0xa6/0xd00 fs/pipe.c:380 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464319 Code: Bad RIP value. RSP: 002b:00007f95f21d9198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000464319 RDX: 000000000208e24b RSI: 0000000020000040 RDI: 0000000000000000 RBP: 0000000000526220 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007f95f21d96bc R14: 00000000ffffffff R15: 0000000000000003 INFO: task syz-executor.1:19887 blocked for more than 140 seconds. Not tainted 4.19.198-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.1 D29784 19887 5850 0x00000000 Call Trace: context_switch kernel/sched/core.c:2828 [inline] __schedule+0x80c/0x1f70 kernel/sched/core.c:3517 schedule+0x7f/0x1b0 kernel/sched/core.c:3561 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3619 __mutex_lock_common kernel/locking/mutex.c:1015 [inline] __mutex_lock+0x4ba/0x1190 kernel/locking/mutex.c:1083 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1098 __pipe_lock fs/pipe.c:83 [inline] pipe_write+0xa6/0xd00 fs/pipe.c:380 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464319 Code: Bad RIP value. RSP: 002b:00007f3024da8198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000464319 RDX: 000000000208e24b RSI: 0000000020000040 RDI: 0000000000000000 RBP: 0000000000526220 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007f3024da86bc R14: 00000000ffffffff R15: 0000000000000003 Showing all locks held in the system: 1 lock held by khungtaskd/1087: #0: 00000000e74f3a94 (rcu_read_lock){....}, at: debug_show_all_locks+0x5b/0x27a kernel/locking/lockdep.c:4443 1 lock held by syz-execprog/5837: #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_lock_nested fs/pipe.c:62 [inline] #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_lock fs/pipe.c:70 [inline] #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_wait+0x185/0x1b0 fs/pipe.c:118 3 locks held by syz-executor.3/5846: #0: 00000000b9987917 (sb_writers#3){.+.+}, at: sb_start_write include/linux/fs.h:1579 [inline] #0: 00000000b9987917 (sb_writers#3){.+.+}, at: mnt_want_write+0x3c/0xa0 fs/namespace.c:360 #1: 0000000065add539 (&type->i_mutex_dir_key#3/1){+.+.}, at: inode_lock_nested include/linux/fs.h:783 [inline] #1: 0000000065add539 (&type->i_mutex_dir_key#3/1){+.+.}, at: do_unlinkat+0x24a/0x650 fs/namei.c:4051 #2: 0000000022b456ee (&sb->s_type->i_mutex_key#13){+.+.}, at: inode_lock include/linux/fs.h:748 [inline] #2: 0000000022b456ee (&sb->s_type->i_mutex_key#13){+.+.}, at: vfs_unlink+0xb3/0x430 fs/namei.c:3993 1 lock held by syz-executor.1/13464: #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: pipe_lock_nested fs/pipe.c:62 [inline] #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: pipe_lock fs/pipe.c:70 [inline] #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: pipe_wait+0x185/0x1b0 fs/pipe.c:118 1 lock held by syz-executor.5/16180: #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_lock_nested fs/pipe.c:62 [inline] #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_lock fs/pipe.c:70 [inline] #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_wait+0x185/0x1b0 fs/pipe.c:118 1 lock held by syz-executor.5/16232: #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: __pipe_lock fs/pipe.c:83 [inline] #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_write+0xa6/0xd00 fs/pipe.c:380 1 lock held by syz-executor.1/16835: #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: __pipe_lock fs/pipe.c:83 [inline] #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: pipe_write+0xa6/0xd00 fs/pipe.c:380 1 lock held by syz-executor.5/16943: #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: __pipe_lock fs/pipe.c:83 [inline] #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_write+0xa6/0xd00 fs/pipe.c:380 1 lock held by syz-executor.5/17998: #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: __pipe_lock fs/pipe.c:83 [inline] #0: 000000004c5056df (&pipe->mutex/1){+.+.}, at: pipe_write+0xa6/0xd00 fs/pipe.c:380 1 lock held by syz-executor.0/19852: #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_lock_nested fs/pipe.c:62 [inline] #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_lock fs/pipe.c:70 [inline] #0: 000000006a7991c9 (&pipe->mutex/1){+.+.}, at: pipe_wait+0x185/0x1b0 fs/pipe.c:118 1 lock held by syz-executor.1/19887: #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: __pipe_lock fs/pipe.c:83 [inline] #0: 00000000413b1332 (&pipe->mutex/1){+.+.}, at: pipe_write+0xa6/0xd00 fs/pipe.c:380 2 locks held by syz-executor.3/19890: #0: 00000000b9987917 (sb_writers#3){.+.+}, at: file_start_write include/linux/fs.h:2779 [inline] #0: 00000000b9987917 (sb_writers#3){.+.+}, at: vfs_write+0x378/0x4d0 fs/read_write.c:548 #1: 0000000022b456ee (&sb->s_type->i_mutex_key#13){+.+.}, at: inode_trylock include/linux/fs.h:768 [inline] #1: 0000000022b456ee (&sb->s_type->i_mutex_key#13){+.+.}, at: ext4_file_write_iter+0x1e2/0xe50 fs/ext4/file.c:238 ============================================= NMI backtrace for cpu 0 CPU: 0 PID: 1087 Comm: khungtaskd Not tainted 4.19.198-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x17c/0x226 lib/dump_stack.c:118 nmi_cpu_backtrace.cold.0+0x3c/0x78 lib/nmi_backtrace.c:101 nmi_trigger_cpumask_backtrace+0xf5/0x120 lib/nmi_backtrace.c:62 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:203 [inline] watchdog+0x5c3/0xb40 kernel/hung_task.c:287 kthread+0x347/0x410 kernel/kthread.c:259 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 10 Comm: rcu_preempt Not tainted 4.19.198-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:161 [inline] RIP: 0010:_raw_spin_unlock_irqrestore+0x63/0xd0 kernel/locking/spinlock.c:184 Code: 00 fc ff df 48 89 fa 48 c1 ea 03 80 3c 02 00 75 77 48 83 3d 16 8f 77 01 00 74 69 48 89 df 57 9d 0f 1f 44 00 00 e8 ed a9 89 fa 01 00 00 00 e8 e3 ff 65 fa 65 8b 05 3c 81 27 79 85 c0 74 3e 5b RSP: 0018:ffff8881f56dfd18 EFLAGS: 00000286 RAX: dffffc0000000000 RBX: 0000000000000286 RCX: 0000000000000000 RDX: 1ffffffff10a3f71 RSI: 0000000000000001 RDI: 0000000000000286 RBP: ffff8881f56dfd28 R08: fffffbfff10b1e74 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff8858f398 R13: ffffffff8858f398 R14: ffff8881f56dfea8 R15: ffffffff8858f3d0 FS: 0000000000000000(0000) GS:ffff8881f6900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fcc45fdf000 CR3: 000000000846d002 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: finish_swait+0x18e/0x290 kernel/sched/swait.c:129 rcu_gp_kthread+0xd79/0x23e0 kernel/rcu/tree.c:2202 kthread+0x347/0x410 kernel/kthread.c:259 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415