syzbot


possible deadlock in iter_file_splice_write (4)

Status: closed as dup on 2024/06/05 11:05
Subsystems: overlayfs
[Documentation on labels]
Reported-by: syzbot+5ce16f43e888965f009d@syzkaller.appspotmail.com
First crash: 54d, last: 36d
Duplicate of
Title Repro Cause bisect Fix bisect Count Last Reported
possible deadlock in kernfs_seq_start kernfs C done 14 9d16h 88d
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [overlayfs?] possible deadlock in iter_file_splice_write (4) 1 (2) 2024/06/05 11:05
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in iter_file_splice_write (3) overlayfs 24 91d 133d 0/27 closed as dup on 2024/03/14 09:27
upstream possible deadlock in iter_file_splice_write overlayfs 1 1422d 1418d 0/27 auto-closed as invalid on 2020/12/30 21:45
upstream possible deadlock in iter_file_splice_write (2) overlayfs C done 2 1096d 1092d 0/27 closed as dup on 2021/07/28 14:57

Sample crash report:
F2FS-fs (loop2): Mounted with checkpoint version = 48b305e5
======================================================
WARNING: possible circular locking dependency detected
6.10.0-rc3-syzkaller-00044-g2ccbdf43d5e7 #0 Not tainted
------------------------------------------------------
syz-executor.2/10146 is trying to acquire lock:
ffff88807d403068 (&pipe->mutex){+.+.}-{3:3}, at: iter_file_splice_write+0x335/0x14e0 fs/splice.c:687

but task is already holding lock:
ffff88802b80e420 (sb_writers#18){.+.+}-{0:0}, at: do_splice+0xcf0/0x1900 fs/splice.c:1353

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (sb_writers#18){.+.+}-{0:0}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
       __sb_start_write include/linux/fs.h:1655 [inline]
       sb_start_write include/linux/fs.h:1791 [inline]
       ovl_start_write+0x11d/0x290 fs/overlayfs/util.c:31
       ovl_do_copy_up fs/overlayfs/copy_up.c:967 [inline]
       ovl_copy_up_one fs/overlayfs/copy_up.c:1168 [inline]
       ovl_copy_up_flags+0x1110/0x4470 fs/overlayfs/copy_up.c:1223
       ovl_rename+0x62a/0x1750 fs/overlayfs/dir.c:1136
       vfs_rename+0xbdb/0xf00 fs/namei.c:4887
       do_renameat2+0xd94/0x13f0 fs/namei.c:5044
       __do_sys_rename fs/namei.c:5091 [inline]
       __se_sys_rename fs/namei.c:5089 [inline]
       __x64_sys_rename+0x86/0xa0 fs/namei.c:5089
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #4 (&ovl_i_lock_key[depth]){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       ovl_inode_lock_interruptible fs/overlayfs/overlayfs.h:657 [inline]
       ovl_copy_up_start+0x53/0x310 fs/overlayfs/util.c:719
       ovl_copy_up_one fs/overlayfs/copy_up.c:1161 [inline]
       ovl_copy_up_flags+0xbc1/0x4470 fs/overlayfs/copy_up.c:1223
       ovl_create_object+0x113/0x370 fs/overlayfs/dir.c:638
       lookup_open fs/namei.c:3505 [inline]
       open_last_lookups fs/namei.c:3574 [inline]
       path_openat+0x1425/0x3280 fs/namei.c:3804
       do_filp_open+0x235/0x490 fs/namei.c:3834
       do_sys_openat2+0x13e/0x1d0 fs/open.c:1405
       do_sys_open fs/open.c:1420 [inline]
       __do_sys_openat fs/open.c:1436 [inline]
       __se_sys_openat fs/open.c:1431 [inline]
       __x64_sys_openat+0x247/0x2a0 fs/open.c:1431
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #3 (&ovl_i_mutex_dir_key[depth]){++++}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
       inode_lock_shared include/linux/fs.h:801 [inline]
       lookup_slow+0x45/0x70 fs/namei.c:1708
       walk_component+0x2e1/0x410 fs/namei.c:2004
       lookup_last fs/namei.c:2469 [inline]
       path_lookupat+0x16f/0x450 fs/namei.c:2493
       filename_lookup+0x256/0x610 fs/namei.c:2522
       kern_path+0x35/0x50 fs/namei.c:2630
       lookup_bdev+0xc5/0x290 block/bdev.c:1157
       resume_store+0x1a0/0x710 kernel/power/hibernate.c:1235
       kernfs_fop_write_iter+0x3a1/0x500 fs/kernfs/file.c:334
       new_sync_write fs/read_write.c:497 [inline]
       vfs_write+0xa72/0xc90 fs/read_write.c:590
       ksys_write+0x1a0/0x2c0 fs/read_write.c:643
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #2 (&of->mutex){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       kernfs_seq_start+0x53/0x3b0 fs/kernfs/file.c:154
       seq_read_iter+0x3d0/0xd60 fs/seq_file.c:225
       copy_splice_read+0x662/0xb60 fs/splice.c:365
       do_splice_read fs/splice.c:985 [inline]
       splice_file_to_pipe+0x299/0x500 fs/splice.c:1295
       do_sendfile+0x515/0xe10 fs/read_write.c:1301
       __do_sys_sendfile64 fs/read_write.c:1362 [inline]
       __se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1348
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #1 (&p->lock){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       seq_read_iter+0xb7/0xd60 fs/seq_file.c:182
       copy_splice_read+0x662/0xb60 fs/splice.c:365
       do_splice_read fs/splice.c:985 [inline]
       splice_file_to_pipe+0x299/0x500 fs/splice.c:1295
       do_sendfile+0x515/0xe10 fs/read_write.c:1301
       __do_sys_sendfile64 fs/read_write.c:1362 [inline]
       __se_sys_sendfile64+0x17c/0x1e0 fs/read_write.c:1348
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&pipe->mutex){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3869
       __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       iter_file_splice_write+0x335/0x14e0 fs/splice.c:687
       do_splice_from fs/splice.c:941 [inline]
       do_splice+0xd77/0x1900 fs/splice.c:1354
       __do_splice fs/splice.c:1436 [inline]
       __do_sys_splice fs/splice.c:1652 [inline]
       __se_sys_splice+0x331/0x4a0 fs/splice.c:1634
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  &pipe->mutex --> &ovl_i_lock_key[depth] --> sb_writers#18

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  rlock(sb_writers#18);
                               lock(&ovl_i_lock_key[depth]);
                               lock(sb_writers#18);
  lock(&pipe->mutex);

 *** DEADLOCK ***

1 lock held by syz-executor.2/10146:
 #0: ffff88802b80e420 (sb_writers#18){.+.+}-{0:0}, at: do_splice+0xcf0/0x1900 fs/splice.c:1353

stack backtrace:
CPU: 1 PID: 10146 Comm: syz-executor.2 Not tainted 6.10.0-rc3-syzkaller-00044-g2ccbdf43d5e7 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3869
 __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
 __mutex_lock_common kernel/locking/mutex.c:608 [inline]
 __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
 iter_file_splice_write+0x335/0x14e0 fs/splice.c:687
 do_splice_from fs/splice.c:941 [inline]
 do_splice+0xd77/0x1900 fs/splice.c:1354
 __do_splice fs/splice.c:1436 [inline]
 __do_sys_splice fs/splice.c:1652 [inline]
 __se_sys_splice+0x331/0x4a0 fs/splice.c:1634
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fa2a0c7cf29
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fa2a07ff0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000113
RAX: ffffffffffffffda RBX: 00007fa2a0db3f80 RCX: 00007fa2a0c7cf29
RDX: 0000000000000009 RSI: 0000000000000000 RDI: 0000000000000008
RBP: 00007fa2a0cec074 R08: 00000000000001f0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fa2a0db3f80 R15: 00007ffd7bf98bc8
 </TASK>

Crashes (11):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/06/18 07:55 upstream 2ccbdf43d5e7 ce6011bc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/17 02:29 upstream 2ccbdf43d5e7 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/06 03:36 upstream 2df0193e62cf 121701b6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/06 01:20 upstream 71d7b52cc33b 5aa1a7c9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/05 21:34 upstream 71d7b52cc33b 5aa1a7c9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/05 00:36 upstream 32f88d65f01b e1e2c66e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/04 00:37 upstream f06ce441457d a1feae05 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/03 16:09 upstream c3f38fa61af7 0aba2352 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/06/02 15:18 upstream 83814698cf48 3113787f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/05/31 22:40 upstream d8ec19857b09 3113787f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
2024/05/31 18:46 upstream 4a4be1ad3a6e 0c378259 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in iter_file_splice_write
* Struck through repros no longer work on HEAD.