syzbot


possible deadlock in mnt_want_write

Status: fixed on 2020/11/16 12:12
Subsystems: fs
[Documentation on labels]
Reported-by: syzbot+ae82084b07d0297e566b@syzkaller.appspotmail.com
Fix commit: 146d62e5a586 ovl: detect overlapping layers
First crash: 2075d, last: 1267d
Cause bisection: introduced by (bisect log) :
commit 8e54cadab447dae779f80f79c87cbeaea9594f60
Author: Al Viro <viro@zeniv.linux.org.uk>
Date: Sun Nov 27 01:05:42 2016 +0000

  fix default_file_splice_read()

Crash: INFO: possible circular locking dependency detected ] (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit 146d62e5a5867fbf84490d82455718bfb10fe824
Author: Amir Goldstein <amir73il@gmail.com>
Date: Thu Apr 18 14:42:08 2019 +0000

  ovl: detect overlapping layers

  
Discussions (5)
Title Replies (including bot) Last reply
Re: [syzbot] possible deadlock in ovl_maybe_copy_up 3 (3) 2021/06/22 11:40
Re: [syzbot] possible deadlock in ovl_maybe_copy_up 1 (1) 2021/06/22 08:53
possible deadlock in ovl_maybe_copy_up 1 (3) 2021/04/04 08:10
possible deadlock in mnt_want_write 4 (9) 2020/11/11 13:52
[PATCH] fs: Evaluate O_WRONLY | O_RDWR to O_RDWR 6 (6) 2018/12/13 14:05
Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 possible deadlock in mnt_want_write origin:upstream missing-backport C done 117 120d 373d 0/3 upstream: reported C repro on 2023/03/21 10:01
linux-6.1 possible deadlock in mnt_want_write origin:upstream missing-backport C done 56 122d 365d 0/3 upstream: reported C repro on 2023/03/28 13:05
linux-4.19 possible deadlock in mnt_want_write romfs C 730 387d 1804d 0/1 upstream: reported C repro on 2019/04/19 16:54
android-49 possible deadlock in mnt_want_write 1 2057d 2057d 0/3 auto-closed as invalid on 2019/02/22 14:57
upstream possible deadlock in mnt_want_write (2) integrity overlayfs C done 867 148d 1016d 25/26 fixed on 2023/12/21 01:43
upstream possible deadlock in mnt_want_write (3) kernfs 3 18d 87d 0/26 upstream: reported on 2024/01/01 07:31
linux-4.14 possible deadlock in mnt_want_write ubifs C 10467 387d 1799d 0/1 upstream: reported C repro on 2019/04/25 05:09

Sample crash report:
overlayfs: workdir and upperdir must reside under the same mount
======================================================
WARNING: possible circular locking dependency detected
5.1.0-rc5+ #74 Not tainted
------------------------------------------------------
syz-executor520/11107 is trying to acquire lock:
00000000efa5fd95 (sb_writers#5){.+.+}, at: sb_start_write include/linux/fs.h:1621 [inline]
00000000efa5fd95 (sb_writers#5){.+.+}, at: mnt_want_write+0x3f/0xc0 fs/namespace.c:358

but task is already holding lock:
00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: inode_lock include/linux/fs.h:772 [inline]
00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: vfs_rmdir fs/namei.c:3868 [inline]
00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: vfs_rmdir+0xe1/0x470 fs/namei.c:3857

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 (&ovl_i_mutex_dir_key[depth]#2){++++}:
       lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
       down_read+0x3b/0x90 kernel/locking/rwsem.c:24
       inode_lock_shared include/linux/fs.h:782 [inline]
       do_last fs/namei.c:3321 [inline]
       path_openat+0x1e98/0x46e0 fs/namei.c:3533
       do_filp_open+0x1a1/0x280 fs/namei.c:3563
       do_open_execat+0x137/0x690 fs/exec.c:856
       __do_execve_file.isra.0+0x178d/0x23f0 fs/exec.c:1758
       do_execveat_common fs/exec.c:1865 [inline]
       do_execve fs/exec.c:1882 [inline]
       __do_sys_execve fs/exec.c:1958 [inline]
       __se_sys_execve fs/exec.c:1953 [inline]
       __x64_sys_execve+0x8f/0xc0 fs/exec.c:1953
       do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #3 (&sig->cred_guard_mutex){+.+.}:
       lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
       __mutex_lock_common kernel/locking/mutex.c:925 [inline]
       __mutex_lock+0xf7/0x1310 kernel/locking/mutex.c:1072
       mutex_lock_killable_nested+0x16/0x20 kernel/locking/mutex.c:1102
       do_io_accounting+0x1f4/0x830 fs/proc/base.c:2740
       proc_tgid_io_accounting+0x23/0x30 fs/proc/base.c:2789
       proc_single_show+0xf6/0x170 fs/proc/base.c:744
       seq_read+0x4db/0x1130 fs/seq_file.c:229
       do_loop_readv_writev fs/read_write.c:701 [inline]
       do_loop_readv_writev fs/read_write.c:688 [inline]
       do_iter_read+0x4a9/0x660 fs/read_write.c:922
       vfs_readv+0xf0/0x160 fs/read_write.c:984
       kernel_readv fs/splice.c:358 [inline]
       default_file_splice_read+0x475/0x890 fs/splice.c:413
       do_splice_to+0x12a/0x190 fs/splice.c:876
       splice_direct_to_actor+0x2d2/0x970 fs/splice.c:953
       do_splice_direct+0x1da/0x2a0 fs/splice.c:1062
       do_sendfile+0x597/0xd00 fs/read_write.c:1443
       __do_sys_sendfile64 fs/read_write.c:1504 [inline]
       __se_sys_sendfile64 fs/read_write.c:1490 [inline]
       __x64_sys_sendfile64+0x1dd/0x220 fs/read_write.c:1490
       do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #2 (&p->lock){+.+.}:
       lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
       __mutex_lock_common kernel/locking/mutex.c:925 [inline]
       __mutex_lock+0xf7/0x1310 kernel/locking/mutex.c:1072
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
       seq_read+0x71/0x1130 fs/seq_file.c:161
       proc_reg_read+0x1fe/0x2c0 fs/proc/inode.c:227
       do_loop_readv_writev fs/read_write.c:701 [inline]
       do_loop_readv_writev fs/read_write.c:688 [inline]
       do_iter_read+0x4a9/0x660 fs/read_write.c:922
       vfs_readv+0xf0/0x160 fs/read_write.c:984
       kernel_readv fs/splice.c:358 [inline]
       default_file_splice_read+0x475/0x890 fs/splice.c:413
       do_splice_to+0x12a/0x190 fs/splice.c:876
       do_splice+0x110b/0x1420 fs/splice.c:1183
       __do_sys_splice fs/splice.c:1424 [inline]
       __se_sys_splice fs/splice.c:1404 [inline]
       __x64_sys_splice+0x2c6/0x330 fs/splice.c:1404
       do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #1 (&pipe->mutex/1){+.+.}:
       lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
       __mutex_lock_common kernel/locking/mutex.c:925 [inline]
       __mutex_lock+0xf7/0x1310 kernel/locking/mutex.c:1072
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
       pipe_lock_nested fs/pipe.c:62 [inline]
       pipe_lock+0x6e/0x80 fs/pipe.c:70
       iter_file_splice_write+0x18b/0xbe0 fs/splice.c:696
       do_splice_from fs/splice.c:847 [inline]
       do_splice+0x70a/0x1420 fs/splice.c:1154
       __do_sys_splice fs/splice.c:1424 [inline]
       __se_sys_splice fs/splice.c:1404 [inline]
       __x64_sys_splice+0x2c6/0x330 fs/splice.c:1404
       do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (sb_writers#5){.+.+}:
       check_prevs_add kernel/locking/lockdep.c:2333 [inline]
       validate_chain kernel/locking/lockdep.c:2714 [inline]
       __lock_acquire+0x239c/0x3fb0 kernel/locking/lockdep.c:3701
       lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
       percpu_down_read include/linux/percpu-rwsem.h:36 [inline]
       __sb_start_write+0x20b/0x360 fs/super.c:1613
       sb_start_write include/linux/fs.h:1621 [inline]
       mnt_want_write+0x3f/0xc0 fs/namespace.c:358
       ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
       ovl_do_remove+0xe9/0xd70 fs/overlayfs/dir.c:840
       ovl_rmdir+0x1b/0x20 fs/overlayfs/dir.c:890
       vfs_rmdir fs/namei.c:3878 [inline]
       vfs_rmdir+0x19c/0x470 fs/namei.c:3857
       do_rmdir+0x39e/0x420 fs/namei.c:3939
       __do_sys_rmdir fs/namei.c:3957 [inline]
       __se_sys_rmdir fs/namei.c:3955 [inline]
       __x64_sys_rmdir+0x36/0x40 fs/namei.c:3955
       do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Chain exists of:
  sb_writers#5 --> &sig->cred_guard_mutex --> &ovl_i_mutex_dir_key[depth]#2

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ovl_i_mutex_dir_key[depth]#2);
                               lock(&sig->cred_guard_mutex);
                               lock(&ovl_i_mutex_dir_key[depth]#2);
  lock(sb_writers#5);

 *** DEADLOCK ***

3 locks held by syz-executor520/11107:
 #0: 00000000e91bc594 (sb_writers#8){.+.+}, at: sb_start_write include/linux/fs.h:1621 [inline]
 #0: 00000000e91bc594 (sb_writers#8){.+.+}, at: mnt_want_write+0x3f/0xc0 fs/namespace.c:358
 #1: 00000000483c3c1a (&ovl_i_mutex_dir_key[depth]/1){+.+.}, at: inode_lock_nested include/linux/fs.h:807 [inline]
 #1: 00000000483c3c1a (&ovl_i_mutex_dir_key[depth]/1){+.+.}, at: do_rmdir+0x271/0x420 fs/namei.c:3927
 #2: 00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: inode_lock include/linux/fs.h:772 [inline]
 #2: 00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: vfs_rmdir fs/namei.c:3868 [inline]
 #2: 00000000cf4aa2b1 (&ovl_i_mutex_dir_key[depth]#2){++++}, at: vfs_rmdir+0xe1/0x470 fs/namei.c:3857

stack backtrace:
CPU: 0 PID: 11107 Comm: syz-executor520 Not tainted 5.1.0-rc5+ #74
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x172/0x1f0 lib/dump_stack.c:113
 print_circular_bug.isra.0.cold+0x1cc/0x28f kernel/locking/lockdep.c:1571
 check_prev_add.constprop.0+0xf11/0x23c0 kernel/locking/lockdep.c:2220
 check_prevs_add kernel/locking/lockdep.c:2333 [inline]
 validate_chain kernel/locking/lockdep.c:2714 [inline]
 __lock_acquire+0x239c/0x3fb0 kernel/locking/lockdep.c:3701
 lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
 percpu_down_read include/linux/percpu-rwsem.h:36 [inline]
 __sb_start_write+0x20b/0x360 fs/super.c:1613
 sb_start_write include/linux/fs.h:1621 [inline]
 mnt_want_write+0x3f/0xc0 fs/namespace.c:358
 ovl_want_write+0x76/0xa0 fs/overlayfs/util.c:24
 ovl_do_remove+0xe9/0xd70 fs/overlayfs/dir.c:840
 ovl_rmdir+0x1b/0x20 fs/overlayfs/dir.c:890
 vfs_rmdir fs/namei.c:3878 [inline]
 vfs_rmdir+0x19c/0x470 fs/namei.c:3857
 do_rmdir+0x39e/0x420 fs/namei.c:3939
 __do_sys_rmdir fs/namei.c:3957 [inline]
 __se_sys_rmdir fs/namei.c:3955 [inline]
 __x64_sys_rmdir+0x36/0x40 fs/namei.c:3955
 do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x447049
Code: e8 ec b9 02 00 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db 06 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fbbfb3f0db8 EFLAGS: 00000246 ORIG_RAX: 0000000000000054
RAX: ffffffffffffffda RBX: 00000000006dcc68 RCX: 0000000000447049
RDX: 0000000000447049 RSI: 0000000000000000 RDI: 0000000020000000
RBP: 00000000006dcc60 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dcc6c
R13: 00007ffea630b6df R14: 00007fbbfb3f19c0 R15: 0000000000000000

Crashes (662):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/04/19 05:55 upstream 6d906f998179 b0e8efcb .config console log report syz C ci-upstream-kasan-gce-smack-root
2018/11/21 16:23 linux-next 442b8cea2477 5d9a3924 .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2020/03/06 17:23 upstream 63623fd44972 c88c7b75 .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/04/26 17:09 upstream 8113a85f8720 b617407b .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/04/26 17:08 upstream 8113a85f8720 b617407b .config console log report syz ci-upstream-kasan-gce-smack-root
2019/04/21 07:55 upstream 9e5de623a0cb b0e8efcb .config console log report syz ci-upstream-kasan-gce-root
2019/04/21 06:02 upstream 9e5de623a0cb b0e8efcb .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/04/21 03:38 upstream 9e5de623a0cb b0e8efcb .config console log report syz ci-upstream-kasan-gce-root
2019/04/21 03:14 upstream 9e5de623a0cb b0e8efcb .config console log report syz ci-upstream-kasan-gce-smack-root
2019/04/21 01:09 upstream 9e5de623a0cb b0e8efcb .config console log report syz ci-upstream-kasan-gce-selinux-root
2018/11/22 05:19 upstream 92b419289cee 9db828b5 .config console log report syz ci-upstream-kasan-gce-selinux-root
2018/11/22 00:34 upstream 92b419289cee 9db828b5 .config console log report syz ci-upstream-kasan-gce-root
2018/11/21 23:56 upstream 92b419289cee 9db828b5 .config console log report syz ci-upstream-kasan-gce-root
2018/11/21 16:13 upstream c8ce94b8fe53 5d9a3924 .config console log report syz ci-upstream-kasan-gce-smack-root
2020/02/29 19:19 linux-next bdc5461b23ca 59b57593 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/04/26 17:10 linux-next 3ddfa8af5dc9 b617407b .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/04/22 09:55 linux-next 3f018f4a019a 0a77c33c .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/04/21 06:25 linux-next 3f018f4a019a b0e8efcb .config console log report syz ci-upstream-linux-next-kasan-gce-root
2018/11/21 20:44 linux-next 442b8cea2477 9db828b5 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2020/10/08 07:44 upstream c85fb28b6f99 1880b4a9 .config console log report info ci-upstream-kasan-gce-root
2020/10/07 10:56 upstream c85fb28b6f99 1880b4a9 .config console log report info ci-upstream-kasan-gce-root
2020/10/07 04:52 upstream c85fb28b6f99 1880b4a9 .config console log report info ci-upstream-kasan-gce-smack-root
2020/10/06 14:11 upstream 7575fdda569b 1880b4a9 .config console log report info ci-upstream-kasan-gce-root
2020/10/06 12:18 upstream 7575fdda569b 1880b4a9 .config console log report info ci-upstream-kasan-gce-root
2020/10/05 21:18 upstream 549738f15da0 1880b4a9 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/10/05 08:35 upstream 549738f15da0 5ef9c291 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/10/05 05:49 upstream 549738f15da0 5ef9c291 .config console log report info ci-upstream-kasan-gce-smack-root
2020/10/02 01:13 upstream fcadab740480 9602ddf4 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/10/01 12:53 upstream 60e720931556 a9767fb2 .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/30 04:05 upstream ccc1d052eff9 5abc3f1a .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/30 02:13 upstream ccc1d052eff9 5abc3f1a .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/29 03:20 upstream fb0155a09b02 1b88c6d5 .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/28 22:42 upstream a1b8638ba132 6bfdbe89 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/09/28 12:03 upstream a1b8638ba132 6bfdbe89 .config console log report info ci-upstream-kasan-gce-root
2020/09/28 06:37 upstream 16bc1d5432eb 5dd8aee8 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/09/27 19:21 upstream eeddbe6841cd 5dd8aee8 .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/27 17:36 upstream a1bffa48745a 5dd8aee8 .config console log report info ci-upstream-kasan-gce-selinux-root
2020/09/27 03:41 upstream eeddbe6841cd 2d5ea0cb .config console log report info ci-upstream-kasan-gce-selinux-root
2020/09/12 07:18 upstream e8878ab82545 79fb24e2 .config console log report ci-upstream-kasan-gce-root
2020/09/11 03:22 upstream 7fe10096c150 409809d8 .config console log report ci-upstream-kasan-gce-root
2020/09/11 01:28 upstream 7fe10096c150 409809d8 .config console log report ci-upstream-kasan-gce-root
2020/09/10 15:16 upstream 34d4ddd359db 409809d8 .config console log report ci-upstream-kasan-gce-smack-root
2020/09/10 13:45 upstream 34d4ddd359db 409809d8 .config console log report ci-upstream-kasan-gce-selinux-root
2020/09/10 02:29 upstream 34d4ddd359db 409809d8 .config console log report ci-upstream-kasan-gce-smack-root
2020/09/08 22:41 upstream 612ab8ad6414 abf9ba4f .config console log report ci-upstream-kasan-gce-selinux-root
2020/09/07 15:20 upstream f4d51dffc6c0 abf9ba4f .config console log report ci-upstream-kasan-gce-root
2020/09/06 08:37 upstream 9322c47b21b9 abf9ba4f .config console log report ci-upstream-kasan-gce-smack-root
2020/09/06 01:51 upstream 9322c47b21b9 abf9ba4f .config console log report ci-upstream-kasan-gce-selinux-root
2020/09/05 23:26 upstream 9322c47b21b9 abf9ba4f .config console log report ci-upstream-kasan-gce-smack-root
2020/09/05 22:02 upstream 9322c47b21b9 abf9ba4f .config console log report ci-upstream-kasan-gce-smack-root
2020/09/05 10:22 upstream c70672d8d316 abf9ba4f .config console log report ci-upstream-kasan-gce-root
2020/01/14 21:34 upstream 452424cdcbca fa12bd3c .config console log report ci-qemu-upstream
2018/08/10 00:19 upstream 112cbae26d18 1fb62d58 .config console log report ci-upstream-kasan-gce
2018/07/22 23:46 upstream 45ae4df92207 8cc079c3 .config console log report ci-upstream-kasan-gce
2018/08/10 00:18 upstream 112cbae26d18 1fb62d58 .config console log report ci-upstream-kasan-gce-386
2020/10/04 01:28 linux-next 2172e358cd17 1a3f9408 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/10/03 19:43 linux-next 2172e358cd17 2653fa43 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/10/01 19:10 linux-next d39294091fee a9767fb2 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/10/01 16:05 linux-next d39294091fee a9767fb2 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/09/30 11:01 linux-next 49e7e3e905e4 5abc3f1a .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/09/25 04:56 linux-next d1d2220c7f39 54289b08 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/09/11 05:01 linux-next 7ce53e3a447b 409809d8 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/08 08:40 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/07 21:22 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/06 13:27 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/06 09:59 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/06 00:29 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/09/05 06:57 linux-next 7a6956579ce6 abf9ba4f .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.