syzbot


possible deadlock in lock_trace (3)

Status: fixed on 2020/08/18 22:40
Subsystems: fs
[Documentation on labels]
Reported-by: syzbot+985c8a3c7f3668d31a49@syzkaller.appspotmail.com
Fix commit: 2db9dbf71bf9 proc: Use new infrastructure to fix deadlocks in execve
First crash: 1616d, last: 1477d
Cause bisection: the cause commit could be any of (bisect log):
  9211bfbff80a netfilter: add missing IS_ENABLED(CONFIG_BRIDGE_NETFILTER) checks to header-file.
  47e640af2e49 netfilter: add missing IS_ENABLED(CONFIG_NF_TABLES) check to header-file.
  a1b2f04ea527 netfilter: add missing includes to a number of header-files.
  0abc8bf4f284 netfilter: add missing IS_ENABLED(CONFIG_NF_CONNTRACK) checks to some header-files.
  bd96b4c75675 netfilter: inline four headers files into another one.
  43dd16efc7f2 netfilter: nf_tables: store data in offload context registers
  78458e3e08cd netfilter: add missing IS_ENABLED(CONFIG_NETFILTER) checks to some header-files.
  20a9379d9a03 netfilter: remove "#ifdef __KERNEL__" guards from some headers.
  bd8699e9e292 netfilter: nft_bitwise: add offload support
  2a475c409fe8 kbuild: remove all netfilter headers from header-test blacklist.
  7e59b3fea2a2 netfilter: remove unnecessary spaces
  1b90af292e71 ipvs: Improve robustness to the ipvs sysctl
  5785cf15fd74 netfilter: nf_tables: add missing prototypes.
  0a30ba509fde netfilter: nf_nat_proto: make tables static
  e84fb4b3666d netfilter: conntrack: use shared sysctl constants
  105333435b4f netfilter: connlabels: prefer static lock initialiser
  8c0bb7873815 netfilter: synproxy: rename mss synproxy_options field
  c162610c7db2 Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next
  
Fix bisection: fixed by (bisect log) :
commit 2db9dbf71bf98d02a0bf33e798e5bfd2a9944696
Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
Date: Fri Mar 20 20:27:24 2020 +0000

  proc: Use new infrastructure to fix deadlocks in execve

  
Discussions (1)
Title Replies (including bot) Last reply
possible deadlock in lock_trace (3) 0 (3) 2020/05/02 10:07
Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 possible deadlock in lock_trace syz error 42 1311d 1817d 0/1 upstream: reported syz repro on 2019/04/28 18:16
linux-4.14 possible deadlock in lock_trace C error 146 1369d 1826d 0/1 upstream: reported C repro on 2019/04/19 09:51
android-49 possible deadlock in lock_trace C 7424 1597d 1833d 0/3 public: reported C repro on 2019/04/12 00:00
upstream possible deadlock in lock_trace (2) fs 3 1798d 1828d 0/26 auto-closed as invalid on 2019/10/25 08:41
upstream possible deadlock in lock_trace fs C 1847 1850d 2331d 0/26 closed as dup on 2017/12/12 21:59
android-414 possible deadlock in lock_trace syz 203 1842d 1833d 0/1 public: reported syz repro on 2019/04/12 00:01
android-44 possible deadlock in lock_trace C 41 1604d 1834d 0/2 public: reported C repro on 2019/04/11 08:44

Sample crash report:
overlayfs: './file0' not a directory
======================================================
WARNING: possible circular locking dependency detected
5.6.0-rc5-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.3/11170 is trying to acquire lock:
ffff888097db9c50 (&sig->cred_guard_mutex){+.+.}, at: lock_trace+0x45/0xe0 fs/proc/base.c:408

but task is already holding lock:
ffff8880913eebe0 (&p->lock){+.+.}, at: seq_read+0x6b/0x1160 fs/seq_file.c:161

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&p->lock){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:956 [inline]
       __mutex_lock+0x156/0x13c0 kernel/locking/mutex.c:1103
       seq_read+0x6b/0x1160 fs/seq_file.c:161
       do_loop_readv_writev fs/read_write.c:714 [inline]
       do_loop_readv_writev fs/read_write.c:701 [inline]
       do_iter_read+0x47f/0x650 fs/read_write.c:935
       vfs_readv+0xf0/0x160 fs/read_write.c:1053
       kernel_readv fs/splice.c:365 [inline]
       default_file_splice_read+0x4fb/0xa20 fs/splice.c:422
       do_splice_to+0x10e/0x160 fs/splice.c:892
       splice_direct_to_actor+0x307/0x980 fs/splice.c:971
       do_splice_direct+0x1a8/0x270 fs/splice.c:1080
       do_sendfile+0x549/0xc40 fs/read_write.c:1520
       __do_sys_sendfile64 fs/read_write.c:1581 [inline]
       __se_sys_sendfile64 fs/read_write.c:1567 [inline]
       __x64_sys_sendfile64+0x1cc/0x210 fs/read_write.c:1567
       do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:294
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #2 (sb_writers#4){.+.+}:
       percpu_down_read include/linux/percpu-rwsem.h:40 [inline]
       __sb_start_write+0x21b/0x430 fs/super.c:1674
       sb_start_write include/linux/fs.h:1650 [inline]
       mnt_want_write+0x3a/0xb0 fs/namespace.c:354
       ovl_setattr+0xdd/0x8b0 fs/overlayfs/inode.c:27
       notify_change+0xb6d/0x1060 fs/attr.c:336
       do_truncate+0x134/0x1f0 fs/open.c:64
       handle_truncate fs/namei.c:3083 [inline]
       do_last fs/namei.c:3496 [inline]
       path_openat+0x291f/0x32b0 fs/namei.c:3607
       do_filp_open+0x192/0x260 fs/namei.c:3637
       do_sys_openat2+0x54c/0x740 fs/open.c:1146
       do_sys_open+0xc3/0x140 fs/open.c:1162
       do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:294
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #1 (&ovl_i_mutex_key[depth]){+.+.}:
       down_write+0x8d/0x150 kernel/locking/rwsem.c:1534
       inode_lock include/linux/fs.h:792 [inline]
       process_measurement+0x68a/0x1740 security/integrity/ima/ima_main.c:230
       ima_file_check+0xb9/0x100 security/integrity/ima/ima_main.c:442
       do_last fs/namei.c:3494 [inline]
       path_openat+0x127f/0x32b0 fs/namei.c:3607
       do_filp_open+0x192/0x260 fs/namei.c:3637
       do_open_execat+0x122/0x600 fs/exec.c:860
       __do_execve_file.isra.0+0x16d5/0x2270 fs/exec.c:1765
       do_execveat_common fs/exec.c:1871 [inline]
       do_execve fs/exec.c:1888 [inline]
       __do_sys_execve fs/exec.c:1964 [inline]
       __se_sys_execve fs/exec.c:1959 [inline]
       __x64_sys_execve+0x8a/0xb0 fs/exec.c:1959
       do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:294
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (&sig->cred_guard_mutex){+.+.}:
       check_prev_add kernel/locking/lockdep.c:2475 [inline]
       check_prevs_add kernel/locking/lockdep.c:2580 [inline]
       validate_chain kernel/locking/lockdep.c:2970 [inline]
       __lock_acquire+0x201b/0x3ca0 kernel/locking/lockdep.c:3954
       lock_acquire+0x197/0x420 kernel/locking/lockdep.c:4484
       __mutex_lock_common kernel/locking/mutex.c:956 [inline]
       __mutex_lock+0x156/0x13c0 kernel/locking/mutex.c:1103
       lock_trace+0x45/0xe0 fs/proc/base.c:408
       proc_pid_personality+0x17/0xc0 fs/proc/base.c:3052
       proc_single_show+0xf7/0x1c0 fs/proc/base.c:758
       seq_read+0x4b9/0x1160 fs/seq_file.c:229
       do_loop_readv_writev fs/read_write.c:714 [inline]
       do_loop_readv_writev fs/read_write.c:701 [inline]
       do_iter_read+0x47f/0x650 fs/read_write.c:935
       vfs_readv+0xf0/0x160 fs/read_write.c:1053
       kernel_readv fs/splice.c:365 [inline]
       default_file_splice_read+0x4fb/0xa20 fs/splice.c:422
       do_splice_to+0x10e/0x160 fs/splice.c:892
       splice_direct_to_actor+0x307/0x980 fs/splice.c:971
       do_splice_direct+0x1a8/0x270 fs/splice.c:1080
       do_sendfile+0x549/0xc40 fs/read_write.c:1520
       __do_sys_sendfile64 fs/read_write.c:1581 [inline]
       __se_sys_sendfile64 fs/read_write.c:1567 [inline]
       __x64_sys_sendfile64+0x1cc/0x210 fs/read_write.c:1567
       do_syscall_64+0xf6/0x7d0 arch/x86/entry/common.c:294
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

Chain exists of:
  &sig->cred_guard_mutex --> sb_writers#4 --> &p->lock

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&p->lock);
                               lock(sb_writers#4);
                               lock(&p->lock);
  lock(&sig->cred_guard_mutex);

 *** DEADLOCK ***

2 locks held by syz-executor.3/11170:
 #0: ffff88809fa24428 (sb_writers#13){.+.+}, at: file_start_write include/linux/fs.h:2904 [inline]
 #0: ffff88809fa24428 (sb_writers#13){.+.+}, at: do_sendfile+0x939/0xc40 fs/read_write.c:1519
 #1: ffff8880913eebe0 (&p->lock){+.+.}, at: seq_read+0x6b/0x1160 fs/seq_file.c:161

stack backtrace:
CPU: 0 PID: 11170 Comm: syz-executor.3 Not tainted 5.6.0-rc5-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x188/0x20d lib/dump_stack.c:118
 check_noncircular+0x32e/0x3e0 kernel/locking/lockdep.c:1808
 check_prev_add kernel/locking/lockdep.c:2475 [inline]
 check_prevs_add kernel/locking/lockdep.c:2580 [inline]
 validate_chain kernel/locking/lockdep.c:2970 [inline]
 __lock_acquire+0x201b/0x3ca0 kernel/locking/lockdep.c:3954
 lock_acqu

Crashes (69):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/03/16 07:25 upstream a42a7bb6f536 749688d2 .config console log report syz ci-upstream-kasan-gce-selinux-root
2020/03/25 00:42 linux-next 770fbb32d34e 68660b21 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2020/04/02 06:03 upstream 919dce24701f a34e2c33 .config console log report ci-upstream-kasan-gce-root
2020/03/31 03:57 upstream 673b41e04a03 c8d1cc20 .config console log report ci-upstream-kasan-gce-root
2020/03/26 16:50 upstream 1b649e0bcae7 6d25c5a0 .config console log report ci-upstream-kasan-gce-root
2020/03/25 19:24 upstream e2cf67f6689a 41f049cc .config console log report ci-upstream-kasan-gce-root
2020/03/20 17:01 upstream 6c90b86a745a 2c31c529 .config console log report ci-upstream-kasan-gce-root
2020/03/20 08:04 upstream cd607737f3b8 2c31c529 .config console log report ci-upstream-kasan-gce-selinux-root
2020/03/19 10:30 upstream 5076190daded 2c31c529 .config console log report ci-upstream-kasan-gce-root
2020/03/15 10:12 upstream d3dca69085e9 749688d2 .config console log report ci-upstream-kasan-gce-root
2020/03/14 19:01 upstream 69a4d0baeeb1 749688d2 .config console log report ci-upstream-kasan-gce-selinux-root
2020/03/14 15:52 upstream fffb08b37df9 749688d2 .config console log report ci-upstream-kasan-gce-root
2020/03/10 04:36 upstream 30bb5572ce7a 35f53e45 .config console log report ci-upstream-kasan-gce-root
2020/03/08 15:12 upstream c20037652700 2e9971bb .config console log report ci-upstream-kasan-gce-root
2020/03/01 18:45 upstream 63623fd44972 c88c7b75 .config console log report ci-upstream-kasan-gce-root
2020/02/11 03:28 upstream 0a679e13ea30 084454ae .config console log report ci-upstream-kasan-gce-root
2020/02/04 06:48 upstream 322bf2d3446a 93e5e335 .config console log report ci-upstream-kasan-gce-selinux-root
2020/02/03 16:50 upstream 754beeec1d90 93e5e335 .config console log report ci-upstream-kasan-gce-selinux-root
2020/02/03 16:22 upstream 754beeec1d90 93e5e335 .config console log report ci-upstream-kasan-gce-root
2020/02/03 11:25 upstream 46d6b7becb1d 93e5e335 .config console log report ci-upstream-kasan-gce-root
2020/01/31 19:25 upstream ccaaaf6fe5a5 c30117b2 .config console log report ci-upstream-kasan-gce-root
2020/01/31 14:21 upstream ccaaaf6fe5a5 5ed23f9a .config console log report ci-upstream-kasan-gce-root
2020/01/23 13:46 upstream 131701c697e8 3334d684 .config console log report ci-upstream-kasan-gce-root
2020/01/19 16:23 upstream 8f8972a3127f 0342f8c7 .config console log report ci-upstream-kasan-gce-root
2020/01/19 01:04 upstream 244dc2689085 bc8bc756 .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/17 20:25 upstream ab7541c3addd 3de7aabb .config console log report ci-upstream-kasan-gce-root
2020/01/17 06:47 upstream f4353c3e2aaf 3de7aabb .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/17 06:11 upstream f4353c3e2aaf 3de7aabb .config console log report ci-upstream-kasan-gce-root
2020/01/17 06:10 upstream f4353c3e2aaf 3de7aabb .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/17 06:03 upstream f4353c3e2aaf 3de7aabb .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/15 16:32 upstream 95e20af9fb9c 069a5a44 .config console log report ci-upstream-kasan-gce-root
2020/01/15 16:15 upstream 95e20af9fb9c 069a5a44 .config console log report ci-upstream-kasan-gce-root
2020/01/15 13:50 upstream 95e20af9fb9c fa12bd3c .config console log report ci-upstream-kasan-gce-root
2020/01/14 20:16 upstream e033e7d4a808 fa12bd3c .config console log report ci-upstream-kasan-gce-root
2020/01/13 12:31 upstream b3a987b0264d 99565c1a .config console log report ci-upstream-kasan-gce-root
2020/01/13 12:05 upstream b3a987b0264d 99565c1a .config console log report ci-upstream-kasan-gce-root
2020/01/13 09:49 upstream 040a3c33623b 53faa9fe .config console log report ci-upstream-kasan-gce-root
2020/01/12 12:40 upstream 6327edceb62b 31290a45 .config console log report ci-upstream-kasan-gce-root
2020/01/12 12:27 upstream 6327edceb62b 31290a45 .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/12 12:17 upstream 6327edceb62b 31290a45 .config console log report ci-upstream-kasan-gce-root
2020/01/12 04:22 upstream ac61145a725a 4c04afaa .config console log report ci-upstream-kasan-gce-root
2020/01/12 04:10 upstream ac61145a725a 4c04afaa .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/12 03:21 upstream ac61145a725a 4c04afaa .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/11 22:20 upstream bef1d88263ff 4c04afaa .config console log report ci-upstream-kasan-gce-root
2019/12/26 22:31 upstream 46cf053efec6 be5c2c81 .config console log report ci-upstream-kasan-gce-root
2019/12/26 22:00 upstream 46cf053efec6 be5c2c81 .config console log report ci-upstream-kasan-gce-selinux-root
2019/12/02 17:49 upstream 596cf45cbf6e ab342da3 .config console log report ci-qemu-upstream
2020/03/14 16:59 linux-next 770fbb32d34e 749688d2 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/03/07 15:03 linux-next 770fbb32d34e 2e9971bb .config console log report ci-upstream-linux-next-kasan-gce-root
2020/03/06 16:53 linux-next c99b17ac0399 c88c7b75 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/03/05 09:09 linux-next c99b17ac0399 c88c7b75 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/03/03 09:38 linux-next c99b17ac0399 c88c7b75 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/29 22:27 linux-next bdc5461b23ca 59b57593 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/28 06:38 linux-next bdc5461b23ca 59b57593 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/27 22:45 linux-next bdc5461b23ca 59b57593 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/20 08:00 linux-next f4aba10148cd 81230308 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/18 16:07 linux-next 02815e777db6 012fbc32 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/15 19:59 linux-next 9f01828e9e16 5d7b90f1 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/14 03:24 linux-next 9f01828e9e16 c5ed587f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/13 21:50 linux-next 51d5d207918d c5ed587f .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/12 03:20 linux-next ac431e2d7b1b a75b198c .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/10 15:37 linux-next 2981de746b18 18847f55 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 05:32 linux-next 6dff1565d69c 06150bf1 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/08 18:35 linux-next 6dff1565d69c 06150bf1 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/01/12 12:07 linux-next 6c09d7dbb7d3 31290a45 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/12/30 11:10 linux-next 7ddd09fc4b74 af6b8ef8 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/12/21 02:17 linux-next 7ddd09fc4b74 bc586918 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/11/15 10:46 linux-next 5a6fcbeabe3e 79248ee8 .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.