syzbot


possible deadlock in do_user_addr_fault

Status: fixed on 2023/10/12 12:48
Subsystems: exfat
[Documentation on labels]
Reported-by: syzbot+278098b0faaf0595072b@syzkaller.appspotmail.com
Fix commit: ff84772fd45d exfat: release s_lock before calling dir_emit()
First crash: 607d, last: 512d
Cause bisection: introduced by (bisect log) [merge commit]:
commit 6b6dc4f40c5264556223ba94693f20d83796ab1f
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date: Sun Sep 5 17:50:12 2021 +0000

  Merge tag 'mtd/for-5.15' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux

Crash: possible deadlock in do_user_addr_fault (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit ff84772fd45d486e4fc78c82e2f70ce5333543e6
Author: Sungjong Seo <sj1557.seo@samsung.com>
Date: Fri Jul 14 08:43:54 2023 +0000

  exfat: release s_lock before calling dir_emit()

  
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] [fat?] possible deadlock in do_user_addr_fault 1 (4) 2023/08/08 11:00
[syzbot] Monthly fat report (Jul 2023) 3 (4) 2023/07/14 23:29
Cause bisection attempts (2)
Created Duration User Patch Repo Result
2023/06/22 09:50 8h47m bisect upstream OK (1) job log log
2023/06/06 19:21 3h39m bisect upstream error job log

Sample crash report:
F2FS-fs (loop0): Mounted with checkpoint version = 48b305e5
======================================================
WARNING: possible circular locking dependency detected
6.4.0-rc7-next-20230621-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.0/5221 is trying to acquire lock:
ffff888079f13120 (&mm->mmap_lock){++++}-{3:3}, at: mmap_read_lock include/linux/mmap_lock.h:142 [inline]
ffff888079f13120 (&mm->mmap_lock){++++}-{3:3}, at: do_user_addr_fault+0xb3d/0x1210 arch/x86/mm/fault.c:1391

but task is already holding lock:
ffff888068134b90 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:771 [inline]
ffff888068134b90 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: f2fs_file_write_iter+0x297/0x2500 fs/f2fs/file.c:4744

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}:
       down_write+0x92/0x200 kernel/locking/rwsem.c:1573
       inode_lock include/linux/fs.h:771 [inline]
       f2fs_file_mmap+0x154/0x290 fs/f2fs/file.c:527
       call_mmap include/linux/fs.h:1876 [inline]
       mmap_region+0x6cf/0x2570 mm/mmap.c:2669
       do_mmap+0x850/0xee0 mm/mmap.c:1373
       vm_mmap_pgoff+0x1a2/0x3b0 mm/util.c:543
       ksys_mmap_pgoff+0x42b/0x5b0 mm/mmap.c:1419
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&mm->mmap_lock){++++}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3142 [inline]
       check_prevs_add kernel/locking/lockdep.c:3261 [inline]
       validate_chain kernel/locking/lockdep.c:3876 [inline]
       __lock_acquire+0x2e9d/0x5e20 kernel/locking/lockdep.c:5144
       lock_acquire.part.0+0x11c/0x370 kernel/locking/lockdep.c:5761
       down_read+0x9c/0x480 kernel/locking/rwsem.c:1520
       mmap_read_lock include/linux/mmap_lock.h:142 [inline]
       do_user_addr_fault+0xb3d/0x1210 arch/x86/mm/fault.c:1391
       handle_page_fault arch/x86/mm/fault.c:1534 [inline]
       exc_page_fault+0x98/0x170 arch/x86/mm/fault.c:1590
       asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:570
       fault_in_readable+0x1a5/0x210 mm/gup.c:1921
       fault_in_iov_iter_readable+0x252/0x2c0 lib/iov_iter.c:216
       f2fs_preallocate_blocks fs/f2fs/file.c:4520 [inline]
       f2fs_file_write_iter+0x516/0x2500 fs/f2fs/file.c:4756
       call_write_iter include/linux/fs.h:1871 [inline]
       new_sync_write fs/read_write.c:491 [inline]
       vfs_write+0x981/0xda0 fs/read_write.c:584
       ksys_write+0x122/0x250 fs/read_write.c:637
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&sb->s_type->i_mutex_key#21);
                               lock(&mm->mmap_lock);
                               lock(&sb->s_type->i_mutex_key#21);
  rlock(&mm->mmap_lock);

 *** DEADLOCK ***

3 locks held by syz-executor.0/5221:
 #0: ffff88802bf0cfc8 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xd7/0xf0 fs/file.c:1047
 #1: ffff888028bc6410 (sb_writers#13){.+.+}-{0:0}, at: ksys_write+0x122/0x250 fs/read_write.c:637
 #2: ffff888068134b90 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:771 [inline]
 #2: ffff888068134b90 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: f2fs_file_write_iter+0x297/0x2500 fs/f2fs/file.c:4744

stack backtrace:
CPU: 0 PID: 5221 Comm: syz-executor.0 Not tainted 6.4.0-rc7-next-20230621-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
 check_noncircular+0x2df/0x3b0 kernel/locking/lockdep.c:2195
 check_prev_add kernel/locking/lockdep.c:3142 [inline]
 check_prevs_add kernel/locking/lockdep.c:3261 [inline]
 validate_chain kernel/locking/lockdep.c:3876 [inline]
 __lock_acquire+0x2e9d/0x5e20 kernel/locking/lockdep.c:5144
 lock_acquire.part.0+0x11c/0x370 kernel/locking/lockdep.c:5761
 down_read+0x9c/0x480 kernel/locking/rwsem.c:1520
 mmap_read_lock include/linux/mmap_lock.h:142 [inline]
 do_user_addr_fault+0xb3d/0x1210 arch/x86/mm/fault.c:1391
 handle_page_fault arch/x86/mm/fault.c:1534 [inline]
 exc_page_fault+0x98/0x170 arch/x86/mm/fault.c:1590
 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:570
RIP: 0010:fault_in_readable+0x1a5/0x210 mm/gup.c:1921
Code: fc ff df 48 c7 04 02 00 00 00 00 48 83 c4 48 4c 89 e0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 45 31 e4 eb ce e8 ce 94 c3 ff 45 31 f6 <41> 8a 45 00 31 ff 44 89 f6 88 44 24 28 e8 d9 90 c3 ff 45 85 f6 75
RSP: 0018:ffffc9000418fb38 EFLAGS: 00050246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff888021961dc0 RSI: ffffffff81c0f242 RDI: 0000000000000007
RBP: 0000000020010127 R08: 0000000000000007 R09: 0000000000000000
R10: 0000000000000280 R11: 0000000000000001 R12: 000000000000fea7
R13: 0000000020000280 R14: 0000000000000000 R15: 1ffff92000831f68
 fault_in_iov_iter_readable+0x252/0x2c0 lib/iov_iter.c:216
 f2fs_preallocate_blocks fs/f2fs/file.c:4520 [inline]
 f2fs_file_write_iter+0x516/0x2500 fs/f2fs/file.c:4756
 call_write_iter include/linux/fs.h:1871 [inline]
 new_sync_write fs/read_write.c:491 [inline]
 vfs_write+0x981/0xda0 fs/read_write.c:584
 ksys_write+0x122/0x250 fs/read_write.c:637
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f224ea8c389
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 f1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f224f83d168 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007f224ebabf80 RCX: 00007f224ea8c389
RDX: 000000000000fea7 RSI: 0000000020000280 RDI: 0000000000000004
RBP: 00007f224ead7493 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007fff79fc4acf R14: 00007f224f83d300 R15: 0000000000022000
 </TASK>
----------------
Code disassembly (best guess), 2 bytes skipped:
   0:	df 48 c7             	fisttps -0x39(%rax)
   3:	04 02                	add    $0x2,%al
   5:	00 00                	add    %al,(%rax)
   7:	00 00                	add    %al,(%rax)
   9:	48 83 c4 48          	add    $0x48,%rsp
   d:	4c 89 e0             	mov    %r12,%rax
  10:	5b                   	pop    %rbx
  11:	5d                   	pop    %rbp
  12:	41 5c                	pop    %r12
  14:	41 5d                	pop    %r13
  16:	41 5e                	pop    %r14
  18:	41 5f                	pop    %r15
  1a:	c3                   	retq
  1b:	45 31 e4             	xor    %r12d,%r12d
  1e:	eb ce                	jmp    0xffffffee
  20:	e8 ce 94 c3 ff       	callq  0xffc394f3
  25:	45 31 f6             	xor    %r14d,%r14d
* 28:	41 8a 45 00          	mov    0x0(%r13),%al <-- trapping instruction
  2c:	31 ff                	xor    %edi,%edi
  2e:	44 89 f6             	mov    %r14d,%esi
  31:	88 44 24 28          	mov    %al,0x28(%rsp)
  35:	e8 d9 90 c3 ff       	callq  0xffc39113
  3a:	45 85 f6             	test   %r14d,%r14d
  3d:	75                   	.byte 0x75

Crashes (39):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/06/21 18:02 linux-next 15e71592dbae 09ffe269 .config console log report syz [disk image] [vmlinux] [kernel image] [mounted in repro] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/06 08:22 upstream f8dba31b0a82 a4ae4f42 .config strace log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci-upstream-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/18 04:28 upstream 1b29d271614a f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in do_user_addr_fault
2023/03/28 15:41 upstream 3a93e40326c8 48c74771 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in do_user_addr_fault
2023/03/28 15:38 upstream 3a93e40326c8 48c74771 .config console log report info ci-upstream-kasan-gce-selinux-root possible deadlock in do_user_addr_fault
2023/03/28 12:55 upstream 3a93e40326c8 48c74771 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/18 05:45 upstream 1b29d271614a f3921d4d .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_user_addr_fault
2023/06/27 13:14 linux-next 60e7c4a25da6 4cd5bb25 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 10:00 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 08:27 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 04:10 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 04:10 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 03:58 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/25 03:09 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/24 21:26 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/23 21:00 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/23 17:34 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/23 12:17 linux-next 8d2be868b42c 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/23 03:43 linux-next c87d46a9e8eb 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/22 13:16 linux-next c87d46a9e8eb 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/22 05:58 linux-next 15e71592dbae 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/22 01:02 linux-next 15e71592dbae 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/21 12:10 linux-next 15e71592dbae 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/21 05:10 linux-next f7efed9f38f8 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/20 21:54 linux-next f7efed9f38f8 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/20 14:24 linux-next f7efed9f38f8 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/20 13:45 linux-next f7efed9f38f8 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/19 03:35 linux-next f7efed9f38f8 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/18 23:30 linux-next f7efed9f38f8 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/18 08:31 linux-next f7efed9f38f8 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/17 11:50 linux-next f7efed9f38f8 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/17 01:34 linux-next f7efed9f38f8 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/16 00:04 linux-next 925294c9aa18 757d26ed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/14 17:00 linux-next b16049b21162 d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/14 15:57 linux-next b16049b21162 d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/14 15:33 linux-next b16049b21162 d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/14 15:22 linux-next b16049b21162 d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/06/14 09:00 linux-next b16049b21162 d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
2023/03/24 19:45 linux-next e5dbf24e8b9e 9700afae .config console log report info ci-upstream-linux-next-kasan-gce-root possible deadlock in do_user_addr_fault
* Struck through repros no longer work on HEAD.