syzbot


possible deadlock in f2fs_handle_error

Status: upstream: reported on 2022/11/28 10:02
Reported-by: syzbot+fbc90a26c932581cfe6b@syzkaller.appspotmail.com
First crash: 70d, last: 3h49m

Sample crash report:
F2FS-fs (loop5): Disable nat_bits due to incorrect cp_ver (9621037545273099749, 1067266233009637)
F2FS-fs (loop5): f2fs_check_nid_range: out-of-range nid=2, run fsck to fix.
======================================================
WARNING: possible circular locking dependency detected
6.2.0-rc6-syzkaller-00258-gdb27c22251e7 #0 Not tainted
------------------------------------------------------
syz-executor.5/9232 is trying to acquire lock:
ffff88807761c088 (&sbi->sb_lock){++++}-{3:3}, at: f2fs_handle_error+0xf0/0x250

but task is already holding lock:
ffff88802a2f3130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_build_free_nids+0x71f/0x1190

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 (&nm_i->nat_tree_lock){++++}-{3:3}:
       lock_acquire+0x20b/0x600
       down_read+0x3d/0x50
       f2fs_get_node_info+0xf3/0x12d0
       __write_node_page+0x71f/0x27b0
       f2fs_sync_node_pages+0x104c/0x1690
       f2fs_write_checkpoint+0xe55/0x1ce0
       __checkpoint_and_complete_reqs+0xda/0x3b0
       issue_checkpoint_thread+0xda/0x250
       kthread+0x270/0x300
       ret_from_fork+0x1f/0x30

-> #3 (&sbi->node_change){++++}-{3:3}:
       lock_acquire+0x20b/0x600
       down_read+0x3d/0x50
       f2fs_vm_page_mkwrite+0x75c/0x1430
       do_page_mkwrite+0x1a1/0x600
       do_wp_page+0x506/0x3270
       handle_mm_fault+0x2388/0x51c0
       exc_page_fault+0x66a/0x880
       asm_exc_page_fault+0x26/0x30

-> #2 (mapping.invalidate_lock#6){++++}-{3:3}:
       lock_acquire+0x20b/0x600
       down_read+0x3d/0x50
       filemap_fault+0x617/0x17d0
       f2fs_filemap_fault+0x81/0x3d0
       __do_fault+0x136/0x500
       handle_mm_fault+0x32f3/0x51c0
       __get_user_pages+0x509/0x11e0
       populate_vma_page_range+0x21b/0x2b0
       __mm_populate+0x279/0x450
       vm_mmap_pgoff+0x212/0x2d0
       ksys_mmap_pgoff+0x4f9/0x6d0
       do_syscall_64+0x41/0xc0
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #1 (&mm->mmap_lock){++++}-{3:3}:
       lock_acquire+0x20b/0x600
       __might_fault+0xba/0x120
       _copy_to_user+0x2a/0x130
       __f2fs_ioctl+0xaccd/0xb2b0
       __se_sys_ioctl+0xf1/0x160
       do_syscall_64+0x41/0xc0
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&sbi->sb_lock){++++}-{3:3}:
       validate_chain+0x166b/0x5860
       __lock_acquire+0x125b/0x1f80
       lock_acquire+0x20b/0x600
       down_write+0x3a/0x60
       f2fs_handle_error+0xf0/0x250
       f2fs_build_free_nids+0xca3/0x1190
       f2fs_fill_super+0x46f3/0x6f30
       mount_bdev+0x271/0x3a0
       legacy_get_tree+0xef/0x190
       vfs_get_tree+0x8c/0x270
       do_new_mount+0x28f/0xae0
       __se_sys_mount+0x2c9/0x3b0
       do_syscall_64+0x41/0xc0
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Chain exists of:
  &sbi->sb_lock --> &sbi->node_change --> &nm_i->nat_tree_lock

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&nm_i->nat_tree_lock);
                               lock(&sbi->node_change);
                               lock(&nm_i->nat_tree_lock);
  lock(&sbi->sb_lock);

 *** DEADLOCK ***

3 locks held by syz-executor.5/9232:
 #0: ffff888078a200e0 (&type->s_umount_key#51/1){+.+.}-{3:3}, at: alloc_super+0x217/0x930
 #1: ffff88802a2f32c8 (&nm_i->build_lock){+.+.}-{3:3}, at: f2fs_build_free_nids+0xcc/0x1190
 #2: ffff88802a2f3130 (&nm_i->nat_tree_lock){++++}-{3:3}, at: f2fs_build_free_nids+0x71f/0x1190

stack backtrace:
CPU: 1 PID: 9232 Comm: syz-executor.5 Not tainted 6.2.0-rc6-syzkaller-00258-gdb27c22251e7 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/12/2023
Call Trace:
 <TASK>
 dump_stack_lvl+0x1b5/0x2a0
 check_noncircular+0x2d1/0x390
 validate_chain+0x166b/0x5860
 __lock_acquire+0x125b/0x1f80
 lock_acquire+0x20b/0x600
 down_write+0x3a/0x60
 f2fs_handle_error+0xf0/0x250
 f2fs_build_free_nids+0xca3/0x1190
 f2fs_fill_super+0x46f3/0x6f30
 mount_bdev+0x271/0x3a0
 legacy_get_tree+0xef/0x190
 vfs_get_tree+0x8c/0x270
 do_new_mount+0x28f/0xae0
 __se_sys_mount+0x2c9/0x3b0
 do_syscall_64+0x41/0xc0
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f228ce8d5fa
Code: 48 c7 c2 b8 ff ff ff f7 d8 64 89 02 b8 ff ff ff ff eb d2 e8 b8 04 00 00 0f 1f 84 00 00 00 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f228dc1af88 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 00000000000105fc RCX: 00007f228ce8d5fa
RDX: 0000000020010600 RSI: 0000000020000000 RDI: 00007f228dc1afe0
RBP: 00007f228dc1b020 R08: 00007f228dc1b020 R09: 0000000000000816
R10: 0000000000000816 R11: 0000000000000202 R12: 0000000020010600
R13: 0000000020000000 R14: 00007f228dc1afe0 R15: 0000000020000080
 </TASK>

Crashes (52):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Title
ci2-upstream-fs 2023/02/05 08:16 upstream db27c22251e7 be607b78 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-root 2023/02/05 04:57 upstream 95078069c1e7 be607b78 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/04 21:54 upstream db27c22251e7 be607b78 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-smack-root 2023/02/04 11:29 upstream 0136d86b7852 1b2f701a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/03 15:21 upstream 66a87fff1a87 1b2f701a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/03 03:06 upstream e7368fd30165 33fc5c09 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/02 06:25 upstream 9f266ccaa2f5 9a6f477c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/01 10:35 upstream c0b67534c95c 9a6f477c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/02/01 02:45 upstream 22b8077d0fce b68fb8d6 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/31 13:01 upstream 6d796c50f84c b68fb8d6 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/31 01:30 upstream 6d796c50f84c b68fb8d6 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-root 2023/01/28 21:52 upstream 5af6ce704936 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/28 07:29 upstream 83abd4d4c4be 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/27 16:46 upstream 7c46948a6e9c 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/27 01:05 upstream 7c46948a6e9c 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/26 02:29 upstream 7c46948a6e9c 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/26 02:29 upstream 7c46948a6e9c 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-smack-root 2023/01/25 21:00 upstream 948ef7bb70c4 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/25 12:00 upstream 948ef7bb70c4 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/23 13:15 upstream 2475bf0250de 44388686 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/23 08:29 upstream 2475bf0250de 44388686 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-smack-root 2023/01/23 04:31 upstream 2241ab53cbb5 559a440a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/22 22:38 upstream 2475bf0250de cc0f9968 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/21 13:51 upstream edc00350d205 cc0f9968 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/20 09:58 upstream d368967cb103 dd15ff29 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/18 10:31 upstream 6e50979a9c87 42660d9e .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/18 08:47 upstream 6e50979a9c87 42660d9e .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-selinux-root 2023/01/14 23:59 upstream 97ec4d559d93 a63719e7 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/14 11:59 upstream 97ec4d559d93 a63719e7 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/14 10:37 upstream 97ec4d559d93 529798b0 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/11 02:46 upstream 40c18f363a08 48bc529a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/09 22:02 upstream 1fe4fd6f5cad 48bc529a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/09 04:25 upstream 1fe4fd6f5cad 1dac8c7a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/05 21:58 upstream 41c03ba9beea 1dac8c7a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/04 21:35 upstream 69b41ac87e4a 1dac8c7a .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/02 07:47 upstream 88603b6dc419 ab32d508 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2023/01/02 07:46 upstream 88603b6dc419 ab32d508 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2022/12/31 12:07 upstream bff687b3dad6 ab32d508 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2022/12/30 21:41 upstream bff687b3dad6 ab32d508 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-root 2022/12/29 18:50 upstream 1b929c02afd3 44712fbc .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2022/12/29 05:26 upstream 1b929c02afd3 44712fbc .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2022/12/27 10:55 upstream 1b929c02afd3 9da18ae8 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci2-upstream-fs 2022/12/27 00:53 upstream 1b929c02afd3 9da18ae8 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-smack-root 2022/12/14 05:09 upstream 764822972d64 e660de91 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-kasan-gce-root 2022/12/13 00:24 upstream 830b3c68c1fb 67be1ae7 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-qemu-upstream-386 2023/01/17 01:04 upstream d532dd102151 a63719e7 .config console log report info possible deadlock in f2fs_handle_error
ci-qemu-upstream-386 2023/01/15 01:56 upstream 7c6984405241 a63719e7 .config console log report info possible deadlock in f2fs_handle_error
ci-upstream-gce-arm64 2023/01/23 16:57 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci edb2f0dc90f2 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-gce-arm64 2023/01/19 08:57 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 9598c377d828 42660d9e .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-gce-arm64 2022/12/21 13:32 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci a5541c0811a0 4067838e .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-gce-arm64 2022/12/10 04:35 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci a5541c0811a0 67be1ae7 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
ci-upstream-gce-arm64 2022/11/26 21:28 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6d464646530f 74a66371 .config console log report info [disk image] [vmlinux] [kernel image] possible deadlock in f2fs_handle_error
* Struck through repros no longer work on HEAD.