syzbot


possible deadlock in lookup_slow (2)

Status: upstream: reported C repro on 2023/12/13 22:35
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+3e6460be44aa33acd779@syzkaller.appspotmail.com
First crash: 139d, last: 20h53m
Bug presence (1)
Date Name Commit Repro Result
2024/03/04 upstream (ToT) 90d35da658da C [report] BUG: MAX_LOCKDEP_KEYS too low!
Similar bugs (6)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in lookup_slow (3) kernfs 279 1d21h 286d 0/26 upstream: reported on 2023/07/19 13:17
linux-5.15 possible deadlock in lookup_slow 11 241d 413d 0/3 auto-obsoleted due to no activity on 2023/12/11 13:10
linux-4.19 possible deadlock in lookup_slow 22 1416d 1842d 0/1 auto-closed as invalid on 2020/10/12 22:59
upstream possible deadlock in lookup_slow fs 139 1802d 2043d 0/26 auto-closed as invalid on 2019/10/25 08:42
linux-4.14 possible deadlock in lookup_slow C 2027 421d 1712d 0/1 upstream: reported C repro on 2019/08/24 01:53
upstream possible deadlock in lookup_slow (2) fs 7 1570d 1615d 0/26 auto-closed as invalid on 2020/05/11 09:17

Sample crash report:
BTRFS info (device loop0): using free space tree
BTRFS info (device loop0): has skinny extents
BTRFS info (device loop0): enabling ssd optimizations
======================================================
WARNING: possible circular locking dependency detected
5.15.150-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor391/3961 is trying to acquire lock:
ffff0000df030fe0 (&type->i_mutex_dir_key#6){++++}-{3:3}, at: inode_lock_shared include/linux/fs.h:799 [inline]
ffff0000df030fe0 (&type->i_mutex_dir_key#6){++++}-{3:3}, at: lookup_slow+0x50/0x84 fs/namei.c:1679

but task is already holding lock:
ffff0000d9ea8ce0 (&fs_devs->device_list_mutex){+.+.}-{3:3}, at: btrfs_free_stale_devices+0x158/0x9ec fs/btrfs/volumes.c:603

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&fs_devs->device_list_mutex){+.+.}-{3:3}:
       __mutex_lock_common+0x194/0x2154 kernel/locking/mutex.c:596
       __mutex_lock kernel/locking/mutex.c:729 [inline]
       mutex_lock_nested+0xa4/0xf8 kernel/locking/mutex.c:743
       write_all_supers+0x134c/0x3a6c fs/btrfs/disk-io.c:4159
       btrfs_sync_log+0x19c8/0x2294 fs/btrfs/tree-log.c:3327
       btrfs_sync_file+0x924/0xe60 fs/btrfs/file.c:2349
       vfs_fsync_range fs/sync.c:188 [inline]
       vfs_fsync fs/sync.c:202 [inline]
       do_fsync fs/sync.c:212 [inline]
       __do_sys_fdatasync fs/sync.c:225 [inline]
       __se_sys_fdatasync fs/sync.c:223 [inline]
       __arm64_sys_fdatasync+0xac/0x104 fs/sync.c:223
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

-> #2 (&fs_info->tree_log_mutex){+.+.}-{3:3}:
       __mutex_lock_common+0x194/0x2154 kernel/locking/mutex.c:596
       __mutex_lock kernel/locking/mutex.c:729 [inline]
       mutex_lock_nested+0xa4/0xf8 kernel/locking/mutex.c:743
       btrfs_sync_log+0x1904/0x2294 fs/btrfs/tree-log.c:3308
       btrfs_sync_file+0x924/0xe60 fs/btrfs/file.c:2349
       vfs_fsync_range fs/sync.c:188 [inline]
       vfs_fsync fs/sync.c:202 [inline]
       do_fsync fs/sync.c:212 [inline]
       __do_sys_fdatasync fs/sync.c:225 [inline]
       __se_sys_fdatasync fs/sync.c:223 [inline]
       __arm64_sys_fdatasync+0xac/0x104 fs/sync.c:223
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

-> #1 (sb_internal#2){.+.+}-{0:0}:
       __lock_release kernel/locking/lockdep.c:5339 [inline]
       lock_release+0x340/0xa1c kernel/locking/lockdep.c:5643
       up_write+0x98/0x438 kernel/locking/rwsem.c:1593
       btrfs_inode_unlock+0x40/0x80 fs/btrfs/inode.c:140
       btrfs_sync_file+0x870/0xe60 fs/btrfs/file.c:2340
       vfs_fsync_range fs/sync.c:188 [inline]
       vfs_fsync fs/sync.c:202 [inline]
       do_fsync fs/sync.c:212 [inline]
       __do_sys_fdatasync fs/sync.c:225 [inline]
       __se_sys_fdatasync fs/sync.c:223 [inline]
       __arm64_sys_fdatasync+0xac/0x104 fs/sync.c:223
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

-> #0 (&type->i_mutex_dir_key#6){++++}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3053 [inline]
       check_prevs_add kernel/locking/lockdep.c:3172 [inline]
       validate_chain kernel/locking/lockdep.c:3788 [inline]
       __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
       lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
       down_read+0xc0/0x398 kernel/locking/rwsem.c:1488
       inode_lock_shared include/linux/fs.h:799 [inline]
       lookup_slow+0x50/0x84 fs/namei.c:1679
       walk_component+0x394/0x4cc fs/namei.c:1976
       link_path_walk+0x5a0/0xc38
       path_lookupat+0x90/0x3d0 fs/namei.c:2454
       filename_lookup+0x1c4/0x4c8 fs/namei.c:2484
       kern_path+0x4c/0x194 fs/namei.c:2582
       lookup_bdev+0xc0/0x25c block/bdev.c:979
       device_matched fs/btrfs/volumes.c:568 [inline]
       btrfs_free_stale_devices+0x658/0x9ec fs/btrfs/volumes.c:608
       btrfs_forget_devices+0x5c/0x98 fs/btrfs/volumes.c:1388
       btrfs_control_ioctl+0x12c/0x248 fs/btrfs/super.c:2451
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:874 [inline]
       __se_sys_ioctl fs/ioctl.c:860 [inline]
       __arm64_sys_ioctl+0x14c/0x1c8 fs/ioctl.c:860
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

other info that might help us debug this:

Chain exists of:
  &type->i_mutex_dir_key#6 --> &fs_info->tree_log_mutex --> &fs_devs->device_list_mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&fs_devs->device_list_mutex);
                               lock(&fs_info->tree_log_mutex);
                               lock(&fs_devs->device_list_mutex);
  lock(&type->i_mutex_dir_key#6);

 *** DEADLOCK ***

2 locks held by syz-executor391/3961:
 #0: ffff800014fa5e28 (uuid_mutex){+.+.}-{3:3}, at: btrfs_forget_devices+0x28/0x98 fs/btrfs/volumes.c:1387
 #1: ffff0000d9ea8ce0 (&fs_devs->device_list_mutex){+.+.}-{3:3}, at: btrfs_free_stale_devices+0x158/0x9ec fs/btrfs/volumes.c:603

stack backtrace:
CPU: 1 PID: 3961 Comm: syz-executor391 Not tainted 5.15.150-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
 check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
 check_prev_add kernel/locking/lockdep.c:3053 [inline]
 check_prevs_add kernel/locking/lockdep.c:3172 [inline]
 validate_chain kernel/locking/lockdep.c:3788 [inline]
 __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 down_read+0xc0/0x398 kernel/locking/rwsem.c:1488
 inode_lock_shared include/linux/fs.h:799 [inline]
 lookup_slow+0x50/0x84 fs/namei.c:1679
 walk_component+0x394/0x4cc fs/namei.c:1976
 link_path_walk+0x5a0/0xc38
 path_lookupat+0x90/0x3d0 fs/namei.c:2454
 filename_lookup+0x1c4/0x4c8 fs/namei.c:2484
 kern_path+0x4c/0x194 fs/namei.c:2582
 lookup_bdev+0xc0/0x25c block/bdev.c:979
 device_matched fs/btrfs/volumes.c:568 [inline]
 btrfs_free_stale_devices+0x658/0x9ec fs/btrfs/volumes.c:608
 btrfs_forget_devices+0x5c/0x98 fs/btrfs/volumes.c:1388
 btrfs_control_ioctl+0x12c/0x248 fs/btrfs/super.c:2451
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:874 [inline]
 __se_sys_ioctl fs/ioctl.c:860 [inline]
 __arm64_sys_ioctl+0x14c/0x1c8 fs/ioctl.c:860
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (14):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/03/04 02:18 linux-5.15.y 80efc6265290 25905f5d .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/13 08:33 linux-5.15.y cdfd0a7f0139 c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in lookup_slow
2024/04/30 07:32 linux-5.15.y b925f60c6ee7 f10afd69 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/21 02:18 linux-5.15.y c52b9710c83d af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/20 14:55 linux-5.15.y c52b9710c83d af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/20 14:46 linux-5.15.y c52b9710c83d af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/14 06:52 linux-5.15.y fa3df276cd36 c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/13 04:16 linux-5.15.y cdfd0a7f0139 c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/10 07:16 linux-5.15.y 9465fef4ae35 171ec371 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/04/08 21:28 linux-5.15.y 9465fef4ae35 53df08b6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/03/28 22:36 linux-5.15.y 9465fef4ae35 e91187ee .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/03/28 11:13 linux-5.15.y 9465fef4ae35 120789fd .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2024/03/04 01:51 linux-5.15.y 80efc6265290 25905f5d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
2023/12/13 22:34 linux-5.15.y d0fc081c6241 3222d10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in lookup_slow
* Struck through repros no longer work on HEAD.