syzbot


possible deadlock in xfs_buf_find_insert

Status: upstream: reported C repro on 2025/01/28 10:29
Subsystems: xfs
[Documentation on labels]
Reported-by: syzbot+acb56162aef712929d3f@syzkaller.appspotmail.com
First crash: 63d, last: 45d
Cause bisection: introduced by (bisect log) :
commit ee10f6fcdb961e810d7b16be1285319c15c78ef6
Author: Christoph Hellwig <hch@lst.de>
Date: Thu Jan 16 06:01:42 2025 +0000

  xfs: fix buffer lookup vs release race

Crash: possible deadlock in xfs_buf_find_insert (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit a9ab28b3d21aec6d0f56fe722953e20ce470237b
Author: Christoph Hellwig <hch@lst.de>
Date: Tue Jan 28 05:22:58 2025 +0000

  xfs: remove xfs_buf_cache.bc_lock

  
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] [xfs?] possible deadlock in xfs_buf_find_insert 3 (10) 2025/03/14 16:19
[syzbot] Monthly xfs report (Feb 2025) 0 (1) 2025/02/20 19:03
Last patch testing requests (3)
Created Duration User Patch Repo Result
2025/02/25 17:22 1h40m retest repro upstream OK log
2025/02/03 08:07 21m hch@lst.de patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master OK log
2025/02/03 07:30 19m hch@lst.de https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/ for-next error

Sample crash report:
XFS (loop2): Ending clean mount
======================================================
WARNING: possible circular locking dependency detected
6.14.0-rc1-syzkaller #0 Not tainted
------------------------------------------------------
syz.2.18/6033 is trying to acquire lock:
ffff88805ad2e120 (&bp->b_lock){+.+.}-{3:3}, at: spin_lock include/linux/spinlock.h:351 [inline]
ffff88805ad2e120 (&bp->b_lock){+.+.}-{3:3}, at: xfs_buf_try_hold fs/xfs/xfs_buf.c:578 [inline]
ffff88805ad2e120 (&bp->b_lock){+.+.}-{3:3}, at: xfs_buf_find_insert+0x1123/0x1610 fs/xfs/xfs_buf.c:663

but task is already holding lock:
ffff888056b7a980 (&bch->bc_lock){+.+.}-{3:3}, at: spin_lock include/linux/spinlock.h:351 [inline]
ffff888056b7a980 (&bch->bc_lock){+.+.}-{3:3}, at: xfs_buf_find_insert+0x3da/0x1610 fs/xfs/xfs_buf.c:655

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&bch->bc_lock){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5851
       __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
       _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
       spin_lock include/linux/spinlock.h:351 [inline]
       xfs_buf_rele_cached fs/xfs/xfs_buf.c:1093 [inline]
       xfs_buf_rele+0x2bf/0x1660 fs/xfs/xfs_buf.c:1147
       xfs_imap_lookup+0x26a/0x750 fs/xfs/libxfs/xfs_ialloc.c:2431
       xfs_imap+0x54d/0x1090 fs/xfs/libxfs/xfs_ialloc.c:2514
       xfs_iget_cache_miss fs/xfs/xfs_icache.c:644 [inline]
       xfs_iget+0xaf6/0x2ec0 fs/xfs/xfs_icache.c:806
       xfs_mountfs+0x14b0/0x24e0 fs/xfs/xfs_mount.c:933
       xfs_fs_fill_super+0x121d/0x1540 fs/xfs/xfs_super.c:1813
       get_tree_bdev_flags+0x48c/0x5c0 fs/super.c:1636
       vfs_get_tree+0x90/0x2b0 fs/super.c:1814
       do_new_mount+0x2be/0xb40 fs/namespace.c:3560
       do_mount fs/namespace.c:3900 [inline]
       __do_sys_mount fs/namespace.c:4111 [inline]
       __se_sys_mount+0x2d6/0x3c0 fs/namespace.c:4088
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&bp->b_lock){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3163 [inline]
       check_prevs_add kernel/locking/lockdep.c:3282 [inline]
       validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3906
       __lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5228
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5851
       __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
       _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
       spin_lock include/linux/spinlock.h:351 [inline]
       xfs_buf_try_hold fs/xfs/xfs_buf.c:578 [inline]
       xfs_buf_find_insert+0x1123/0x1610 fs/xfs/xfs_buf.c:663
       xfs_buf_get_map+0x149a/0x1ac0 fs/xfs/xfs_buf.c:754
       xfs_buf_read_map+0x110/0xa50 fs/xfs/xfs_buf.c:862
       xfs_trans_read_buf_map+0x260/0xab0 fs/xfs/xfs_trans_buf.c:304
       xfs_trans_read_buf fs/xfs/xfs_trans.h:212 [inline]
       xfs_read_agf+0x2dc/0x630 fs/xfs/libxfs/xfs_alloc.c:3378
       xfs_alloc_read_agf+0x196/0xbe0 fs/xfs/libxfs/xfs_alloc.c:3413
       xfs_bmap_longest_free_extent fs/xfs/libxfs/xfs_bmap.c:3308 [inline]
       xfs_bmap_btalloc_select_lengths fs/xfs/libxfs/xfs_bmap.c:3371 [inline]
       xfs_bmap_btalloc_best_length fs/xfs/libxfs/xfs_bmap.c:3746 [inline]
       xfs_bmap_btalloc fs/xfs/libxfs/xfs_bmap.c:3808 [inline]
       xfs_bmapi_allocate+0x13b6/0x35f0 fs/xfs/libxfs/xfs_bmap.c:4221
       xfs_bmapi_write+0xb91/0x1b50 fs/xfs/libxfs/xfs_bmap.c:4550
       xfs_da_grow_inode_int+0x2ae/0x920 fs/xfs/libxfs/xfs_da_btree.c:2315
       xfs_dir2_grow_inode+0x243/0x720 fs/xfs/libxfs/xfs_dir2.c:628
       xfs_dir2_sf_to_block+0x3d6/0x1ab0 fs/xfs/libxfs/xfs_dir2_block.c:1129
       xfs_dir2_sf_addname+0x7b0/0x1b60 fs/xfs/libxfs/xfs_dir2_sf.c:418
       xfs_dir_createname_args+0x152/0x200 fs/xfs/libxfs/xfs_dir2.c:308
       xfs_dir_createname+0x4b3/0x640 fs/xfs/libxfs/xfs_dir2.c:361
       xfs_dir_rename_children+0x3fc/0xb30 fs/xfs/libxfs/xfs_dir2.c:1268
       xfs_rename+0x172f/0x1e70 fs/xfs/xfs_inode.c:2309
       xfs_vn_rename+0x388/0x4e0 fs/xfs/xfs_iops.c:500
       vfs_rename+0xbdb/0xf00 fs/namei.c:5069
       do_renameat2+0xd94/0x13f0 fs/namei.c:5226
       __do_sys_rename fs/namei.c:5273 [inline]
       __se_sys_rename fs/namei.c:5271 [inline]
       __x64_sys_rename+0x82/0x90 fs/namei.c:5271
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&bch->bc_lock);
                               lock(&bp->b_lock);
                               lock(&bch->bc_lock);
  lock(&bp->b_lock);

 *** DEADLOCK ***

6 locks held by syz.2.18/6033:
 #0: ffff88802a628420 (sb_writers#16){.+.+}-{0:0}, at: mnt_want_write+0x3f/0x90 fs/namespace.c:547
 #1: ffff88805afef770 (&inode->i_sb->s_type->i_mutex_dir_key/1){+.+.}-{4:4}, at: inode_lock_nested include/linux/fs.h:900 [inline]
 #1: ffff88805afef770 (&inode->i_sb->s_type->i_mutex_dir_key/1){+.+.}-{4:4}, at: lock_rename fs/namei.c:3217 [inline]
 #1: ffff88805afef770 (&inode->i_sb->s_type->i_mutex_dir_key/1){+.+.}-{4:4}, at: do_renameat2+0x62c/0x13f0 fs/namei.c:5161
 #2: ffff88802a628610 (sb_internal#2){.+.+}-{0:0}, at: xfs_rename+0xb20/0x1e70 fs/xfs/xfs_inode.c:2179
 #3: ffff88805afef558 (&xfs_dir_ilock_class){++++}-{4:4}, at: xfs_lock_inodes+0x2d2/0x540 fs/xfs/xfs_inode.c:443
 #4: ffff888056c06d98 (&xfs_dir_ilock_class/2){+.+.}-{4:4}, at: xfs_lock_inodes+0x2d2/0x540 fs/xfs/xfs_inode.c:443
 #5: ffff888056b7a980 (&bch->bc_lock){+.+.}-{3:3}, at: spin_lock include/linux/spinlock.h:351 [inline]
 #5: ffff888056b7a980 (&bch->bc_lock){+.+.}-{3:3}, at: xfs_buf_find_insert+0x3da/0x1610 fs/xfs/xfs_buf.c:655

stack backtrace:
CPU: 1 UID: 0 PID: 6033 Comm: syz.2.18 Not tainted 6.14.0-rc1-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_circular_bug+0x13a/0x1b0 kernel/locking/lockdep.c:2076
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2208
 check_prev_add kernel/locking/lockdep.c:3163 [inline]
 check_prevs_add kernel/locking/lockdep.c:3282 [inline]
 validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3906
 __lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5228
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5851
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:351 [inline]
 xfs_buf_try_hold fs/xfs/xfs_buf.c:578 [inline]
 xfs_buf_find_insert+0x1123/0x1610 fs/xfs/xfs_buf.c:663
 xfs_buf_get_map+0x149a/0x1ac0 fs/xfs/xfs_buf.c:754
 xfs_buf_read_map+0x110/0xa50 fs/xfs/xfs_buf.c:862
 xfs_trans_read_buf_map+0x260/0xab0 fs/xfs/xfs_trans_buf.c:304
 xfs_trans_read_buf fs/xfs/xfs_trans.h:212 [inline]
 xfs_read_agf+0x2dc/0x630 fs/xfs/libxfs/xfs_alloc.c:3378
 xfs_alloc_read_agf+0x196/0xbe0 fs/xfs/libxfs/xfs_alloc.c:3413
 xfs_bmap_longest_free_extent fs/xfs/libxfs/xfs_bmap.c:3308 [inline]
 xfs_bmap_btalloc_select_lengths fs/xfs/libxfs/xfs_bmap.c:3371 [inline]
 xfs_bmap_btalloc_best_length fs/xfs/libxfs/xfs_bmap.c:3746 [inline]
 xfs_bmap_btalloc fs/xfs/libxfs/xfs_bmap.c:3808 [inline]
 xfs_bmapi_allocate+0x13b6/0x35f0 fs/xfs/libxfs/xfs_bmap.c:4221
 xfs_bmapi_write+0xb91/0x1b50 fs/xfs/libxfs/xfs_bmap.c:4550
 xfs_da_grow_inode_int+0x2ae/0x920 fs/xfs/libxfs/xfs_da_btree.c:2315
 xfs_dir2_grow_inode+0x243/0x720 fs/xfs/libxfs/xfs_dir2.c:628
 xfs_dir2_sf_to_block+0x3d6/0x1ab0 fs/xfs/libxfs/xfs_dir2_block.c:1129
 xfs_dir2_sf_addname+0x7b0/0x1b60 fs/xfs/libxfs/xfs_dir2_sf.c:418
 xfs_dir_createname_args+0x152/0x200 fs/xfs/libxfs/xfs_dir2.c:308
 xfs_dir_createname+0x4b3/0x640 fs/xfs/libxfs/xfs_dir2.c:361
 xfs_dir_rename_children+0x3fc/0xb30 fs/xfs/libxfs/xfs_dir2.c:1268
 xfs_rename+0x172f/0x1e70 fs/xfs/xfs_inode.c:2309
 xfs_vn_rename+0x388/0x4e0 fs/xfs/xfs_iops.c:500
 vfs_rename+0xbdb/0xf00 fs/namei.c:5069
 do_renameat2+0xd94/0x13f0 fs/namei.c:5226
 __do_sys_rename fs/namei.c:5273 [inline]
 __se_sys_rename fs/namei.c:5271 [inline]
 __x64_sys_rename+0x82/0x90 fs/namei.c:5271
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fdaf1f8cda9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fdaf2d44038 EFLAGS: 00000246 ORIG_RAX: 0000000000000052
RAX: ffffffffffffffda RBX: 00007fdaf21a5fa0 RCX: 00007fdaf1f8cda9
RDX: 0000000000000000 RSI: 0000000020001040 RDI: 00000000200009c0
RBP: 00007fdaf200e2a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007fdaf21a5fa0 R15: 00007ffeab676d78
 </TASK>

Crashes (15):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/02/03 12:14 upstream 2014c95afece a21a8419 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/02/02 06:40 upstream 69b8923f5003 568559e4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/02/02 00:53 upstream 69b8923f5003 568559e4 .config console log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/02/02 00:43 upstream 69b8923f5003 568559e4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/02/02 00:16 upstream 69b8923f5003 568559e4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/01/30 19:42 upstream d3d90cc2891c 9c8ab845 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in xfs_buf_find_insert
2025/01/26 15:51 upstream aa22f4da2a46 9fbd772e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/01/24 10:23 upstream bc8198dc7ebc 521b0ce3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs possible deadlock in xfs_buf_find_insert
2025/02/03 18:25 upstream 2014c95afece a21a8419 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in xfs_buf_find_insert
2025/02/02 23:18 upstream 69e858e0b8b2 568559e4 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in xfs_buf_find_insert
2025/02/11 10:28 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2014c95afece 43f51a00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in xfs_buf_find_insert
2025/02/10 11:06 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2014c95afece ef44b750 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in xfs_buf_find_insert
2025/02/08 22:46 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2014c95afece ef44b750 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in xfs_buf_find_insert
2025/02/07 20:18 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2014c95afece a4f327c2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in xfs_buf_find_insert
2025/02/05 06:04 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2014c95afece 4baca3d6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in xfs_buf_find_insert
* Struck through repros no longer work on HEAD.