syzbot


possible deadlock in wait_transaction_locked

Status: upstream: reported on 2024/05/18 18:31
Subsystems: ext4
[Documentation on labels]
Reported-by: syzbot+007d621ed59411ae5fd0@syzkaller.appspotmail.com
First crash: 38d, last: 16d
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [ext4?] possible deadlock in wait_transaction_locked 0 (1) 2024/05/18 18:31

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.10.0-rc2-syzkaller-00097-g2df0193e62cf #0 Not tainted
------------------------------------------------------
kswapd0/112 is trying to acquire lock:
ffff88801e522950 (jbd2_handle){++++}-{0:0}, at: wait_transaction_locked+0x188/0x230 fs/jbd2/transaction.c:177

but task is already holding lock:
ffffffff8dd3a9e0 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x166/0x1970 mm/vmscan.c:6798

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (fs_reclaim){+.+.}-{0:0}:
       __fs_reclaim_acquire mm/page_alloc.c:3783 [inline]
       fs_reclaim_acquire+0x102/0x160 mm/page_alloc.c:3797
       might_alloc include/linux/sched/mm.h:334 [inline]
       slab_pre_alloc_hook mm/slub.c:3890 [inline]
       slab_alloc_node mm/slub.c:3980 [inline]
       __do_kmalloc_node mm/slub.c:4120 [inline]
       __kmalloc_node_noprof+0xbb/0x450 mm/slub.c:4128
       kmalloc_node_noprof include/linux/slab.h:681 [inline]
       kvmalloc_node_noprof+0x9d/0x1a0 mm/util.c:634
       ext4_xattr_inode_cache_find fs/ext4/xattr.c:1535 [inline]
       ext4_xattr_inode_lookup_create fs/ext4/xattr.c:1581 [inline]
       ext4_xattr_inode_lookup_create+0x4e8/0x1910 fs/ext4/xattr.c:1568
       ext4_xattr_block_set+0x77a/0x3090 fs/ext4/xattr.c:1910
       ext4_xattr_move_to_block fs/ext4/xattr.c:2663 [inline]
       ext4_xattr_make_inode_space fs/ext4/xattr.c:2738 [inline]
       ext4_expand_extra_isize_ea+0xf57/0x1990 fs/ext4/xattr.c:2830
       __ext4_expand_extra_isize+0x322/0x450 fs/ext4/inode.c:5782
       ext4_try_to_expand_extra_isize fs/ext4/inode.c:5825 [inline]
       __ext4_mark_inode_dirty+0x55a/0x890 fs/ext4/inode.c:5903
       ext4_set_acl+0x453/0x5a0 fs/ext4/acl.c:263
       set_posix_acl+0x25c/0x320 fs/posix_acl.c:955
       vfs_set_acl+0x53d/0x940 fs/posix_acl.c:1134
       do_set_acl+0xd9/0x1b0 fs/posix_acl.c:1279
       do_setxattr+0xeb/0x170 fs/xattr.c:626
       setxattr+0x15d/0x180 fs/xattr.c:652
       path_setxattr+0x179/0x1e0 fs/xattr.c:671
       __do_sys_setxattr fs/xattr.c:687 [inline]
       __se_sys_setxattr fs/xattr.c:683 [inline]
       __ia32_sys_setxattr+0xc0/0x160 fs/xattr.c:683
       do_syscall_32_irqs_on arch/x86/entry/common.c:165 [inline]
       __do_fast_syscall_32+0x73/0x120 arch/x86/entry/common.c:386
       do_fast_syscall_32+0x32/0x80 arch/x86/entry/common.c:411
       entry_SYSENTER_compat_after_hwframe+0x84/0x8e

-> #1 (&ei->xattr_sem){++++}-{3:3}:
       down_write+0x3a/0x50 kernel/locking/rwsem.c:1579
       ext4_write_lock_xattr fs/ext4/xattr.h:155 [inline]
       ext4_xattr_set_handle+0x156/0x16d0 fs/ext4/xattr.c:2367
       ext4_xattr_set+0x149/0x380 fs/ext4/xattr.c:2554
       ext4_xattr_user_set+0xbd/0x100 fs/ext4/xattr_user.c:41
       __vfs_removexattr+0x153/0x1c0 fs/xattr.c:518
       __vfs_removexattr_locked+0x166/0x480 fs/xattr.c:553
       vfs_removexattr+0xcf/0x260 fs/xattr.c:575
       removexattr+0x137/0x1b0 fs/xattr.c:917
       path_removexattr+0x161/0x1c0 fs/xattr.c:931
       __do_sys_removexattr fs/xattr.c:945 [inline]
       __se_sys_removexattr fs/xattr.c:942 [inline]
       __ia32_sys_removexattr+0x57/0x80 fs/xattr.c:942
       do_syscall_32_irqs_on arch/x86/entry/common.c:165 [inline]
       __do_fast_syscall_32+0x73/0x120 arch/x86/entry/common.c:386
       do_fast_syscall_32+0x32/0x80 arch/x86/entry/common.c:411
       entry_SYSENTER_compat_after_hwframe+0x84/0x8e

-> #0 (jbd2_handle){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
       wait_transaction_locked+0x192/0x230 fs/jbd2/transaction.c:177
       add_transaction_credits+0x10a/0xb80 fs/jbd2/transaction.c:241
       start_this_handle+0x3a3/0x15e0 fs/jbd2/transaction.c:422
       jbd2__journal_start+0x394/0x6a0 fs/jbd2/transaction.c:520
       __ext4_journal_start_sb+0x358/0x660 fs/ext4/ext4_jbd2.c:112
       __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
       ext4_dirty_inode+0xa1/0x130 fs/ext4/inode.c:5932
       __mark_inode_dirty+0x1f0/0xe70 fs/fs-writeback.c:2486
       mark_inode_dirty_sync include/linux/fs.h:2409 [inline]
       iput.part.0+0x5b/0x7f0 fs/inode.c:1764
       iput+0x5c/0x80 fs/inode.c:1757
       dentry_unlink_inode+0x295/0x480 fs/dcache.c:400
       __dentry_kill+0x1d0/0x600 fs/dcache.c:603
       shrink_kill fs/dcache.c:1048 [inline]
       shrink_dentry_list+0x140/0x5d0 fs/dcache.c:1075
       prune_dcache_sb+0xeb/0x150 fs/dcache.c:1156
       super_cache_scan+0x32a/0x550 fs/super.c:221
       do_shrink_slab+0x44f/0x11c0 mm/shrinker.c:435
       shrink_slab_memcg mm/shrinker.c:548 [inline]
       shrink_slab+0xa87/0x1310 mm/shrinker.c:626
       shrink_one+0x493/0x7c0 mm/vmscan.c:4790
       shrink_many mm/vmscan.c:4851 [inline]
       lru_gen_shrink_node+0x89f/0x1750 mm/vmscan.c:4951
       shrink_node mm/vmscan.c:5910 [inline]
       kswapd_shrink_node mm/vmscan.c:6720 [inline]
       balance_pgdat+0x1105/0x1970 mm/vmscan.c:6911
       kswapd+0x5ea/0xbf0 mm/vmscan.c:7180
       kthread+0x2c1/0x3a0 kernel/kthread.c:389
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

other info that might help us debug this:

Chain exists of:
  jbd2_handle --> &ei->xattr_sem --> fs_reclaim

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(fs_reclaim);
                               lock(&ei->xattr_sem);
                               lock(fs_reclaim);
  lock(jbd2_handle);

 *** DEADLOCK ***

2 locks held by kswapd0/112:
 #0: ffffffff8dd3a9e0 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x166/0x1970 mm/vmscan.c:6798
 #1: ffff88802799c0e0 (&type->s_umount_key#33){++++}-{3:3}, at: super_trylock_shared fs/super.c:562 [inline]
 #1: ffff88802799c0e0 (&type->s_umount_key#33){++++}-{3:3}, at: super_cache_scan+0x96/0x550 fs/super.c:196

stack backtrace:
CPU: 0 PID: 112 Comm: kswapd0 Not tainted 6.10.0-rc2-syzkaller-00097-g2df0193e62cf #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:114
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 wait_transaction_locked+0x192/0x230 fs/jbd2/transaction.c:177
 add_transaction_credits+0x10a/0xb80 fs/jbd2/transaction.c:241
 start_this_handle+0x3a3/0x15e0 fs/jbd2/transaction.c:422
 jbd2__journal_start+0x394/0x6a0 fs/jbd2/transaction.c:520
 __ext4_journal_start_sb+0x358/0x660 fs/ext4/ext4_jbd2.c:112
 __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
 ext4_dirty_inode+0xa1/0x130 fs/ext4/inode.c:5932
 __mark_inode_dirty+0x1f0/0xe70 fs/fs-writeback.c:2486
 mark_inode_dirty_sync include/linux/fs.h:2409 [inline]
 iput.part.0+0x5b/0x7f0 fs/inode.c:1764
 iput+0x5c/0x80 fs/inode.c:1757
 dentry_unlink_inode+0x295/0x480 fs/dcache.c:400
 __dentry_kill+0x1d0/0x600 fs/dcache.c:603
 shrink_kill fs/dcache.c:1048 [inline]
 shrink_dentry_list+0x140/0x5d0 fs/dcache.c:1075
 prune_dcache_sb+0xeb/0x150 fs/dcache.c:1156
 super_cache_scan+0x32a/0x550 fs/super.c:221
 do_shrink_slab+0x44f/0x11c0 mm/shrinker.c:435
 shrink_slab_memcg mm/shrinker.c:548 [inline]
 shrink_slab+0xa87/0x1310 mm/shrinker.c:626
 shrink_one+0x493/0x7c0 mm/vmscan.c:4790
 shrink_many mm/vmscan.c:4851 [inline]
 lru_gen_shrink_node+0x89f/0x1750 mm/vmscan.c:4951
 shrink_node mm/vmscan.c:5910 [inline]
 kswapd_shrink_node mm/vmscan.c:6720 [inline]
 balance_pgdat+0x1105/0x1970 mm/vmscan.c:6911
 kswapd+0x5ea/0xbf0 mm/vmscan.c:7180
 kthread+0x2c1/0x3a0 kernel/kthread.c:389
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/06/06 08:20 upstream 2df0193e62cf c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in wait_transaction_locked
2024/05/16 10:32 upstream 3c999d1ae3c7 ef5d53ed .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in wait_transaction_locked
2024/05/14 18:20 upstream 6bfd2d442af5 fdb4c10c .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in wait_transaction_locked
* Struck through repros no longer work on HEAD.