syzbot

 sign-in | mailing list | source | docs
🐞 Open [969] ≡ Subsystems 🐞 Fixed [4558] 🐞 Invalid [10512] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

possible deadlock in start_this_handle (4)

Status: upstream: reported on 2023/03/01 00:02
Labels: fscrypt ext4 (incorrect?)
Reported-by: syzbot+cf0b4280f19be4031cf2@syzkaller.appspotmail.com
First crash: 97d, last: 76d
▶ ▼ Duplicate bugs (1)
duplicates (1):
Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
possible deadlock in fscrypt_initialize (2) fscrypt 63 37d 93d 0/24 closed as dup on 2023/03/01 19:02
▶ ▼ Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [ext4?] possible deadlock in start_this_handle (4) 1 (2) 2023/03/01 00:19
▶ ▼ Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in start_this_handle (3) ext4 8 135d 323d 24/24 fixed on 2023/02/24 13:50
upstream possible deadlock in start_this_handle (2) 8 809d 842d 0/24 auto-closed as invalid on 2021/07/13 16:11
upstream possible deadlock in start_this_handle 8 1691d 1729d 0/24 auto-closed as invalid on 2019/04/13 16:27

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.3.0-rc2-syzkaller-00363-g478a351ce0d6 #0 Not tainted
------------------------------------------------------
kswapd0/101 is trying to acquire lock:
ffff88801b334990 (jbd2_handle){++++}-{0:0}, at: start_this_handle+0xfb4/0x14e0 fs/jbd2/transaction.c:461

but task is already holding lock:
ffffffff8c8e74e0 (fs_reclaim){+.+.}-{0:0}, at: set_task_reclaim_state mm/vmscan.c:200 [inline]
ffffffff8c8e74e0 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x170/0x1ac0 mm/vmscan.c:7338

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (fs_reclaim){+.+.}-{0:0}:
       __fs_reclaim_acquire mm/page_alloc.c:4716 [inline]
       fs_reclaim_acquire+0x11d/0x160 mm/page_alloc.c:4730
       might_alloc include/linux/sched/mm.h:271 [inline]
       slab_pre_alloc_hook mm/slab.h:728 [inline]
       slab_alloc_node mm/slub.c:3434 [inline]
       __kmem_cache_alloc_node+0x41/0x320 mm/slub.c:3491
       __do_kmalloc_node mm/slab_common.c:966 [inline]
       __kmalloc_node+0x51/0x1a0 mm/slab_common.c:974
       kmalloc_node include/linux/slab.h:610 [inline]
       kvmalloc_node+0xa2/0x1a0 mm/util.c:603
       kvmalloc include/linux/slab.h:737 [inline]
       ext4_xattr_inode_cache_find fs/ext4/xattr.c:1552 [inline]
       ext4_xattr_inode_lookup_create fs/ext4/xattr.c:1595 [inline]
       ext4_xattr_set_entry+0x1d82/0x39e0 fs/ext4/xattr.c:1737
       ext4_xattr_block_set+0x618/0x3030 fs/ext4/xattr.c:1974
       ext4_xattr_move_to_block fs/ext4/xattr.c:2668 [inline]
       ext4_xattr_make_inode_space fs/ext4/xattr.c:2743 [inline]
       ext4_expand_extra_isize_ea+0xa84/0x1880 fs/ext4/xattr.c:2835
       __ext4_expand_extra_isize+0x33e/0x470 fs/ext4/inode.c:5960
       ext4_try_to_expand_extra_isize fs/ext4/inode.c:6003 [inline]
       __ext4_mark_inode_dirty+0x51b/0x800 fs/ext4/inode.c:6081
       __ext4_unlink+0x667/0xcd0 fs/ext4/namei.c:3269
       ext4_unlink+0x41f/0x590 fs/ext4/namei.c:3298
       vfs_unlink+0x355/0x930 fs/namei.c:4250
       do_unlinkat+0x3df/0x670 fs/namei.c:4316
       __do_sys_unlinkat fs/namei.c:4359 [inline]
       __se_sys_unlinkat fs/namei.c:4352 [inline]
       __ia32_sys_unlinkat+0xc1/0x130 fs/namei.c:4352
       do_syscall_32_irqs_on arch/x86/entry/common.c:112 [inline]
       __do_fast_syscall_32+0x65/0xf0 arch/x86/entry/common.c:178
       do_fast_syscall_32+0x33/0x70 arch/x86/entry/common.c:203
       entry_SYSENTER_compat_after_hwframe+0x70/0x82

-> #1 (&ei->xattr_sem){++++}-{3:3}:
       down_write+0x92/0x200 kernel/locking/rwsem.c:1573
       ext4_write_lock_xattr fs/ext4/xattr.h:155 [inline]
       ext4_xattr_set_handle+0x160/0x1510 fs/ext4/xattr.c:2373
       ext4_set_context+0x406/0x5a0 fs/ext4/crypto.c:198
       set_encryption_policy+0x282/0x430 fs/crypto/policy.c:473
       fscrypt_ioctl_set_policy+0x3ce/0x460 fs/crypto/policy.c:526
       __ext4_ioctl+0x2100/0x4b00 fs/ext4/ioctl.c:1501
       ext4_ioctl fs/ext4/ioctl.c:1608 [inline]
       ext4_compat_ioctl+0xca/0x430 fs/ext4/ioctl.c:1684
       __do_compat_sys_ioctl+0x255/0x2b0 fs/ioctl.c:968
       do_syscall_32_irqs_on arch/x86/entry/common.c:112 [inline]
       __do_fast_syscall_32+0x65/0xf0 arch/x86/entry/common.c:178
       do_fast_syscall_32+0x33/0x70 arch/x86/entry/common.c:203
       entry_SYSENTER_compat_after_hwframe+0x70/0x82

-> #0 (jbd2_handle){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3098 [inline]
       check_prevs_add kernel/locking/lockdep.c:3217 [inline]
       validate_chain kernel/locking/lockdep.c:3832 [inline]
       __lock_acquire+0x2ec7/0x5d40 kernel/locking/lockdep.c:5056
       lock_acquire kernel/locking/lockdep.c:5669 [inline]
       lock_acquire+0x1af/0x520 kernel/locking/lockdep.c:5634
       start_this_handle+0xfe7/0x14e0 fs/jbd2/transaction.c:463
       jbd2__journal_start+0x390/0x850 fs/jbd2/transaction.c:520
       __ext4_journal_start_sb+0x411/0x5d0 fs/ext4/ext4_jbd2.c:111
       __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
       ext4_dirty_inode+0xa5/0x130 fs/ext4/inode.c:6110
       __mark_inode_dirty+0x1e0/0xd60 fs/fs-writeback.c:2421
       mark_inode_dirty_sync include/linux/fs.h:2132 [inline]
       iput.part.0+0x57/0x740 fs/inode.c:1771
       iput+0x5c/0x80 fs/inode.c:1764
       dentry_unlink_inode+0x2b1/0x460 fs/dcache.c:401
       __dentry_kill+0x3c0/0x640 fs/dcache.c:607
       shrink_dentry_list+0x12c/0x4f0 fs/dcache.c:1201
       prune_dcache_sb+0xeb/0x150 fs/dcache.c:1282
       super_cache_scan+0x33a/0x590 fs/super.c:104
       do_shrink_slab+0x428/0xaa0 mm/vmscan.c:853
       shrink_slab_memcg mm/vmscan.c:922 [inline]
       shrink_slab+0x388/0x660 mm/vmscan.c:1001
       shrink_one+0x502/0x810 mm/vmscan.c:5343
       shrink_many mm/vmscan.c:5394 [inline]
       lru_gen_shrink_node mm/vmscan.c:5511 [inline]
       shrink_node+0x2064/0x35f0 mm/vmscan.c:6459
       kswapd_shrink_node mm/vmscan.c:7262 [inline]
       balance_pgdat+0xa02/0x1ac0 mm/vmscan.c:7452
       kswapd+0x677/0xd60 mm/vmscan.c:7712
       kthread+0x2e8/0x3a0 kernel/kthread.c:376
       ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308

other info that might help us debug this:

Chain exists of:
  jbd2_handle --> &ei->xattr_sem --> fs_reclaim

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(fs_reclaim);
                               lock(&ei->xattr_sem);
                               lock(fs_reclaim);
  lock(jbd2_handle);

 *** DEADLOCK ***

3 locks held by kswapd0/101:
 #0: ffffffff8c8e74e0 (fs_reclaim){+.+.}-{0:0}, at: set_task_reclaim_state mm/vmscan.c:200 [inline]
 #0: ffffffff8c8e74e0 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat+0x170/0x1ac0 mm/vmscan.c:7338
 #1: ffffffff8c89e310 (shrinker_rwsem){++++}-{3:3}, at: shrink_slab_memcg mm/vmscan.c:895 [inline]
 #1: ffffffff8c89e310 (shrinker_rwsem){++++}-{3:3}, at: shrink_slab+0x2a0/0x660 mm/vmscan.c:1001
 #2: ffff88801aace0e0 (&type->s_umount_key#31){++++}-{3:3}, at: trylock_super fs/super.c:414 [inline]
 #2: ffff88801aace0e0 (&type->s_umount_key#31){++++}-{3:3}, at: super_cache_scan+0x70/0x590 fs/super.c:79

stack backtrace:
CPU: 3 PID: 101 Comm: kswapd0 Not tainted 6.3.0-rc2-syzkaller-00363-g478a351ce0d6 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.14.0-2 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
 check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2178
 check_prev_add kernel/locking/lockdep.c:3098 [inline]
 check_prevs_add kernel/locking/lockdep.c:3217 [inline]
 validate_chain kernel/locking/lockdep.c:3832 [inline]
 __lock_acquire+0x2ec7/0x5d40 kernel/locking/lockdep.c:5056
 lock_acquire kernel/locking/lockdep.c:5669 [inline]
 lock_acquire+0x1af/0x520 kernel/locking/lockdep.c:5634
 start_this_handle+0xfe7/0x14e0 fs/jbd2/transaction.c:463
 jbd2__journal_start+0x390/0x850 fs/jbd2/transaction.c:520
 __ext4_journal_start_sb+0x411/0x5d0 fs/ext4/ext4_jbd2.c:111
 __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
 ext4_dirty_inode+0xa5/0x130 fs/ext4/inode.c:6110
 __mark_inode_dirty+0x1e0/0xd60 fs/fs-writeback.c:2421
 mark_inode_dirty_sync include/linux/fs.h:2132 [inline]
 iput.part.0+0x57/0x740 fs/inode.c:1771
 iput+0x5c/0x80 fs/inode.c:1764
 dentry_unlink_inode+0x2b1/0x460 fs/dcache.c:401
 __dentry_kill+0x3c0/0x640 fs/dcache.c:607
 shrink_dentry_list+0x12c/0x4f0 fs/dcache.c:1201
 prune_dcache_sb+0xeb/0x150 fs/dcache.c:1282
 super_cache_scan+0x33a/0x590 fs/super.c:104
 do_shrink_slab+0x428/0xaa0 mm/vmscan.c:853
 shrink_slab_memcg mm/vmscan.c:922 [inline]
 shrink_slab+0x388/0x660 mm/vmscan.c:1001
 shrink_one+0x502/0x810 mm/vmscan.c:5343
 shrink_many mm/vmscan.c:5394 [inline]
 lru_gen_shrink_node mm/vmscan.c:5511 [inline]
 shrink_node+0x2064/0x35f0 mm/vmscan.c:6459
 kswapd_shrink_node mm/vmscan.c:7262 [inline]
 balance_pgdat+0xa02/0x1ac0 mm/vmscan.c:7452
 kswapd+0x677/0xd60 mm/vmscan.c:7712
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Manager Title
2023/03/18 13:47 upstream 478a351ce0d6 7939252e .config console log report info ci-qemu-upstream-386 possible deadlock in start_this_handle
2023/03/17 10:16 upstream 38e04b3e4240 18b58603 .config console log report info ci-qemu-upstream-386 possible deadlock in start_this_handle
2023/03/12 22:41 upstream 134231664868 5205ef30 .config console log report info ci-qemu-upstream-386 possible deadlock in start_this_handle
2023/03/05 16:18 upstream b01fe98d34f3 f8902b57 .config console log report info ci-qemu-upstream-386 possible deadlock in start_this_handle
2023/02/24 23:50 upstream a93e884edf61 630c6bc9 .config console log report info ci-qemu-upstream-386 possible deadlock in start_this_handle
* Struck through repros no longer work on HEAD.