syzbot


possible deadlock in ext4_da_get_block_prep

Status: upstream: reported on 2023/07/09 11:14
Subsystems: ext4
[Documentation on labels]
Reported-by: syzbot+a86b193140e10df1aff2@syzkaller.appspotmail.com
First crash: 328d, last: 78d
Discussions (3)
Title Replies (including bot) Last reply
[syzbot] Monthly ext4 report (Mar 2024) 0 (1) 2024/03/12 09:29
[syzbot] Monthly ext4 report (Feb 2024) 0 (1) 2024/02/09 20:57
[syzbot] [ext4?] possible deadlock in ext4_da_get_block_prep 0 (1) 2023/07/09 11:14

Sample crash report:
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 000000000000000b R14: 00007f28e45abf80 R15: 00007fffea5134a8
 </TASK>
======================================================
WARNING: possible circular locking dependency detected
6.8.0-rc7-syzkaller-00238-g005f6f34bd47 #0 Not tainted
------------------------------------------------------
syz-executor.0/10916 is trying to acquire lock:
ffffffff8d91e080 (fs_reclaim){+.+.}-{0:0}, at: might_alloc include/linux/sched/mm.h:303 [inline]
ffffffff8d91e080 (fs_reclaim){+.+.}-{0:0}, at: slab_pre_alloc_hook mm/slub.c:3761 [inline]
ffffffff8d91e080 (fs_reclaim){+.+.}-{0:0}, at: slab_alloc_node mm/slub.c:3842 [inline]
ffffffff8d91e080 (fs_reclaim){+.+.}-{0:0}, at: kmem_cache_alloc+0x4f/0x320 mm/slub.c:3867

but task is already holding lock:
ffff888031ffe688 (&ei->i_data_sem){++++}-{3:3}, at: ext4_da_map_blocks fs/ext4/inode.c:1776 [inline]
ffff888031ffe688 (&ei->i_data_sem){++++}-{3:3}, at: ext4_da_get_block_prep+0x770/0x1320 fs/ext4/inode.c:1817

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&ei->i_data_sem){++++}-{3:3}:
       down_write+0x3a/0x50 kernel/locking/rwsem.c:1579
       ext4_map_blocks+0x5ab/0x17d0 fs/ext4/inode.c:616
       mpage_map_one_extent fs/ext4/inode.c:2163 [inline]
       mpage_map_and_submit_extent fs/ext4/inode.c:2216 [inline]
       ext4_do_writepages+0x1852/0x3360 fs/ext4/inode.c:2679
       ext4_writepages+0x30c/0x780 fs/ext4/inode.c:2768
       do_writepages+0x1b4/0x690 mm/page-writeback.c:2553
       __writeback_single_inode+0x158/0xe90 fs/fs-writeback.c:1625
       writeback_sb_inodes+0x59d/0x1080 fs/fs-writeback.c:1916
       __writeback_inodes_wb+0xff/0x2e0 fs/fs-writeback.c:1987
       wb_writeback+0x7f8/0xaa0 fs/fs-writeback.c:2094
       wb_check_background_flush fs/fs-writeback.c:2164 [inline]
       wb_do_writeback fs/fs-writeback.c:2252 [inline]
       wb_workfn+0x880/0xfe0 fs/fs-writeback.c:2279
       process_one_work+0x889/0x15e0 kernel/workqueue.c:2633
       process_scheduled_works kernel/workqueue.c:2706 [inline]
       worker_thread+0x8b9/0x12a0 kernel/workqueue.c:2787
       kthread+0x2c6/0x3b0 kernel/kthread.c:388
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1b/0x30 arch/x86/entry/entry_64.S:243

-> #2 (jbd2_handle){++++}-{0:0}:
       start_this_handle+0x1103/0x15e0 fs/jbd2/transaction.c:463
       jbd2__journal_start+0x395/0x850 fs/jbd2/transaction.c:520
       __ext4_journal_start_sb+0x347/0x5e0 fs/ext4/ext4_jbd2.c:112
       ext4_sample_last_mounted fs/ext4/file.c:837 [inline]
       ext4_file_open+0x636/0xc80 fs/ext4/file.c:866
       do_dentry_open+0x8da/0x18c0 fs/open.c:953
       do_open fs/namei.c:3645 [inline]
       path_openat+0x1e00/0x29a0 fs/namei.c:3802
       do_filp_open+0x1de/0x440 fs/namei.c:3829
       do_sys_openat2+0x17a/0x1e0 fs/open.c:1404
       do_sys_open fs/open.c:1419 [inline]
       __do_sys_openat fs/open.c:1435 [inline]
       __se_sys_openat fs/open.c:1430 [inline]
       __x64_sys_openat+0x175/0x210 fs/open.c:1430
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xd5/0x270 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x6f/0x77

-> #1 (sb_internal){.+.+}-{0:0}:
       percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
       __sb_start_write include/linux/fs.h:1641 [inline]
       sb_start_intwrite include/linux/fs.h:1824 [inline]
       ext4_evict_inode+0xe5f/0x1a40 fs/ext4/inode.c:212
       evict+0x2ed/0x6c0 fs/inode.c:665
       iput_final fs/inode.c:1739 [inline]
       iput.part.0+0x563/0x7b0 fs/inode.c:1765
       iput+0x5c/0x80 fs/inode.c:1755
       dentry_unlink_inode+0x295/0x440 fs/dcache.c:400
       __dentry_kill+0x1d0/0x600 fs/dcache.c:603
       shrink_kill fs/dcache.c:1048 [inline]
       shrink_dentry_list+0x140/0x5d0 fs/dcache.c:1075
       prune_dcache_sb+0xeb/0x150 fs/dcache.c:1156
       super_cache_scan+0x32a/0x550 fs/super.c:221
       do_shrink_slab+0x426/0x1120 mm/shrinker.c:435
       shrink_slab_memcg mm/shrinker.c:548 [inline]
       shrink_slab+0xa87/0x1310 mm/shrinker.c:626
       shrink_one+0x493/0x7b0 mm/vmscan.c:4767
       shrink_many mm/vmscan.c:4828 [inline]
       lru_gen_shrink_node mm/vmscan.c:4929 [inline]
       shrink_node+0x2158/0x3780 mm/vmscan.c:5888
       kswapd_shrink_node mm/vmscan.c:6696 [inline]
       balance_pgdat+0x9d2/0x1a90 mm/vmscan.c:6886
       kswapd+0x5be/0xc00 mm/vmscan.c:7146
       kthread+0x2c6/0x3b0 kernel/kthread.c:388
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1b/0x30 arch/x86/entry/entry_64.S:243

-> #0 (fs_reclaim){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x244f/0x3b40 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719
       __fs_reclaim_acquire mm/page_alloc.c:3692 [inline]
       fs_reclaim_acquire+0x104/0x150 mm/page_alloc.c:3706
       might_alloc include/linux/sched/mm.h:303 [inline]
       slab_pre_alloc_hook mm/slub.c:3761 [inline]
       slab_alloc_node mm/slub.c:3842 [inline]
       kmem_cache_alloc+0x4f/0x320 mm/slub.c:3867
       kmem_cache_zalloc include/linux/slab.h:701 [inline]
       __es_alloc_extent fs/ext4/extents_status.c:483 [inline]
       ext4_es_insert_delayed_block+0x546/0x760 fs/ext4/extents_status.c:2090
       ext4_insert_delayed_block fs/ext4/inode.c:1676 [inline]
       ext4_da_map_blocks fs/ext4/inode.c:1777 [inline]
       ext4_da_get_block_prep+0x850/0x1320 fs/ext4/inode.c:1817
       ext4_block_write_begin+0x3da/0xef0 fs/ext4/inode.c:1055
       ext4_da_write_begin+0x40e/0x8c0 fs/ext4/inode.c:2894
       generic_perform_write+0x273/0x620 mm/filemap.c:3930
       ext4_buffered_write_iter+0x11f/0x3d0 fs/ext4/file.c:299
       ext4_file_write_iter+0x819/0x1960 fs/ext4/file.c:698
       call_write_iter include/linux/fs.h:2087 [inline]
       aio_write+0x3bc/0x8d0 fs/aio.c:1639
       __io_submit_one fs/aio.c:2011 [inline]
       io_submit_one+0x11b6/0x1df0 fs/aio.c:2058
       __do_sys_io_submit fs/aio.c:2117 [inline]
       __se_sys_io_submit fs/aio.c:2087 [inline]
       __x64_sys_io_submit+0x1c3/0x360 fs/aio.c:2087
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xd5/0x270 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x6f/0x77

other info that might help us debug this:

Chain exists of:
  fs_reclaim --> jbd2_handle --> &ei->i_data_sem

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ei->i_data_sem);
                               lock(jbd2_handle);
                               lock(&ei->i_data_sem);
  lock(fs_reclaim);

 *** DEADLOCK ***

2 locks held by syz-executor.0/10916:
 #0: ffff888031ffe800 (&sb->s_type->i_mutex_key#7){++++}-{3:3}, at: inode_lock include/linux/fs.h:804 [inline]
 #0: ffff888031ffe800 (&sb->s_type->i_mutex_key#7){++++}-{3:3}, at: ext4_buffered_write_iter+0xb0/0x3d0 fs/ext4/file.c:294
 #1: ffff888031ffe688 (&ei->i_data_sem){++++}-{3:3}, at: ext4_da_map_blocks fs/ext4/inode.c:1776 [inline]
 #1: ffff888031ffe688 (&ei->i_data_sem){++++}-{3:3}, at: ext4_da_get_block_prep+0x770/0x1320 fs/ext4/inode.c:1817

stack backtrace:
CPU: 0 PID: 10916 Comm: syz-executor.0 Not tainted 6.8.0-rc7-syzkaller-00238-g005f6f34bd47 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
 check_noncircular+0x31b/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x244f/0x3b40 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719
 __fs_reclaim_acquire mm/page_alloc.c:3692 [inline]
 fs_reclaim_acquire+0x104/0x150 mm/page_alloc.c:3706
 might_alloc include/linux/sched/mm.h:303 [inline]
 slab_pre_alloc_hook mm/slub.c:3761 [inline]
 slab_alloc_node mm/slub.c:3842 [inline]
 kmem_cache_alloc+0x4f/0x320 mm/slub.c:3867
 kmem_cache_zalloc include/linux/slab.h:701 [inline]
 __es_alloc_extent fs/ext4/extents_status.c:483 [inline]
 ext4_es_insert_delayed_block+0x546/0x760 fs/ext4/extents_status.c:2090
 ext4_insert_delayed_block fs/ext4/inode.c:1676 [inline]
 ext4_da_map_blocks fs/ext4/inode.c:1777 [inline]
 ext4_da_get_block_prep+0x850/0x1320 fs/ext4/inode.c:1817
 ext4_block_write_begin+0x3da/0xef0 fs/ext4/inode.c:1055
 ext4_da_write_begin+0x40e/0x8c0 fs/ext4/inode.c:2894
 generic_perform_write+0x273/0x620 mm/filemap.c:3930
 ext4_buffered_write_iter+0x11f/0x3d0 fs/ext4/file.c:299
 ext4_file_write_iter+0x819/0x1960 fs/ext4/file.c:698
 call_write_iter include/linux/fs.h:2087 [inline]
 aio_write+0x3bc/0x8d0 fs/aio.c:1639
 __io_submit_one fs/aio.c:2011 [inline]
 io_submit_one+0x11b6/0x1df0 fs/aio.c:2058
 __do_sys_io_submit fs/aio.c:2117 [inline]
 __se_sys_io_submit fs/aio.c:2087 [inline]
 __x64_sys_io_submit+0x1c3/0x360 fs/aio.c:2087
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xd5/0x270 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x6f/0x77
RIP: 0033:0x7f28e447dda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f28e514f0c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000d1
RAX: ffffffffffffffda RBX: 00007f28e45abf80 RCX: 00007f28e447dda9
RDX: 0000000020000540 RSI: 0000000000001801 RDI: 00007f28e5126000
RBP: 00007f28e514f120 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 000000000000000b R14: 00007f28e45abf80 R15: 00007fffea5134a8
 </TASK>

Crashes (23):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/03/10 15:55 upstream 005f6f34bd47 6ee49f2e .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in ext4_da_get_block_prep
2024/03/05 08:43 upstream 90d35da658da 5fc53669 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in ext4_da_get_block_prep
2024/02/18 00:29 upstream ced590523156 578f7538 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in ext4_da_get_block_prep
2024/02/11 14:10 upstream 7521f258ea30 77b23aa1 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/02/07 07:37 upstream 99bd3cb0d12e 6404acf9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/02/04 12:54 upstream 3a0e92207940 a67b2c42 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/02/02 16:39 upstream 021533194476 60bf9982 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/01/31 06:34 upstream 861c0981648f 373b66cd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/01/24 06:45 upstream 615d30064886 1e153dc8 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/12/31 23:35 upstream 2639772a11c8 fb427a07 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/12/28 07:54 upstream f5837722ffec fb427a07 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/12/15 14:23 upstream 3f7168591ebf 3222d10c .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/11/09 05:56 upstream 34f763262743 4862372a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/10/28 04:20 upstream 56567a20b22b 3c418d72 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/10/04 03:40 upstream 5e62ed3b1c8a 65faba36 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/08/30 18:40 upstream 6c1b980a7e79 84803932 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/07/05 11:07 upstream d528014517f2 80298b6f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in ext4_da_get_block_prep
2023/07/27 15:44 upstream 0a8db05b571a 92476829 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2023/07/07 11:10 upstream a452483508d7 22ae5830 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in ext4_da_get_block_prep
2024/02/29 12:38 bpf dced881ead78 352ab904 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in ext4_da_get_block_prep
2023/12/09 08:02 net 1ae4cd3cbdd0 28b24332 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in ext4_da_get_block_prep
2023/12/08 18:58 net bd4a816752ba 28b24332 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in ext4_da_get_block_prep
2023/10/18 02:04 net 95535e37e895 342b9c55 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in ext4_da_get_block_prep
* Struck through repros no longer work on HEAD.