syzbot


possible deadlock in __jbd2_log_wait_for_space

Status: upstream: reported C repro on 2023/03/07 17:44
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+385d1b32404207ed55d6@syzkaller.appspotmail.com
First crash: 635d, last: 59d
Bug presence (2)
Date Name Commit Repro Result
2023/12/09 linux-5.15.y (ToT) 8a1d809b0545 C [report] possible deadlock in __jbd2_log_wait_for_space
2023/12/09 upstream (ToT) f2e8a57ee903 C Didn't crash
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in __jbd2_log_wait_for_space ext4 C error 34993 598d 847d 22/28 fixed on 2023/06/08 14:41
linux-6.1 possible deadlock in __jbd2_log_wait_for_space origin:lts-only C done 1121 153d 635d 0/3 upstream: reported C repro on 2023/03/07 18:21
Last patch testing requests (4)
Created Duration User Patch Repo Result
2024/10/03 18:58 11m retest repro linux-5.15.y report log
2024/10/03 18:58 10m retest repro linux-5.15.y report log
2024/10/03 18:58 10m retest repro linux-5.15.y report log
2024/10/03 18:58 12m retest repro linux-5.15.y report log
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/01/02 17:25 1h00m fix candidate upstream OK (0) job log

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
5.15.157-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor950/3559 is trying to acquire lock:
ffff88807ef123f8 (&journal->j_checkpoint_mutex){+.+.}-{3:3}, at: __jbd2_log_wait_for_space+0x213/0x760 fs/jbd2/checkpoint.c:71

but task is already holding lock:
ffff888071a2dda8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: inode_lock include/linux/fs.h:789 [inline]
ffff888071a2dda8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: ext4_buffered_write_iter+0xa0/0x360 fs/ext4/file.c:262

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&sb->s_type->i_mutex_key#9){++++}-{3:3}:
       lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
       down_read+0x45/0x2e0 kernel/locking/rwsem.c:1498
       inode_lock_shared include/linux/fs.h:799 [inline]
       ext4_bmap+0x4b/0x410 fs/ext4/inode.c:3152
       bmap+0xa1/0xd0 fs/inode.c:1756
       jbd2_journal_bmap fs/jbd2/journal.c:980 [inline]
       __jbd2_journal_erase fs/jbd2/journal.c:1790 [inline]
       jbd2_journal_flush+0x7a2/0xc90 fs/jbd2/journal.c:2496
       ext4_ioctl_checkpoint fs/ext4/ioctl.c:849 [inline]
       __ext4_ioctl fs/ext4/ioctl.c:1267 [inline]
       ext4_ioctl+0x3249/0x5b80 fs/ext4/ioctl.c:1276
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:874 [inline]
       __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:860
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #0 (&journal->j_checkpoint_mutex){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3053 [inline]
       check_prevs_add kernel/locking/lockdep.c:3172 [inline]
       validate_chain+0x1649/0x5930 kernel/locking/lockdep.c:3788
       __lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5012
       lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
       __mutex_lock_common+0x1da/0x25a0 kernel/locking/mutex.c:596
       mutex_lock_io_nested+0x45/0x60 kernel/locking/mutex.c:777
       __jbd2_log_wait_for_space+0x213/0x760 fs/jbd2/checkpoint.c:71
       add_transaction_credits+0x950/0xc00 fs/jbd2/transaction.c:299
       start_this_handle+0x747/0x1570 fs/jbd2/transaction.c:423
       jbd2__journal_start+0x2d1/0x5c0 fs/jbd2/transaction.c:521
       __ext4_journal_start_sb+0x175/0x370 fs/ext4/ext4_jbd2.c:105
       __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
       ext4_dirty_inode+0x8b/0x100 fs/ext4/inode.c:5990
       __mark_inode_dirty+0x2fd/0xd60 fs/fs-writeback.c:2464
       generic_update_time+0x1cb/0x1e0 fs/inode.c:1817
       inode_update_time fs/inode.c:1830 [inline]
       file_update_time+0x3eb/0x460 fs/inode.c:2019
       ext4_write_checks+0x24a/0x2c0 fs/ext4/file.c:247
       ext4_buffered_write_iter+0xac/0x360 fs/ext4/file.c:263
       ext4_file_write_iter+0x87c/0x1990
       __kernel_write+0x5b1/0xa60 fs/read_write.c:539
       __dump_emit+0x264/0x3a0 fs/coredump.c:875
       dump_user_range+0x91/0x320 fs/coredump.c:949
       elf_core_dump+0x3c7d/0x4570 fs/binfmt_elf.c:2285
       do_coredump+0x1852/0x31e0 fs/coredump.c:826
       get_signal+0xc06/0x14e0 kernel/signal.c:2875
       arch_do_signal_or_restart+0xc3/0x1890 arch/x86/kernel/signal.c:867
       handle_signal_work kernel/entry/common.c:154 [inline]
       exit_to_user_mode_loop+0x97/0x130 kernel/entry/common.c:178
       exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:214
       irqentry_exit_to_user_mode+0x5/0x30 kernel/entry/common.c:320
       exc_page_fault+0x342/0x740 arch/x86/mm/fault.c:1535
       asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:568

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&sb->s_type->i_mutex_key#9);
                               lock(&journal->j_checkpoint_mutex);
                               lock(&sb->s_type->i_mutex_key#9);
  lock(&journal->j_checkpoint_mutex);

 *** DEADLOCK ***

2 locks held by syz-executor950/3559:
 #0: ffff88807ef0e460
 (sb_writers#5){.+.+}-{0:0}, at: do_coredump+0x1825/0x31e0 fs/coredump.c:825
 #1: ffff888071a2dda8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: inode_lock include/linux/fs.h:789 [inline]
 #1: ffff888071a2dda8 (&sb->s_type->i_mutex_key#9){++++}-{3:3}, at: ext4_buffered_write_iter+0xa0/0x360 fs/ext4/file.c:262

stack backtrace:
CPU: 0 PID: 3559 Comm: syz-executor950 Not tainted 5.15.157-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 check_noncircular+0x2f8/0x3b0 kernel/locking/lockdep.c:2133
 check_prev_add kernel/locking/lockdep.c:3053 [inline]
 check_prevs_add kernel/locking/lockdep.c:3172 [inline]
 validate_chain+0x1649/0x5930 kernel/locking/lockdep.c:3788
 __lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5012
 lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
 __mutex_lock_common+0x1da/0x25a0 kernel/locking/mutex.c:596
 mutex_lock_io_nested+0x45/0x60 kernel/locking/mutex.c:777
 __jbd2_log_wait_for_space+0x213/0x760 fs/jbd2/checkpoint.c:71
 add_transaction_credits+0x950/0xc00 fs/jbd2/transaction.c:299
 start_this_handle+0x747/0x1570 fs/jbd2/transaction.c:423
 jbd2__journal_start+0x2d1/0x5c0 fs/jbd2/transaction.c:521
 __ext4_journal_start_sb+0x175/0x370 fs/ext4/ext4_jbd2.c:105
 __ext4_journal_start fs/ext4/ext4_jbd2.h:326 [inline]
 ext4_dirty_inode+0x8b/0x100 fs/ext4/inode.c:5990
 __mark_inode_dirty+0x2fd/0xd60 fs/fs-writeback.c:2464
 generic_update_time+0x1cb/0x1e0 fs/inode.c:1817
 inode_update_time fs/inode.c:1830 [inline]
 file_update_time+0x3eb/0x460 fs/inode.c:2019
 ext4_write_checks+0x24a/0x2c0 fs/ext4/file.c:247
 ext4_buffered_write_iter+0xac/0x360 fs/ext4/file.c:263
 ext4_file_write_iter+0x87c/0x1990
 __kernel_write+0x5b1/0xa60 fs/read_write.c:539
 __dump_emit+0x264/0x3a0 fs/coredump.c:875
 dump_user_range+0x91/0x320 fs/coredump.c:949
 elf_core_dump+0x3c7d/0x4570 fs/binfmt_elf.c:2285
 do_coredump+0x1852/0x31e0 fs/coredump.c:826
 get_signal+0xc06/0x14e0 kernel/signal.c:2875
 arch_do_signal_or_restart+0xc3/0x1890 arch/x86/kernel/signal.c:867
 handle_signal_work kernel/entry/common.c:154 [inline]
 exit_to_user_mode_loop+0x97/0x130 kernel/entry/common.c:178
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:214
 irqentry_exit_to_user_mode+0x5/0x30 kernel/entry/common.c:320
 exc_page_fault+0x342/0x740 arch/x86/mm/fault.c:1535
 asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:568
RIP: 0033:0x7fc1a195e89e
Code: fd d7 c9 0f bc d1 c5 fe 7f 27 c5 fe 7f 6f 20 c5 fe 7f 77 40 c5 fe 7f 7f 60 49 83 c0 1f 49 29 d0 48 8d 7c 17 61 e9 d2 04 00 00 <c5> fe 6f 1e c5 fe 6f 56 20 c5 fd 74 cb c5 fd d7 d1 49 83 f8 21 0f
RSP: 002b:00007fc1a1901108 EFLAGS: 00010287
RAX: 00007fc1a1901110 RBX: 00007fc1a1901110 RCX: 0000000000000000
RDX: 00000000000000e0 RSI: 0000000000000000 RDI: 00007fc1a1901110
RBP: 0000000000000001 R08: 00000000000000e0 R09: 00007fc1a19016c0
R10: 00007ffc66c52457 R11: 0000000000000202 R12: 00007fc1a19b91dc
R13: 00007fc1a19ec31c R14: 0030656c69662f2e R15: 00007ffc66c52458
 </TASK>

Crashes (1257):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/01 20:15 linux-5.15.y b925f60c6ee7 3ba885bc .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2023/12/09 06:48 linux-5.15.y 8a1d809b0545 28b24332 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/03/22 18:35 linux-5.15.y b95c01af2113 4b6cdce6 .config console log report syz [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/03/08 00:42 linux-5.15.y 574362648507 2b789849 .config console log report syz [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/07/02 04:54 linux-5.15.y 4878aadf2d15 b294e901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/07/01 19:56 linux-5.15.y 4878aadf2d15 b294e901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/26 14:49 linux-5.15.y 4878aadf2d15 c6d33a01 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/19 16:23 linux-5.15.y 4878aadf2d15 41b7e219 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/18 16:21 linux-5.15.y 4878aadf2d15 639d6cdf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/18 02:55 linux-5.15.y 4878aadf2d15 ce6011bc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/18 02:06 linux-5.15.y 4878aadf2d15 ce6011bc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/17 14:58 linux-5.15.y 4878aadf2d15 1f11cfd7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/17 00:17 linux-5.15.y 4878aadf2d15 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/14 16:53 linux-5.15.y c61bd26ae81a 8d849073 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/13 14:08 linux-5.15.y c61bd26ae81a a9616ff5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/12 08:54 linux-5.15.y c61bd26ae81a 4d75f4f7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/07 01:28 linux-5.15.y c61bd26ae81a 121701b6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/05 05:27 linux-5.15.y c61bd26ae81a e1e2c66e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/06/04 04:16 linux-5.15.y c61bd26ae81a a1feae05 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/25 18:10 linux-5.15.y c61bd26ae81a a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/21 08:53 linux-5.15.y 83655231580b 4c0d3ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/21 04:00 linux-5.15.y 83655231580b c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/18 05:16 linux-5.15.y 83655231580b c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/17 10:42 linux-5.15.y 83655231580b c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/16 19:36 linux-5.15.y 284087d4f7d5 c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/15 16:03 linux-5.15.y 284087d4f7d5 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/15 05:23 linux-5.15.y 284087d4f7d5 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/14 22:44 linux-5.15.y 284087d4f7d5 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/11 04:28 linux-5.15.y 284087d4f7d5 f7c35481 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/10 04:51 linux-5.15.y 284087d4f7d5 de979bc2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/10 01:05 linux-5.15.y 284087d4f7d5 de979bc2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/09 09:00 linux-5.15.y 284087d4f7d5 05079661 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/08 23:22 linux-5.15.y 284087d4f7d5 20bf80e1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/08 19:28 linux-5.15.y 284087d4f7d5 20bf80e1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/05/08 18:27 linux-5.15.y 284087d4f7d5 20bf80e1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
2024/03/25 07:16 linux-5.15.y b95c01af2113 0ea90952 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in __jbd2_log_wait_for_space
2024/07/02 03:07 linux-5.15.y 4878aadf2d15 b294e901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/28 23:17 linux-5.15.y 4878aadf2d15 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/28 21:22 linux-5.15.y 4878aadf2d15 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/28 16:32 linux-5.15.y 4878aadf2d15 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/27 01:05 linux-5.15.y 4878aadf2d15 c6d33a01 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/22 04:56 linux-5.15.y 4878aadf2d15 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/19 05:36 linux-5.15.y 4878aadf2d15 41b7e219 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/18 00:42 linux-5.15.y 4878aadf2d15 ce6011bc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/17 22:02 linux-5.15.y 4878aadf2d15 1f11cfd7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/12 17:46 linux-5.15.y c61bd26ae81a f815599d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/12 01:27 linux-5.15.y c61bd26ae81a 4d75f4f7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/11 23:55 linux-5.15.y c61bd26ae81a 4d75f4f7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/08 16:47 linux-5.15.y c61bd26ae81a 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/06/03 17:44 linux-5.15.y c61bd26ae81a 0aba2352 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/05/22 08:39 linux-5.15.y 83655231580b 1014eca7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/05/17 23:04 linux-5.15.y 83655231580b a12e99e7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2024/05/08 01:47 linux-5.15.y 284087d4f7d5 4cf3f9b3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in __jbd2_log_wait_for_space
2023/03/07 17:43 linux-5.15.y d9b4a0c83a2d d7ea8bc4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in __jbd2_log_wait_for_space
* Struck through repros no longer work on HEAD.