syzbot


possible deadlock in do_qc

Status: upstream: reported on 2024/04/07 05:18
Subsystems: gfs2
[Documentation on labels]
Reported-by: syzbot+8ef337b733667f9a7ec8@syzkaller.appspotmail.com
First crash: 48d, last: 9h46m
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] Monthly gfs2 report (Apr 2024) 0 (1) 2024/04/29 12:34
[syzbot] [gfs2?] possible deadlock in do_qc 0 (1) 2024/04/07 05:18
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 possible deadlock in do_qc 107 4h40m 23d 0/3 upstream: reported on 2024/04/27 22:18
linux-6.1 possible deadlock in do_qc 61 14h51m 23d 0/3 upstream: reported on 2024/04/28 01:25

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.9.0-syzkaller-09429-g61307b7be41a #0 Not tainted
------------------------------------------------------
syz-executor.4/10379 is trying to acquire lock:
ffff8880769c4ae8 (&sdp->sd_quota_mutex){+.+.}-{3:3}, at: do_qc+0xac/0x5e0 fs/gfs2/quota.c:720

but task is already holding lock:
ffff88805a4bbba8 (&ip->i_rw_mutex){++++}-{3:3}, at: sweep_bh_for_rgrps fs/gfs2/bmap.c:1529 [inline]
ffff88805a4bbba8 (&ip->i_rw_mutex){++++}-{3:3}, at: punch_hole+0x1bb6/0x37b0 fs/gfs2/bmap.c:1853

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&ip->i_rw_mutex){++++}-{3:3}:
       down_read+0x9a/0x330 kernel/locking/rwsem.c:1526
       __gfs2_iomap_get+0x146/0x1850 fs/gfs2/bmap.c:859
       gfs2_iomap_get+0xb1/0x1f0 fs/gfs2/bmap.c:1413
       bh_get+0x25e/0x6e0 fs/gfs2/quota.c:417
       qdsb_get.part.0+0x116/0x3e0 fs/gfs2/quota.c:566
       qdsb_get fs/gfs2/quota.c:678 [inline]
       gfs2_quota_hold+0x2be/0x7d0 fs/gfs2/quota.c:646
       punch_hole+0xe3c/0x37b0 fs/gfs2/bmap.c:1814
       gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
       iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
       iomap_file_buffered_write+0x219/0xc20 fs/iomap/buffered-io.c:1025
       gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1060
       gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1164
       call_write_iter include/linux/fs.h:2120 [inline]
       new_sync_write fs/read_write.c:497 [inline]
       vfs_write+0x6b6/0x1120 fs/read_write.c:590
       ksys_write+0x12f/0x260 fs/read_write.c:643
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&sdp->sd_quota_mutex){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       do_qc+0xac/0x5e0 fs/gfs2/quota.c:720
       gfs2_quota_change+0x36a/0x550 fs/gfs2/quota.c:1315
       punch_hole+0x29c0/0x37b0 fs/gfs2/bmap.c:1954
       gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
       iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
       iomap_file_buffered_write+0x219/0xc20 fs/iomap/buffered-io.c:1025
       gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1060
       gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1164
       call_write_iter include/linux/fs.h:2120 [inline]
       new_sync_write fs/read_write.c:497 [inline]
       vfs_write+0x6b6/0x1120 fs/read_write.c:590
       ksys_write+0x12f/0x260 fs/read_write.c:643
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ip->i_rw_mutex);
                               lock(&sdp->sd_quota_mutex);
                               lock(&ip->i_rw_mutex);
  lock(&sdp->sd_quota_mutex);

 *** DEADLOCK ***

6 locks held by syz-executor.4/10379:
 #0: ffff88802d3bc5c8 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xeb/0x180 fs/file.c:1191
 #1: ffff888024c1c420 (sb_writers#38){.+.+}-{0:0}, at: ksys_write+0x12f/0x260 fs/read_write.c:643
 #2: ffff88805a4bb710 (&sb->s_type->i_mutex_key#46){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:791 [inline]
 #2: ffff88805a4bb710 (&sb->s_type->i_mutex_key#46){+.+.}-{3:3}, at: gfs2_file_write_iter+0x1fb/0x10b0 fs/gfs2/file.c:1115
 #3: ffff888024c1c610 (sb_internal#6){.+.+}-{0:0}, at: gfs2_trans_begin+0x74/0x100 fs/gfs2/trans.c:118
 #4: ffff8880769c5060 (&sdp->sd_log_flush_lock){++++}-{3:3}, at: __gfs2_trans_begin+0x533/0xb80 fs/gfs2/trans.c:87
 #5: ffff88805a4bbba8 (&ip->i_rw_mutex){++++}-{3:3}, at: sweep_bh_for_rgrps fs/gfs2/bmap.c:1529 [inline]
 #5: ffff88805a4bbba8 (&ip->i_rw_mutex){++++}-{3:3}, at: punch_hole+0x1bb6/0x37b0 fs/gfs2/bmap.c:1853

stack backtrace:
CPU: 1 PID: 10379 Comm: syz-executor.4 Not tainted 6.9.0-syzkaller-09429-g61307b7be41a #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:114
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 __mutex_lock_common kernel/locking/mutex.c:608 [inline]
 __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
 do_qc+0xac/0x5e0 fs/gfs2/quota.c:720
 gfs2_quota_change+0x36a/0x550 fs/gfs2/quota.c:1315
 punch_hole+0x29c0/0x37b0 fs/gfs2/bmap.c:1954
 gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
 iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
 iomap_file_buffered_write+0x219/0xc20 fs/iomap/buffered-io.c:1025
 gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1060
 gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1164
 call_write_iter include/linux/fs.h:2120 [inline]
 new_sync_write fs/read_write.c:497 [inline]
 vfs_write+0x6b6/0x1120 fs/read_write.c:590
 ksys_write+0x12f/0x260 fs/read_write.c:643
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f814b87cee9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f814c64f0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007f814b9abf80 RCX: 00007f814b87cee9
RDX: 00000000ffffff6a RSI: 0000000020000000 RDI: 0000000000000005
RBP: 00007f814b8c949e R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007f814b9abf80 R15: 00007ffe096a2e28
 </TASK>

Crashes (268):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/20 08:56 upstream 61307b7be41a c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in do_qc
2024/05/19 15:56 upstream 0450d2083be6 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in do_qc
2024/05/19 12:47 upstream 0450d2083be6 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root possible deadlock in do_qc
2024/05/16 17:47 upstream 3c999d1ae3c7 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root possible deadlock in do_qc
2024/05/16 17:22 upstream 3c999d1ae3c7 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in do_qc
2024/05/08 09:29 upstream dccb07f2914c 4cf3f9b3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in do_qc
2024/05/21 06:06 upstream 6e51b4b5bbc0 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/20 15:29 upstream eb6a9339efeb c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/19 01:26 upstream 0cc6f45cecb4 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/19 00:07 upstream 0cc6f45cecb4 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/16 19:10 upstream 3c999d1ae3c7 ad5321c6 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/14 20:47 upstream 6bfd2d442af5 fdb4c10c .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/12 10:41 upstream cf87f46fd34d 9026e142 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/11 20:06 upstream cf87f46fd34d 9026e142 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/09 15:38 upstream 45db3ab70092 de979bc2 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/08 04:52 upstream dccb07f2914c 4cf3f9b3 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/03 07:44 upstream f03359bca01b ddfc15a1 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/03 06:20 upstream f03359bca01b ddfc15a1 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/02 18:16 upstream 49a73b1652c5 22ee48a2 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/02 03:17 upstream 0106679839f7 3ba885bc .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/05/21 10:59 upstream 8f6a15f095a6 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/20 09:01 upstream eb6a9339efeb c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/19 14:38 upstream 0450d2083be6 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/18 10:37 upstream 4b377b4868ef c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/17 14:41 upstream ea5f6ad9ad96 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/17 08:38 upstream ea5f6ad9ad96 c2e07261 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/16 13:50 upstream 3c999d1ae3c7 ef5d53ed .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/11 14:06 upstream cf87f46fd34d 9026e142 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/11 03:33 upstream f4345f05c0df 9026e142 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/08 13:49 upstream dccb07f2914c 9473be88 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/06 00:24 upstream b9158815de52 610f2a54 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/02 14:24 upstream 0106679839f7 3ba885bc .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/05/21 08:53 linux-next 124cfbcd6d18 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/19 13:36 linux-next c75962170e49 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/18 07:35 linux-next c75962170e49 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/16 12:28 linux-next dbd9e2e056d8 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/11 12:27 linux-next 75fa778d74b7 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/09 10:25 linux-next 704ba27ac555 20bf80e1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/09 10:08 linux-next 704ba27ac555 20bf80e1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in do_qc
2024/05/21 08:20 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/20 05:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/18 20:38 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/17 05:26 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/15 19:28 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c 0b3dad46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/14 11:54 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/13 01:24 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/12 03:40 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/10 10:19 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci c4d6737dc9da de979bc2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/05/08 11:41 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 1c9135d29e9e 4cf3f9b3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/03 05:08 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 707081b61156 7925100d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
* Struck through repros no longer work on HEAD.