syzbot


possible deadlock in do_qc

Status: upstream: reported on 2024/04/07 05:18
Subsystems: gfs2
[Documentation on labels]
Reported-by: syzbot+8ef337b733667f9a7ec8@syzkaller.appspotmail.com
First crash: 27d, last: 3h25m
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] Monthly gfs2 report (Apr 2024) 0 (1) 2024/04/29 12:34
[syzbot] [gfs2?] possible deadlock in do_qc 0 (1) 2024/04/07 05:18
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 possible deadlock in do_qc 12 1h48m 2d18h 0/3 upstream: reported on 2024/04/27 22:18
linux-6.1 possible deadlock in do_qc 9 22h27m 2d15h 0/3 upstream: reported on 2024/04/28 01:25

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.9.0-rc6-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.2/6614 is trying to acquire lock:
ffff888046750ae8 (&sdp->sd_quota_mutex){+.+.}-{3:3}, at: do_qc+0xac/0x5e0 fs/gfs2/quota.c:720

but task is already holding lock:
ffff8880487685e8 (&ip->i_rw_mutex){++++}-{3:3}, at: sweep_bh_for_rgrps fs/gfs2/bmap.c:1529 [inline]
ffff8880487685e8 (&ip->i_rw_mutex){++++}-{3:3}, at: punch_hole+0x1bb6/0x37b0 fs/gfs2/bmap.c:1853

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&ip->i_rw_mutex){++++}-{3:3}:
       down_read+0x9a/0x330 kernel/locking/rwsem.c:1526
       __gfs2_iomap_get+0x146/0x1850 fs/gfs2/bmap.c:859
       gfs2_iomap_get+0xb1/0x1f0 fs/gfs2/bmap.c:1413
       bh_get+0x25e/0x6e0 fs/gfs2/quota.c:417
       qdsb_get.part.0+0x116/0x3e0 fs/gfs2/quota.c:566
       qdsb_get fs/gfs2/quota.c:678 [inline]
       gfs2_quota_hold+0x2be/0x7d0 fs/gfs2/quota.c:646
       punch_hole+0xe3c/0x37b0 fs/gfs2/bmap.c:1814
       gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
       iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
       iomap_file_buffered_write+0x202/0x9c0 fs/iomap/buffered-io.c:1014
       gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1059
       gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1163
       call_write_iter include/linux/fs.h:2110 [inline]
       new_sync_write fs/read_write.c:497 [inline]
       vfs_write+0x6db/0x1100 fs/read_write.c:590
       ksys_write+0x12f/0x260 fs/read_write.c:643
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&sdp->sd_quota_mutex){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       do_qc+0xac/0x5e0 fs/gfs2/quota.c:720
       gfs2_quota_change+0x36a/0x550 fs/gfs2/quota.c:1315
       punch_hole+0x29c0/0x37b0 fs/gfs2/bmap.c:1954
       gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
       iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
       iomap_file_buffered_write+0x202/0x9c0 fs/iomap/buffered-io.c:1014
       gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1059
       gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1163
       call_write_iter include/linux/fs.h:2110 [inline]
       new_sync_write fs/read_write.c:497 [inline]
       vfs_write+0x6db/0x1100 fs/read_write.c:590
       ksys_write+0x12f/0x260 fs/read_write.c:643
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ip->i_rw_mutex);
                               lock(&sdp->sd_quota_mutex);
                               lock(&ip->i_rw_mutex);
  lock(&sdp->sd_quota_mutex);

 *** DEADLOCK ***

6 locks held by syz-executor.2/6614:
 #0: ffff888024ef4ac8 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xeb/0x180 fs/file.c:1191
 #1: ffff888046470420 (sb_writers#25){.+.+}-{0:0}, at: ksys_write+0x12f/0x260 fs/read_write.c:643
 #2: ffff888048768150 (&sb->s_type->i_mutex_key#32){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:795 [inline]
 #2: ffff888048768150 (&sb->s_type->i_mutex_key#32){+.+.}-{3:3}, at: gfs2_file_write_iter+0x1fb/0x10b0 fs/gfs2/file.c:1114
 #3: ffff888046470610 (sb_internal#5){.+.+}-{0:0}, at: gfs2_trans_begin+0x74/0x100 fs/gfs2/trans.c:118
 #4: ffff888046751060 (&sdp->sd_log_flush_lock){.+.+}-{3:3}, at: __gfs2_trans_begin+0x533/0xb80 fs/gfs2/trans.c:87
 #5: ffff8880487685e8 (&ip->i_rw_mutex){++++}-{3:3}, at: sweep_bh_for_rgrps fs/gfs2/bmap.c:1529 [inline]
 #5: ffff8880487685e8 (&ip->i_rw_mutex){++++}-{3:3}, at: punch_hole+0x1bb6/0x37b0 fs/gfs2/bmap.c:1853

stack backtrace:
CPU: 3 PID: 6614 Comm: syz-executor.2 Not tainted 6.9.0-rc6-syzkaller #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:114
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 __mutex_lock_common kernel/locking/mutex.c:608 [inline]
 __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
 do_qc+0xac/0x5e0 fs/gfs2/quota.c:720
 gfs2_quota_change+0x36a/0x550 fs/gfs2/quota.c:1315
 punch_hole+0x29c0/0x37b0 fs/gfs2/bmap.c:1954
 gfs2_iomap_end+0x422/0x780 fs/gfs2/bmap.c:1174
 iomap_iter+0x249/0x10b0 fs/iomap/iter.c:79
 iomap_file_buffered_write+0x202/0x9c0 fs/iomap/buffered-io.c:1014
 gfs2_file_buffered_write+0x3ee/0xaa0 fs/gfs2/file.c:1059
 gfs2_file_write_iter+0x97d/0x10b0 fs/gfs2/file.c:1163
 call_write_iter include/linux/fs.h:2110 [inline]
 new_sync_write fs/read_write.c:497 [inline]
 vfs_write+0x6db/0x1100 fs/read_write.c:590
 ksys_write+0x12f/0x260 fs/read_write.c:643
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fd1b4c7dea9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fd1b59050c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007fd1b4dabf80 RCX: 00007fd1b4c7dea9
RDX: 000000000208e24b RSI: 0000000020000000 RDI: 0000000000000004
RBP: 00007fd1b4cca4a4 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fd1b4dabf80 R15: 00007ffda12f2338
 </TASK>

Crashes (92):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/04/29 10:51 upstream e67572cd2204 27e33c58 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/29 04:17 upstream 245c8e81741b 07b455f9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/28 22:47 upstream 245c8e81741b 07b455f9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/28 11:14 upstream 2c8159388952 07b455f9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/28 07:26 upstream 2c8159388952 07b455f9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/25 05:54 upstream e88c4cfcb7b8 8bdc0f22 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/23 07:08 upstream a2c63a3f3d68 21339d7b .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/23 02:48 upstream a2c63a3f3d68 21339d7b .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/21 23:54 upstream 48cf398f15fc af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/20 06:03 upstream dbe0a7be2838 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/20 06:02 upstream dbe0a7be2838 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/19 10:20 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/19 07:20 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/19 07:20 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/19 03:58 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/19 03:58 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/18 17:18 upstream 8cd26fd90c1a af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/17 03:30 upstream 96fca68c4fbf 18f6e127 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in do_qc
2024/04/29 07:59 upstream e67572cd2204 27e33c58 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/28 02:42 upstream 2c8159388952 07b455f9 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/25 16:24 upstream e88c4cfcb7b8 8bdc0f22 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/22 15:35 upstream ed30a4a51bb1 36c961ad .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/20 13:55 upstream 13a2e429f644 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/20 05:36 upstream dbe0a7be2838 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/19 10:23 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/19 07:23 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/19 07:23 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/19 04:02 upstream 2668e3ae2ef3 af24b050 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/17 03:41 upstream 96fca68c4fbf 18f6e127 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/17 03:39 upstream 96fca68c4fbf 18f6e127 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/17 03:39 upstream 96fca68c4fbf 18f6e127 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in do_qc
2024/04/30 13:28 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6a71d2909427 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/28 01:21 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6a71d2909427 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/25 13:18 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6a71d2909427 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/22 03:43 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6a71d2909427 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/21 15:32 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6a71d2909427 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/19 10:26 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/17 03:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 18f6e127 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/17 03:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 18f6e127 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/17 03:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 18f6e127 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/15 21:11 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/15 21:06 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/15 21:06 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/15 21:06 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci b5d2afe8745b 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
2024/04/03 05:08 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 707081b61156 7925100d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in do_qc
* Struck through repros no longer work on HEAD.