syzbot


possible deadlock in __bch2_trans_relock

Status: upstream: reported C repro on 2024/10/23 18:30
Subsystems: bcachefs
[Documentation on labels]
Reported-by: syzbot+e088be3c2d5c05aaac35@syzkaller.appspotmail.com
Fix commit: bcachefs: bch2_trans_relock() is trylock for lockdep
Patched on: [ci-upstream-linux-next-kasan-gce-root], missing on: [ci-qemu-gce-upstream-auto ci-qemu-native-arm64-kvm ci-qemu-upstream ci-qemu-upstream-386 ci-qemu2-arm32 ci-qemu2-arm64 ci-qemu2-arm64-compat ci-qemu2-arm64-mte ci-qemu2-riscv64 ci-snapshot-upstream-root ci-upstream-bpf-kasan-gce ci-upstream-bpf-next-kasan-gce ci-upstream-gce-arm64 ci-upstream-gce-leak ci-upstream-kasan-badwrites-root ci-upstream-kasan-gce ci-upstream-kasan-gce-386 ci-upstream-kasan-gce-root ci-upstream-kasan-gce-selinux-root ci-upstream-kasan-gce-smack-root ci-upstream-kmsan-gce-386-root ci-upstream-kmsan-gce-root ci-upstream-net-kasan-gce ci-upstream-net-this-kasan-gce ci2-upstream-fs ci2-upstream-kcsan-gce ci2-upstream-usb]
First crash: 47d, last: 4d04h
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] [bcachefs?] possible deadlock in trans_set_locked 4 (5) 2024/12/02 14:01
[syzbot] [bcachefs?] possible deadlock in __bch2_trans_relock 0 (2) 2024/11/28 23:06

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.12.0-rc3-syzkaller-00389-g3d5ad2d4eca3 #0 Not tainted
------------------------------------------------------
syz-executor733/5115 is trying to acquire lock:
ffff888038550128 (bcachefs_btree){+.+.}-{0:0}, at: trans_set_locked fs/bcachefs/btree_locking.h:194 [inline]
ffff888038550128 (bcachefs_btree){+.+.}-{0:0}, at: __bch2_trans_relock+0x382/0x5f0 fs/bcachefs/btree_locking.c:785

but task is already holding lock:
ffff8880424e1548 (&c->fsck_error_msgs_lock){+.+.}-{3:3}, at: __bch2_fsck_err+0x3dc/0x15f0 fs/bcachefs/error.c:279

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&c->fsck_error_msgs_lock){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       __bch2_fsck_err+0x3dc/0x15f0 fs/bcachefs/error.c:279
       bch2_check_alloc_hole_freespace+0x816/0x1180 fs/bcachefs/alloc_background.c:1278
       bch2_check_alloc_info+0x20f8/0x5330 fs/bcachefs/alloc_background.c:1547
       bch2_run_recovery_pass+0xf0/0x1e0 fs/bcachefs/recovery_passes.c:185
       bch2_run_online_recovery_passes+0x85/0x150 fs/bcachefs/recovery_passes.c:206
       bch2_fsck_online_thread_fn+0x1da/0x410 fs/bcachefs/chardev.c:798
       thread_with_stdio_fn+0x5f/0x130 fs/bcachefs/thread_with_file.c:298
       kthread+0x2f0/0x390 kernel/kthread.c:389
       ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

-> #0 (bcachefs_btree){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3161 [inline]
       check_prevs_add kernel/locking/lockdep.c:3280 [inline]
       validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
       __lock_acquire+0x1384/0x2050 kernel/locking/lockdep.c:5202
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
       trans_set_locked fs/bcachefs/btree_locking.h:194 [inline]
       __bch2_trans_relock+0x397/0x5f0 fs/bcachefs/btree_locking.c:785
       __bch2_fsck_err+0x131d/0x15f0 fs/bcachefs/error.c:360
       bch2_check_alloc_hole_freespace+0x816/0x1180 fs/bcachefs/alloc_background.c:1278
       bch2_check_alloc_info+0x20f8/0x5330 fs/bcachefs/alloc_background.c:1547
       bch2_run_recovery_pass+0xf0/0x1e0 fs/bcachefs/recovery_passes.c:185
       bch2_run_online_recovery_passes+0x85/0x150 fs/bcachefs/recovery_passes.c:206
       bch2_fsck_online_thread_fn+0x1da/0x410 fs/bcachefs/chardev.c:798
       thread_with_stdio_fn+0x5f/0x130 fs/bcachefs/thread_with_file.c:298
       kthread+0x2f0/0x390 kernel/kthread.c:389
       ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&c->fsck_error_msgs_lock);
                               lock(bcachefs_btree);
                               lock(&c->fsck_error_msgs_lock);
  lock(bcachefs_btree);

 *** DEADLOCK ***

3 locks held by syz-executor733/5115:
 #0: ffff888042480278 (&c->state_lock){++++}-{3:3}, at: bch2_run_online_recovery_passes+0x32/0x150 fs/bcachefs/recovery_passes.c:198
 #1: ffff888042484398 (&c->btree_trans_barrier){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:151 [inline]
 #1: ffff888042484398 (&c->btree_trans_barrier){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:250 [inline]
 #1: ffff888042484398 (&c->btree_trans_barrier){.+.+}-{0:0}, at: __bch2_trans_get+0x7de/0xd20 fs/bcachefs/btree_iter.c:3215
 #2: ffff8880424e1548 (&c->fsck_error_msgs_lock){+.+.}-{3:3}, at: __bch2_fsck_err+0x3dc/0x15f0 fs/bcachefs/error.c:279

stack backtrace:
CPU: 0 UID: 0 PID: 5115 Comm: syz-executor733 Not tainted 6.12.0-rc3-syzkaller-00389-g3d5ad2d4eca3 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_circular_bug+0x13a/0x1b0 kernel/locking/lockdep.c:2074
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2206
 check_prev_add kernel/locking/lockdep.c:3161 [inline]
 check_prevs_add kernel/locking/lockdep.c:3280 [inline]
 validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
 __lock_acquire+0x1384/0x2050 kernel/locking/lockdep.c:5202
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
 trans_set_locked fs/bcachefs/btree_locking.h:194 [inline]
 __bch2_trans_relock+0x397/0x5f0 fs/bcachefs/btree_locking.c:785
 __bch2_fsck_err+0x131d/0x15f0 fs/bcachefs/error.c:360
 bch2_check_alloc_hole_freespace+0x816/0x1180 fs/bcachefs/alloc_background.c:1278
 bch2_check_alloc_info+0x20f8/0x5330 fs/bcachefs/alloc_background.c:1547
 bch2_run_recovery_pass+0xf0/0x1e0 fs/bcachefs/recovery_passes.c:185
 bch2_run_online_recovery_passes+0x85/0x150 fs/bcachefs/recovery_passes.c:206
 bch2_fsck_online_thread_fn+0x1da/0x410 fs/bcachefs/chardev.c:798
 thread_with_stdio_fn+0x5f/0x130 fs/bcachefs/thread_with_file.c:298
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
syz-executor733 (5115) used greatest stack depth: 11824 bytes left

Crashes (34):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/10/19 20:18 upstream 3d5ad2d4eca3 cd6fc0a3 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] [mounted in repro #1] [mounted in repro #2] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/19 19:26 upstream 3d5ad2d4eca3 cd6fc0a3 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] [mounted in repro #1] [mounted in repro #2] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/12/01 21:01 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/27 17:37 upstream aaf20f870da0 5df23865 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/23 18:12 upstream 228a1157fb9f 68da6d95 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/23 09:26 upstream 06afb0f36106 68da6d95 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/23 09:26 upstream 06afb0f36106 68da6d95 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/17 12:43 upstream 4a5df3796467 cfe3a04a .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/14 21:21 upstream cfaaa7d010d1 77f3eeb7 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/13 23:49 upstream f1b785f4c787 4dfba277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/13 22:22 upstream f1b785f4c787 4dfba277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/11 01:57 upstream a9cda7c0ffed 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/10 13:16 upstream de2f378f2b77 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/08 21:40 upstream f1dce1f09380 d40c056e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/07 05:44 upstream 7758b206117d df3dc63b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/06 11:22 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/06 09:35 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/05 22:21 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/05 22:12 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/05 08:35 upstream 557329bcecc2 509da429 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/03 22:43 upstream b9021de3ec2f f00eed24 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/03 22:11 upstream b9021de3ec2f f00eed24 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/02 02:12 upstream 6c52d4da1c74 f00eed24 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/11/01 05:13 upstream 90602c251cda 96eb609f .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/30 00:02 upstream e42b1a9a2557 66aeb999 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/29 19:36 upstream e42b1a9a2557 66aeb999 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/29 19:36 upstream e42b1a9a2557 66aeb999 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/28 17:33 upstream 819837584309 9efb3cc7 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/28 06:48 upstream ea1fda89f5b2 65e8686b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/26 07:01 upstream c71f8fb4dc91 65e8686b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/24 20:10 upstream c2ee9f594da8 c08e46d6 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/21 20:55 upstream 42f7652d3eb5 14517542 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/21 04:47 upstream db87114dcf13 cd6fc0a3 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
2024/10/19 18:28 upstream 3d5ad2d4eca3 cd6fc0a3 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __bch2_trans_relock
* Struck through repros no longer work on HEAD.