syzbot


possible deadlock in bch2_alloc_sectors_start_trans

Status: upstream: reported C repro on 2024/11/12 03:25
Subsystems: bcachefs
[Documentation on labels]
Reported-by: syzbot+d4b38c802ea425ccf857@syzkaller.appspotmail.com
First crash: 9d22h, last: 5d03h
Cause bisection: the cause commit could be any of (bisect log):
  9104fc192870 bcachefs: Fix accounting read + device removal
  49fd90b2cc33 bcachefs: Fix unlocked access to c->disk_sb.sb in bch2_replicas_entry_validate()
  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [bcachefs?] possible deadlock in bch2_alloc_sectors_start_trans 0 (1) 2024/11/12 03:25
Cause bisection attempts (2)
Created Duration User Patch Repo Result
2024/11/16 08:52 8h50m bisect linux-next OK (2) job log
2024/11/08 06:19 5h58m bisect linux-next error job log

Sample crash report:
  io_time[WRITE]    256
  fragmentation     0
  bp_start          8
  incorrectly set at freespace:0:27:0 (free 0, genbits 0 should be 0), fixing
============================================
WARNING: possible recursive locking detected
6.12.0-rc6-next-20241107-syzkaller #0 Not tainted
--------------------------------------------
kworker/1:2/58 is trying to acquire lock:
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_trans_mutex_lock_norelock fs/bcachefs/alloc_foreground.c:43 [inline]
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: writepoint_find fs/bcachefs/alloc_foreground.c:1249 [inline]
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_alloc_sectors_start_trans+0x956/0x2030 fs/bcachefs/alloc_foreground.c:1355

but task is already holding lock:
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_trans_mutex_lock_norelock fs/bcachefs/alloc_foreground.c:41 [inline]
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: writepoint_find fs/bcachefs/alloc_foreground.c:1249 [inline]
ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_alloc_sectors_start_trans+0x2e8/0x2030 fs/bcachefs/alloc_foreground.c:1355

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&wp->lock);
  lock(&wp->lock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

7 locks held by kworker/1:2/58:
 #0: ffff88802070fd48 ((wq_completion)bcachefs_write_ref){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3204 [inline]
 #0: ffff88802070fd48 ((wq_completion)bcachefs_write_ref){+.+.}-{0:0}, at: process_scheduled_works+0x93b/0x1850 kernel/workqueue.c:3310
 #1: ffffc9000133fd00 ((work_completion)(&ca->invalidate_work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3205 [inline]
 #1: ffffc9000133fd00 ((work_completion)(&ca->invalidate_work)){+.+.}-{0:0}, at: process_scheduled_works+0x976/0x1850 kernel/workqueue.c:3310
 #2: ffff888078704750 (&wb->flushing.lock){+.+.}-{4:4}, at: bch2_btree_write_buffer_flush_nocheck_rw fs/bcachefs/btree_write_buffer.c:543 [inline]
 #2: ffff888078704750 (&wb->flushing.lock){+.+.}-{4:4}, at: bch2_btree_write_buffer_tryflush+0x14b/0x1c0 fs/bcachefs/btree_write_buffer.c:558
 #3: ffff8880787043a8 (&c->btree_trans_barrier){.+.+}-{0:0}, at: srcu_lock_acquire include/linux/srcu.h:158 [inline]
 #3: ffff8880787043a8 (&c->btree_trans_barrier){.+.+}-{0:0}, at: srcu_read_lock include/linux/srcu.h:255 [inline]
 #3: ffff8880787043a8 (&c->btree_trans_barrier){.+.+}-{0:0}, at: bch2_trans_srcu_lock+0x9a/0x1a0 fs/bcachefs/btree_iter.c:3195
 #4: ffff888078726710 (&c->gc_lock){++++}-{4:4}, at: bch2_btree_update_start+0x682/0x14e0 fs/bcachefs/btree_update_interior.c:1191
 #5: ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_trans_mutex_lock_norelock fs/bcachefs/alloc_foreground.c:41 [inline]
 #5: ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: writepoint_find fs/bcachefs/alloc_foreground.c:1249 [inline]
 #5: ffff88807871dc38 (&wp->lock){+.+.}-{4:4}, at: bch2_alloc_sectors_start_trans+0x2e8/0x2030 fs/bcachefs/alloc_foreground.c:1355
 #6: ffff888078726710 (&c->gc_lock){++++}-{4:4}, at: bch2_btree_update_start+0x682/0x14e0 fs/bcachefs/btree_update_interior.c:1191

stack backtrace:
CPU: 1 UID: 0 PID: 58 Comm: kworker/1:2 Not tainted 6.12.0-rc6-next-20241107-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/30/2024
Workqueue: bcachefs_write_ref bch2_do_invalidates_work
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 print_deadlock_bug+0x483/0x620 kernel/locking/lockdep.c:3037
 check_deadlock kernel/locking/lockdep.c:3089 [inline]
 validate_chain+0x15e2/0x5920 kernel/locking/lockdep.c:3891
 __lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5226
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
 __mutex_lock_common kernel/locking/mutex.c:585 [inline]
 __mutex_lock+0x1ac/0xee0 kernel/locking/mutex.c:735
 bch2_trans_mutex_lock_norelock fs/bcachefs/alloc_foreground.c:43 [inline]
 writepoint_find fs/bcachefs/alloc_foreground.c:1249 [inline]
 bch2_alloc_sectors_start_trans+0x956/0x2030 fs/bcachefs/alloc_foreground.c:1355
 __bch2_btree_node_alloc fs/bcachefs/btree_update_interior.c:333 [inline]
 bch2_btree_reserve_get+0x612/0x1890 fs/bcachefs/btree_update_interior.c:543
 bch2_btree_update_start+0xe56/0x14e0 fs/bcachefs/btree_update_interior.c:1241
 bch2_btree_split_leaf+0x121/0x880 fs/bcachefs/btree_update_interior.c:1857
 bch2_trans_commit_error+0x212/0x1390 fs/bcachefs/btree_trans_commit.c:918
 __bch2_trans_commit+0x8069/0x9610 fs/bcachefs/btree_trans_commit.c:1099
 bch2_trans_commit fs/bcachefs/btree_update.h:182 [inline]
 bch2_check_discard_freespace_key+0xba7/0x1120 fs/bcachefs/alloc_background.c:1393
 try_alloc_bucket fs/bcachefs/alloc_foreground.c:287 [inline]
 bch2_bucket_alloc_freelist fs/bcachefs/alloc_foreground.c:463 [inline]
 bch2_bucket_alloc_trans+0x1526/0x31a0 fs/bcachefs/alloc_foreground.c:590
 bch2_bucket_alloc_set_trans+0x517/0xd30 fs/bcachefs/alloc_foreground.c:750
 __open_bucket_add_buckets+0x13d0/0x1ec0 fs/bcachefs/alloc_foreground.c:999
 open_bucket_add_buckets+0x33a/0x410 fs/bcachefs/alloc_foreground.c:1043
 bch2_alloc_sectors_start_trans+0xce9/0x2030
 __bch2_btree_node_alloc fs/bcachefs/btree_update_interior.c:333 [inline]
 bch2_btree_reserve_get+0x612/0x1890 fs/bcachefs/btree_update_interior.c:543
 bch2_btree_update_start+0xe56/0x14e0 fs/bcachefs/btree_update_interior.c:1241
 bch2_btree_split_leaf+0x121/0x880 fs/bcachefs/btree_update_interior.c:1857
 bch2_trans_commit_error+0x212/0x1390 fs/bcachefs/btree_trans_commit.c:918
 __bch2_trans_commit+0x8069/0x9610 fs/bcachefs/btree_trans_commit.c:1099
 wb_flush_one fs/bcachefs/btree_write_buffer.c:183 [inline]
 bch2_btree_write_buffer_flush_locked+0x2b23/0x5a40 fs/bcachefs/btree_write_buffer.c:375
 bch2_btree_write_buffer_flush_nocheck_rw fs/bcachefs/btree_write_buffer.c:544 [inline]
 bch2_btree_write_buffer_tryflush+0x16a/0x1c0 fs/bcachefs/btree_write_buffer.c:558
 bch2_do_invalidates_work+0x131/0x2400 fs/bcachefs/alloc_background.c:2078
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (18):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/08 03:26 linux-next 74741a050b79 c069283c .config strace log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 21:57 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 21:57 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 21:05 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 21:05 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 14:17 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/12 14:16 linux-next 929beafbe7ac 75bb1b32 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/11 10:13 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/11 10:13 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/10 20:14 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/10 20:12 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/10 20:12 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/10 20:12 linux-next 929beafbe7ac 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/08 15:16 linux-next 929beafbe7ac 179b040e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/08 15:15 linux-next 929beafbe7ac 179b040e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/08 03:17 linux-next 74741a050b79 c069283c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/08 03:16 linux-next 74741a050b79 c069283c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
2024/11/08 03:16 linux-next 74741a050b79 c069283c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root possible deadlock in bch2_alloc_sectors_start_trans
* Struck through repros no longer work on HEAD.