syzbot


possible deadlock in blk_mq_freeze_queue_nomemsave

Status: upstream: reported C repro on 2025/03/31 13:34
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+9dd7dbb1a4b915dee638@syzkaller.appspotmail.com
First crash: 27d, last: 25m
Cause bisection: introduced by (bisect log) :
commit ffa1e7ada456087c2402b37cd6b2863ced29aff0
Author: Thomas HellstrΓΆm <thomas.hellstrom@linux.intel.com>
Date: Tue Mar 18 09:55:48 2025 +0000

  block: Make request_queue lockdep splats show up earlier

Crash: possible deadlock in blk_mq_freeze_queue_nomemsave (log)
Repro: C syz .config
  
Discussions (2)
Title Replies (including bot) Last reply
[PATCH] loop: replace freezing queue with quiesce when changing loop specific setting 7 (7) 2025/04/08 05:23
[syzbot] [block?] possible deadlock in blk_mq_freeze_queue_nomemsave 1 (5) 2025/04/04 15:01
Last patch testing requests (2)
Created Duration User Patch Repo Result
2025/04/04 10:07 36m hdanton@sina.com patch upstream report log
2025/04/04 13:50 22m ming.lei@redhat.com https://github.com/ming1/linux.git syzbot-test report log

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.14.0-syzkaller-12508-g92b71befc349 #0 Not tainted
------------------------------------------------------
syz-executor351/5836 is trying to acquire lock:
ffffffff906949e8 (uevent_sock_mutex){+.+.}-{4:4}, at: uevent_net_broadcast_untagged lib/kobject_uevent.c:317 [inline]
ffffffff906949e8 (uevent_sock_mutex){+.+.}-{4:4}, at: kobject_uevent_net_broadcast lib/kobject_uevent.c:410 [inline]
ffffffff906949e8 (uevent_sock_mutex){+.+.}-{4:4}, at: kobject_uevent_env+0xb36/0x1870 lib/kobject_uevent.c:608

but task is already holding lock:
ffff888025baa7a8 (&q->q_usage_counter(io)#24){++++}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:215

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&q->q_usage_counter(io)#24){++++}-{0:0}:
       blk_alloc_queue+0x619/0x760 block/blk-core.c:461
       blk_mq_alloc_queue+0x179/0x290 block/blk-mq.c:4349
       __blk_mq_alloc_disk+0x29/0x120 block/blk-mq.c:4396
       loop_add+0x496/0xb70 drivers/block/loop.c:2067
       loop_init+0x164/0x270 drivers/block/loop.c:2302
       do_one_initcall+0x120/0x6e0 init/main.c:1257
       do_initcall_level init/main.c:1319 [inline]
       do_initcalls init/main.c:1335 [inline]
       do_basic_setup init/main.c:1354 [inline]
       kernel_init_freeable+0x5c2/0x900 init/main.c:1567
       kernel_init+0x1c/0x2b0 init/main.c:1457
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:153
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

-> #1 (fs_reclaim){+.+.}-{0:0}:
       __fs_reclaim_acquire mm/page_alloc.c:4028 [inline]
       fs_reclaim_acquire+0x102/0x150 mm/page_alloc.c:4042
       might_alloc include/linux/sched/mm.h:318 [inline]
       slab_pre_alloc_hook mm/slub.c:4102 [inline]
       slab_alloc_node mm/slub.c:4180 [inline]
       kmem_cache_alloc_node_noprof+0x57/0x3b0 mm/slub.c:4252
       __alloc_skb+0x2b2/0x380 net/core/skbuff.c:658
       alloc_skb include/linux/skbuff.h:1340 [inline]
       alloc_uevent_skb+0x7d/0x210 lib/kobject_uevent.c:289
       uevent_net_broadcast_untagged lib/kobject_uevent.c:326 [inline]
       kobject_uevent_net_broadcast lib/kobject_uevent.c:410 [inline]
       kobject_uevent_env+0xca4/0x1870 lib/kobject_uevent.c:608
       kobject_synth_uevent+0x7d4/0x8a0 lib/kobject_uevent.c:207
       bus_uevent_store+0x3d/0x90 drivers/base/bus.c:832
       bus_attr_store+0x71/0xb0 drivers/base/bus.c:172
       sysfs_kf_write+0xef/0x150 fs/sysfs/file.c:145
       kernfs_fop_write_iter+0x351/0x510 fs/kernfs/file.c:334
       new_sync_write fs/read_write.c:591 [inline]
       vfs_write+0x5ba/0x1180 fs/read_write.c:684
       ksys_write+0x12a/0x240 fs/read_write.c:736
       do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
       do_syscall_64+0xcd/0x260 arch/x86/entry/syscall_64.c:94
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (uevent_sock_mutex){+.+.}-{4:4}:
       check_prev_add kernel/locking/lockdep.c:3166 [inline]
       check_prevs_add kernel/locking/lockdep.c:3285 [inline]
       validate_chain kernel/locking/lockdep.c:3909 [inline]
       __lock_acquire+0x1173/0x1ba0 kernel/locking/lockdep.c:5235
       lock_acquire kernel/locking/lockdep.c:5866 [inline]
       lock_acquire+0x179/0x350 kernel/locking/lockdep.c:5823
       __mutex_lock_common kernel/locking/mutex.c:601 [inline]
       __mutex_lock+0x199/0xb90 kernel/locking/mutex.c:746
       uevent_net_broadcast_untagged lib/kobject_uevent.c:317 [inline]
       kobject_uevent_net_broadcast lib/kobject_uevent.c:410 [inline]
       kobject_uevent_env+0xb36/0x1870 lib/kobject_uevent.c:608
       loop_set_size drivers/block/loop.c:211 [inline]
       loop_set_size drivers/block/loop.c:208 [inline]
       loop_set_status+0x9c7/0xb90 drivers/block/loop.c:1306
       loop_set_status_old+0x162/0x1d0 drivers/block/loop.c:1405
       lo_ioctl+0x695/0x26d0 drivers/block/loop.c:1575
       blkdev_ioctl+0x274/0x6d0 block/ioctl.c:698
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:906 [inline]
       __se_sys_ioctl fs/ioctl.c:892 [inline]
       __x64_sys_ioctl+0x190/0x200 fs/ioctl.c:892
       do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
       do_syscall_64+0xcd/0x260 arch/x86/entry/syscall_64.c:94
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  uevent_sock_mutex --> fs_reclaim --> &q->q_usage_counter(io)#24

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&q->q_usage_counter(io)#24);
                               lock(fs_reclaim);
                               lock(&q->q_usage_counter(io)#24);
  lock(uevent_sock_mutex);

 *** DEADLOCK ***

3 locks held by syz-executor351/5836:
 #0: ffff888025bb5b68 (&lo->lo_mutex){+.+.}-{4:4}, at: loop_set_status+0x2a/0xb90 drivers/block/loop.c:1275
 #1: ffff888025baa7a8 (&q->q_usage_counter(io)#24){++++}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:215
 #2: ffff888025baa7e0 (&q->q_usage_counter(queue)#20){+.+.}-{0:0}, at: blk_mq_freeze_queue_nomemsave+0x15/0x20 block/blk-mq.c:215

stack backtrace:
CPU: 1 UID: 0 PID: 5836 Comm: syz-executor351 Not tainted 6.14.0-syzkaller-12508-g92b71befc349 #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/12/2025
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
 print_circular_bug+0x275/0x350 kernel/locking/lockdep.c:2079
 check_noncircular+0x14c/0x170 kernel/locking/lockdep.c:2211
 check_prev_add kernel/locking/lockdep.c:3166 [inline]
 check_prevs_add kernel/locking/lockdep.c:3285 [inline]
 validate_chain kernel/locking/lockdep.c:3909 [inline]
 __lock_acquire+0x1173/0x1ba0 kernel/locking/lockdep.c:5235
 lock_acquire kernel/locking/lockdep.c:5866 [inline]
 lock_acquire+0x179/0x350 kernel/locking/lockdep.c:5823
 __mutex_lock_common kernel/locking/mutex.c:601 [inline]
 __mutex_lock+0x199/0xb90 kernel/locking/mutex.c:746
 uevent_net_broadcast_untagged lib/kobject_uevent.c:317 [inline]
 kobject_uevent_net_broadcast lib/kobject_uevent.c:410 [inline]
 kobject_uevent_env+0xb36/0x1870 lib/kobject_uevent.c:608
 loop_set_size drivers/block/loop.c:211 [inline]
 loop_set_size drivers/block/loop.c:208 [inline]
 loop_set_status+0x9c7/0xb90 drivers/block/loop.c:1306
 loop_set_status_old+0x162/0x1d0 drivers/block/loop.c:1405
 lo_ioctl+0x695/0x26d0 drivers/block/loop.c:1575
 blkdev_ioctl+0x274/0x6d0 block/ioctl.c:698
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:906 [inline]
 __se_sys_ioctl fs/ioctl.c:892 [inline]
 __x64_sys_ioctl+0x190/0x200 fs/ioctl.c:892
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xcd/0x260 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f13e1676a69
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 c1 17 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fffa4ebb938 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f13e1676a69
RDX: 0000200000000200 RSI: 0000000000004c02 RDI: 0000000000000004
RBP: 00000000000f4240 R08: 0023706f6f6c2f76 R09: 00000000000000a0
R10: 000000000000001f R11: 0000000000000246 R12: 000000000000d925
R13: 00007fffa4ebb94c R14: 00007fffa4ebb960 R15: 00007fffa4ebb950
 </TASK>

Crashes (3345):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/04/03 08:57 upstream 92b71befc349 996a9618 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/03 08:34 upstream a1b5bd45d4ee 996a9618 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/24 06:33 upstream a79be02bba5c 9882047a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/24 00:52 upstream a79be02bba5c 9882047a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 20:50 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 20:08 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 18:42 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 18:38 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 17:28 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 15:43 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 14:46 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 13:15 upstream bc3372351d0c d971f7e2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 11:34 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 07:18 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 06:04 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 02:59 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 01:55 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 01:54 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 19:34 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 19:21 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 17:35 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 16:24 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 13:08 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 13:05 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 11:10 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 09:42 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 07:38 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 05:49 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/03/31 07:10 upstream aa918db707fb d3999433 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-badwrites-root possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/24 03:17 upstream a79be02bba5c 9882047a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 21:20 upstream a79be02bba5c 9882047a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 09:23 upstream bc3372351d0c 57d54c08 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 00:50 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 22:41 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 22:06 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 20:48 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 15:26 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 08:52 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/03/27 13:27 upstream 1a9239bb4253 928390c4 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/24 05:35 upstream a79be02bba5c 9882047a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 22:37 upstream a79be02bba5c 9882047a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 22:30 upstream a79be02bba5c 9882047a .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 10:34 upstream bc3372351d0c 57d54c08 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/23 05:12 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 23:50 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 23:50 upstream bc3372351d0c 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 14:12 upstream a33b5a08cbbd 53a8b9bd .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 05:47 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 04:45 upstream a33b5a08cbbd 2a20f901 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/22 00:08 upstream 9d7a0577c9db 2a20f901 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu2-arm64-compat possible deadlock in blk_mq_freeze_queue_nomemsave
2025/04/06 18:51 upstream f4d2ef48250a 1c65791e .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu2-arm64 possible deadlock in blk_mq_freeze_queue_nomemsave
* Struck through repros no longer work on HEAD.