syzbot


possible deadlock in __kernfs_remove (2)

Status: upstream: reported C repro on 2024/11/08 10:18
Subsystems: kernfs
[Documentation on labels]
Reported-by: syzbot+aa419d82b68e6a7e96c5@syzkaller.appspotmail.com
First crash: 5d13h, last: 11m
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [kernfs?] possible deadlock in __kernfs_remove (2) 0 (1) 2024/11/08 10:18
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in __kernfs_remove kernfs C 438 95d 138d 28/28 fixed on 2024/10/03 15:25

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.12.0-rc6-syzkaller-00192-gf1dce1f09380 #0 Not tainted
------------------------------------------------------
syz-executor336/6364 is trying to acquire lock:
ffff888025cc2e18 (kn->active#5){++++}-{0:0}, at: __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486

but task is already holding lock:
ffff8880251264c8 (&disk->open_mutex){+.+.}-{3:3}, at: loop_reread_partitions+0x47/0x140 drivers/block/loop.c:533

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&disk->open_mutex){+.+.}-{3:3}:
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       bdev_open+0x41a/0xe20 block/bdev.c:904
       bdev_file_open_by_dev block/bdev.c:1018 [inline]
       bdev_file_open_by_dev+0x17d/0x210 block/bdev.c:993
       disk_scan_partitions+0x1ed/0x320 block/genhd.c:367
       device_add_disk+0xfbb/0x12d0 block/genhd.c:510
       pmem_attach_disk+0x9af/0x13f0 drivers/nvdimm/pmem.c:576
       nd_pmem_probe+0x1a9/0x1f0 drivers/nvdimm/pmem.c:649
       nvdimm_bus_probe+0x169/0x5d0 drivers/nvdimm/bus.c:94
       call_driver_probe drivers/base/dd.c:579 [inline]
       really_probe+0x23e/0xa90 drivers/base/dd.c:658
       __driver_probe_device+0x1de/0x440 drivers/base/dd.c:800
       driver_probe_device+0x4c/0x1b0 drivers/base/dd.c:830
       __driver_attach+0x283/0x580 drivers/base/dd.c:1216
       bus_for_each_dev+0x13c/0x1d0 drivers/base/bus.c:370
       bus_add_driver+0x2e9/0x690 drivers/base/bus.c:675
       driver_register+0x15c/0x4b0 drivers/base/driver.c:246
       __nd_driver_register+0x103/0x1a0 drivers/nvdimm/bus.c:622
       do_one_initcall+0x128/0x700 init/main.c:1269
       do_initcall_level init/main.c:1331 [inline]
       do_initcalls init/main.c:1347 [inline]
       do_basic_setup init/main.c:1366 [inline]
       kernel_init_freeable+0x5c7/0x900 init/main.c:1580
       kernel_init+0x1c/0x2b0 init/main.c:1469
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

-> #1 (&nvdimm_namespace_key){+.+.}-{3:3}:
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       device_lock include/linux/device.h:1014 [inline]
       uevent_show+0x188/0x3b0 drivers/base/core.c:2736
       dev_attr_show+0x53/0xe0 drivers/base/core.c:2430
       sysfs_kf_seq_show+0x23e/0x410 fs/sysfs/file.c:59
       seq_read_iter+0x4f4/0x12b0 fs/seq_file.c:230
       kernfs_fop_read_iter+0x414/0x580 fs/kernfs/file.c:279
       new_sync_read fs/read_write.c:488 [inline]
       vfs_read+0x87f/0xbe0 fs/read_write.c:569
       ksys_read+0x12f/0x260 fs/read_write.c:712
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (kn->active#5){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3161 [inline]
       check_prevs_add kernel/locking/lockdep.c:3280 [inline]
       validate_chain kernel/locking/lockdep.c:3904 [inline]
       __lock_acquire+0x250b/0x3ce0 kernel/locking/lockdep.c:5202
       lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5825
       kernfs_drain+0x48f/0x590 fs/kernfs/dir.c:500
       __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486
       kernfs_remove_by_name_ns+0xb2/0x130 fs/kernfs/dir.c:1694
       sysfs_remove_file include/linux/sysfs.h:773 [inline]
       device_remove_file drivers/base/core.c:3054 [inline]
       device_remove_file drivers/base/core.c:3050 [inline]
       device_del+0x381/0x9f0 drivers/base/core.c:3859
       drop_partition+0x109/0x1c0 block/partitions/core.c:273
       bdev_disk_changed+0x24d/0x1520 block/partitions/core.c:666
       loop_reread_partitions+0x70/0x140 drivers/block/loop.c:534
       loop_configure+0xef6/0x11f0 drivers/block/loop.c:1119
       lo_ioctl+0xa6b/0x18b0 drivers/block/loop.c:1533
       blkdev_ioctl+0x276/0x6d0 block/ioctl.c:693
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:907 [inline]
       __se_sys_ioctl fs/ioctl.c:893 [inline]
       __x64_sys_ioctl+0x18f/0x220 fs/ioctl.c:893
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  kn->active#5 --> &nvdimm_namespace_key --> &disk->open_mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&disk->open_mutex);
                               lock(&nvdimm_namespace_key);
                               lock(&disk->open_mutex);
  lock(kn->active#5);

 *** DEADLOCK ***

1 lock held by syz-executor336/6364:
 #0: ffff8880251264c8 (&disk->open_mutex){+.+.}-{3:3}, at: loop_reread_partitions+0x47/0x140 drivers/block/loop.c:533

stack backtrace:
CPU: 2 UID: 0 PID: 6364 Comm: syz-executor336 Not tainted 6.12.0-rc6-syzkaller-00192-gf1dce1f09380 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
 print_circular_bug+0x41c/0x610 kernel/locking/lockdep.c:2074
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2206
 check_prev_add kernel/locking/lockdep.c:3161 [inline]
 check_prevs_add kernel/locking/lockdep.c:3280 [inline]
 validate_chain kernel/locking/lockdep.c:3904 [inline]
 __lock_acquire+0x250b/0x3ce0 kernel/locking/lockdep.c:5202
 lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5825
 kernfs_drain+0x48f/0x590 fs/kernfs/dir.c:500
 __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486
 kernfs_remove_by_name_ns+0xb2/0x130 fs/kernfs/dir.c:1694
 sysfs_remove_file include/linux/sysfs.h:773 [inline]
 device_remove_file drivers/base/core.c:3054 [inline]
 device_remove_file drivers/base/core.c:3050 [inline]
 device_del+0x381/0x9f0 drivers/base/core.c:3859
 drop_partition+0x109/0x1c0 block/partitions/core.c:273
 bdev_disk_changed+0x24d/0x1520 block/partitions/core.c:666
 loop_reread_partitions+0x70/0x140 drivers/block/loop.c:534
 loop_configure+0xef6/0x11f0 drivers/block/loop.c:1119
 lo_ioctl+0xa6b/0x18b0 drivers/block/loop.c:1533
 blkdev_ioctl+0x276/0x6d0 block/ioctl.c:693
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:907 [inline]
 __se_sys_ioctl fs/ioctl.c:893 [inline]
 __x64_sys_ioctl+0x18f/0x220 fs/ioctl.c:893
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fced3ca73a9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 51 18 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fced3c5f218 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007fced3d2e328 RCX: 00007fced3ca73a9
RDX: 00000000200002c0 RSI: 0000000000004c0a RDI: 0000000000000003
RBP: 00007fced3d2e320 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fced3cfb074
R13: 307a797300010009 R14: 6f6f6c2f7665642f R15: 732e70756f726763
 </TASK>

Crashes (128):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/09 13:24 upstream f1dce1f09380 6b856513 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/04 18:16 upstream 59b723cd2adb 0754ea12 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] [mounted in repro] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 22:15 upstream da4373fbcf00 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 17:08 upstream da4373fbcf00 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 15:47 upstream da4373fbcf00 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 12:18 upstream f1dce1f09380 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 11:23 upstream f1dce1f09380 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 09:19 upstream f1dce1f09380 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 06:22 upstream f1dce1f09380 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 04:39 upstream f1dce1f09380 6b856513 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/09 00:23 upstream f1dce1f09380 d40c056e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 22:31 upstream f1dce1f09380 d40c056e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 16:40 upstream 906bd684e4b1 d40c056e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 11:59 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 10:56 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 09:21 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 07:37 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 06:10 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/08 04:21 upstream 906bd684e4b1 179b040e .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/08 02:28 upstream 906bd684e4b1 179b040e .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 23:28 upstream ff7afaeca1a1 867e44df .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 20:49 upstream ff7afaeca1a1 867e44df .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 19:12 upstream ff7afaeca1a1 867e44df .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 16:49 upstream ff7afaeca1a1 867e44df .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 13:18 upstream ff7afaeca1a1 867e44df .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/07 10:05 upstream 7758b206117d df3dc63b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 08:38 upstream 7758b206117d df3dc63b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 06:32 upstream 7758b206117d df3dc63b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/07 05:17 upstream 7758b206117d df3dc63b .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/07 03:38 upstream 7758b206117d df3dc63b .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 23:59 upstream 7758b206117d 7b852900 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 21:11 upstream 7758b206117d 7b852900 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 17:53 upstream 2e1b3cc9d7f7 7b852900 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 15:49 upstream 2e1b3cc9d7f7 7b852900 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 14:47 upstream 2e1b3cc9d7f7 7b852900 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 11:14 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 10:03 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 07:11 upstream 2e1b3cc9d7f7 3a465482 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/06 05:30 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 04:15 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/06 02:15 upstream 2e1b3cc9d7f7 3a465482 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 23:33 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 21:15 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 20:04 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 18:53 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 18:52 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 18:36 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/11/05 17:52 upstream 2e1b3cc9d7f7 da38b4c9 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
* Struck through repros no longer work on HEAD.