syzbot


possible deadlock in __kernfs_remove (2)

Status: upstream: reported C repro on 2024/11/08 10:18
Subsystems: kernfs
[Documentation on labels]
Reported-by: syzbot+aa419d82b68e6a7e96c5@syzkaller.appspotmail.com
First crash: 31d, last: 2h20m
Discussions (2)
Title Replies (including bot) Last reply
[syzbot] Monthly kernfs report (Nov 2024) 0 (1) 2024/11/15 11:25
[syzbot] [kernfs?] possible deadlock in __kernfs_remove (2) 0 (1) 2024/11/08 10:18
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in __kernfs_remove kernfs C 438 121d 164d 28/28 fixed on 2024/10/03 15:25

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.12.0-syzkaller-09073-g9f16d5e6f220 #0 Not tainted
------------------------------------------------------
syz-executor348/6018 is trying to acquire lock:
ffff88810bc1fc38 (kn->active#5){++++}-{0:0}, at: __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486

but task is already holding lock:
ffff8880259c84c8 (&disk->open_mutex){+.+.}-{4:4}, at: loop_reread_partitions+0x47/0x140 drivers/block/loop.c:533

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&disk->open_mutex){+.+.}-{4:4}:
       __mutex_lock_common kernel/locking/mutex.c:585 [inline]
       __mutex_lock+0x19b/0xa60 kernel/locking/mutex.c:735
       bdev_open+0x41a/0xe20 block/bdev.c:904
       bdev_file_open_by_dev block/bdev.c:1018 [inline]
       bdev_file_open_by_dev+0x17d/0x210 block/bdev.c:993
       disk_scan_partitions+0x1ed/0x320 block/genhd.c:367
       add_disk_fwnode+0xfdf/0x1300 block/genhd.c:514
       pmem_attach_disk+0x9a1/0x13e0 drivers/nvdimm/pmem.c:576
       nd_pmem_probe+0x1a9/0x1f0 drivers/nvdimm/pmem.c:649
       nvdimm_bus_probe+0x169/0x5d0 drivers/nvdimm/bus.c:94
       call_driver_probe drivers/base/dd.c:579 [inline]
       really_probe+0x23e/0xa90 drivers/base/dd.c:658
       __driver_probe_device+0x1de/0x440 drivers/base/dd.c:800
       driver_probe_device+0x4c/0x1b0 drivers/base/dd.c:830
       __driver_attach+0x283/0x580 drivers/base/dd.c:1216
       bus_for_each_dev+0x13c/0x1d0 drivers/base/bus.c:370
       bus_add_driver+0x2e9/0x690 drivers/base/bus.c:675
       driver_register+0x15c/0x4b0 drivers/base/driver.c:246
       __nd_driver_register+0x103/0x1a0 drivers/nvdimm/bus.c:622
       do_one_initcall+0x128/0x700 init/main.c:1266
       do_initcall_level init/main.c:1328 [inline]
       do_initcalls init/main.c:1344 [inline]
       do_basic_setup init/main.c:1363 [inline]
       kernel_init_freeable+0x5c7/0x900 init/main.c:1577
       kernel_init+0x1c/0x2b0 init/main.c:1466
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

-> #1 (&nvdimm_namespace_key){+.+.}-{4:4}:
       __mutex_lock_common kernel/locking/mutex.c:585 [inline]
       __mutex_lock+0x19b/0xa60 kernel/locking/mutex.c:735
       device_lock include/linux/device.h:1014 [inline]
       uevent_show+0x188/0x3b0 drivers/base/core.c:2736
       dev_attr_show+0x53/0xe0 drivers/base/core.c:2430
       sysfs_kf_seq_show+0x23e/0x410 fs/sysfs/file.c:59
       seq_read_iter+0x4f4/0x12b0 fs/seq_file.c:230
       kernfs_fop_read_iter+0x414/0x580 fs/kernfs/file.c:279
       new_sync_read fs/read_write.c:484 [inline]
       vfs_read+0x87f/0xbe0 fs/read_write.c:565
       ksys_read+0x12b/0x250 fs/read_write.c:708
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (kn->active#5){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3161 [inline]
       check_prevs_add kernel/locking/lockdep.c:3280 [inline]
       validate_chain kernel/locking/lockdep.c:3904 [inline]
       __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5226
       lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5849
       kernfs_drain+0x48f/0x590 fs/kernfs/dir.c:500
       __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486
       kernfs_remove_by_name_ns+0xb2/0x130 fs/kernfs/dir.c:1694
       sysfs_remove_file include/linux/sysfs.h:773 [inline]
       device_remove_file drivers/base/core.c:3054 [inline]
       device_remove_file drivers/base/core.c:3050 [inline]
       device_del+0x381/0x9f0 drivers/base/core.c:3859
       drop_partition+0x109/0x1c0 block/partitions/core.c:278
       bdev_disk_changed+0x24d/0x1520 block/partitions/core.c:674
       loop_reread_partitions+0x70/0x140 drivers/block/loop.c:534
       loop_configure+0xef5/0x11f0 drivers/block/loop.c:1118
       lo_ioctl+0xa6a/0x18b0 drivers/block/loop.c:1532
       blkdev_ioctl+0x276/0x6d0 block/ioctl.c:693
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:906 [inline]
       __se_sys_ioctl fs/ioctl.c:892 [inline]
       __x64_sys_ioctl+0x190/0x200 fs/ioctl.c:892
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  kn->active#5 --> &nvdimm_namespace_key --> &disk->open_mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&disk->open_mutex);
                               lock(&nvdimm_namespace_key);
                               lock(&disk->open_mutex);
  lock(kn->active#5);

 *** DEADLOCK ***

1 lock held by syz-executor348/6018:
 #0: ffff8880259c84c8 (&disk->open_mutex){+.+.}-{4:4}, at: loop_reread_partitions+0x47/0x140 drivers/block/loop.c:533

stack backtrace:
CPU: 1 UID: 0 PID: 6018 Comm: syz-executor348 Not tainted 6.12.0-syzkaller-09073-g9f16d5e6f220 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
 print_circular_bug+0x419/0x5d0 kernel/locking/lockdep.c:2074
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2206
 check_prev_add kernel/locking/lockdep.c:3161 [inline]
 check_prevs_add kernel/locking/lockdep.c:3280 [inline]
 validate_chain kernel/locking/lockdep.c:3904 [inline]
 __lock_acquire+0x249e/0x3c40 kernel/locking/lockdep.c:5226
 lock_acquire.part.0+0x11b/0x380 kernel/locking/lockdep.c:5849
 kernfs_drain+0x48f/0x590 fs/kernfs/dir.c:500
 __kernfs_remove+0x281/0x670 fs/kernfs/dir.c:1486
 kernfs_remove_by_name_ns+0xb2/0x130 fs/kernfs/dir.c:1694
 sysfs_remove_file include/linux/sysfs.h:773 [inline]
 device_remove_file drivers/base/core.c:3054 [inline]
 device_remove_file drivers/base/core.c:3050 [inline]
 device_del+0x381/0x9f0 drivers/base/core.c:3859
 drop_partition+0x109/0x1c0 block/partitions/core.c:278
 bdev_disk_changed+0x24d/0x1520 block/partitions/core.c:674
 loop_reread_partitions+0x70/0x140 drivers/block/loop.c:534
 loop_configure+0xef5/0x11f0 drivers/block/loop.c:1118
 lo_ioctl+0xa6a/0x18b0 drivers/block/loop.c:1532
 blkdev_ioctl+0x276/0x6d0 block/ioctl.c:693
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:906 [inline]
 __se_sys_ioctl fs/ioctl.c:892 [inline]
 __x64_sys_ioctl+0x190/0x200 fs/ioctl.c:892
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f3e76c4f9f9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 c1 17 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffdb34ea238 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f3e76c4f9f9
RDX: 00000000200002c0 RSI: 0000000000004c0a RDI: 0000000000000003
RBP: 000000000000a54d R08: 0000000000000006 R09: 0000000000000006
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffdb34ea24c
R13: 431bde82d7b634db R14: 0000000000000001 R15: 0000000000000001
 </TASK>
Dev loop2: unable to read RDB block 7
 loop2: AHDI p1 p2
loop2: partition table partially beyond EOD, truncated
loop2: p1 size 150995200 extends beyond EOD, truncated

Crashes (817):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/25 14:09 upstream 9f16d5e6f220 36dfdd05 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/09 13:24 upstream f1dce1f09380 6b856513 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/11/04 18:16 upstream 59b723cd2adb 0754ea12 .config console log report syz / log C [disk image (non-bootable)] [vmlinux] [kernel image] [mounted in repro] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 23:37 upstream 5076001689e4 1c533826 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 21:36 upstream 5076001689e4 1c533826 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 16:51 upstream feffde684ac2 1c533826 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 15:48 upstream feffde684ac2 1c533826 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 11:14 upstream feffde684ac2 29f61fce .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 09:53 upstream feffde684ac2 29f61fce .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 08:03 upstream feffde684ac2 29f61fce .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 04:39 upstream feffde684ac2 29f61fce .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 03:05 upstream feffde684ac2 29f61fce .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/05 00:00 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 22:30 upstream feffde684ac2 b50eb251 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/12/04 21:03 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 19:57 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 18:34 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 17:24 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 15:04 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 14:01 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 13:47 upstream feffde684ac2 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 09:36 upstream ceb8bf2ceaa7 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 07:53 upstream ceb8bf2ceaa7 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 06:49 upstream ceb8bf2ceaa7 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 03:21 upstream ceb8bf2ceaa7 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/04 00:24 upstream ceb8bf2ceaa7 b50eb251 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 20:42 upstream ceb8bf2ceaa7 330db277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 16:49 upstream cdd30ebb1b9f 330db277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 14:56 upstream cdd30ebb1b9f 330db277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 12:30 upstream cdd30ebb1b9f 330db277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 11:44 upstream cdd30ebb1b9f 330db277 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 08:18 upstream cdd30ebb1b9f bb326ffb .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 06:50 upstream cdd30ebb1b9f bb326ffb .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 05:42 upstream cdd30ebb1b9f bb326ffb .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in __kernfs_remove
2024/12/03 03:50 upstream cdd30ebb1b9f bb326ffb .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 01:33 upstream cdd30ebb1b9f bb326ffb .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 23:56 upstream cdd30ebb1b9f bb326ffb .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 17:38 upstream e70140ba0d2b b499ea68 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 16:23 upstream e70140ba0d2b b499ea68 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 14:14 upstream e70140ba0d2b b499ea68 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 13:01 upstream e70140ba0d2b b499ea68 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 08:32 upstream f788b5ef1ca9 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 07:30 upstream f788b5ef1ca9 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 05:49 upstream f788b5ef1ca9 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 04:00 upstream f788b5ef1ca9 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/02 01:54 upstream f788b5ef1ca9 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/01 21:06 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/01 19:13 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/01 18:10 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/01 15:14 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/01 14:07 upstream bcc8eda6d349 68914665 .config console log report [disk image (non-bootable)] [vmlinux] [kernel image] ci-snapshot-upstream-root possible deadlock in __kernfs_remove
2024/12/03 09:33 upstream cdd30ebb1b9f bb326ffb .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in __kernfs_remove
2024/12/02 21:05 upstream e70140ba0d2b b499ea68 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in __kernfs_remove
* Struck through repros no longer work on HEAD.