bisecting cause commit starting from a67ba3cf9551f8c92d5ec9d7eae1aadbb9127b57 building syzkaller on 6bdac76629d4d80501671c0c312d6b81411481e7 testing commit a67ba3cf9551f8c92d5ec9d7eae1aadbb9127b57 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e8c7d256e6f7ad22b6e8dda241133a4d365a8a8ae8369b7034d98ac54a08702b all runs: crashed: possible deadlock in deactivate_super testing release v5.17 testing commit f443e374ae131c168a065ea1748feac6b2e76613 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1ba98d2ff9a20481884f82b08041137e12c8dd208147ccd9f44989128ff1ac27 run #0: basic kernel testing failed: BUG: program execution failed: executor NUM: exit status NUM run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect start a67ba3cf9551f8c92d5ec9d7eae1aadbb9127b57 f443e374ae131c168a065ea1748feac6b2e76613 Bisecting: 7151 revisions left to test after this (roughly 13 steps) [169e77764adc041b1dacba84ea90516a895d43b2] Merge tag 'net-next-5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next testing commit 169e77764adc041b1dacba84ea90516a895d43b2 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1549f98de2c6147db186b850c0c81c47f4bb7e6fc51064466990c961e6e024de run #0: OK run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: boot failed: BUG: sleeping function called from invalid context in blk_release_queue # git bisect good 169e77764adc041b1dacba84ea90516a895d43b2 Bisecting: 3566 revisions left to test after this (roughly 12 steps) [710f5d627a98e86f821aceb840b8f2f1fcc6cf75] Merge tag 'usb-5.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit 710f5d627a98e86f821aceb840b8f2f1fcc6cf75 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: b46807eba15145ab211db12925e54ec87be2eec5ad1e3b75e9c9843efdb0e3a0 all runs: OK # git bisect good 710f5d627a98e86f821aceb840b8f2f1fcc6cf75 Bisecting: 1796 revisions left to test after this (roughly 11 steps) [ca8f6ed43e3582dc3d7a9183e746b3a11775d3da] PCI: Remove the deprecated "pci-dma-compat.h" API testing commit ca8f6ed43e3582dc3d7a9183e746b3a11775d3da compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: fdfcaf850b640793c9c8408cea1e5d684365a027688e38510e1ca027fceb1cf8 all runs: OK # git bisect good ca8f6ed43e3582dc3d7a9183e746b3a11775d3da Bisecting: 905 revisions left to test after this (roughly 10 steps) [0f3a7ef78620b2444e27d4f430fe6e8f9f6e084e] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git testing commit 0f3a7ef78620b2444e27d4f430fe6e8f9f6e084e compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: c2e2ca4cb50ced9cf1abe7689cf2106c4232fc7daa9ea7350391b10c50501174 all runs: OK # git bisect good 0f3a7ef78620b2444e27d4f430fe6e8f9f6e084e Bisecting: 452 revisions left to test after this (roughly 9 steps) [1f24668d36ed7bfbd653901a99a1c229582d4d3c] Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git testing commit 1f24668d36ed7bfbd653901a99a1c229582d4d3c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 4a9936bfd2f73f3c7c482630e9029bb28264642b4c6f83f45f14f1984e140181 all runs: crashed: possible deadlock in deactivate_super # git bisect bad 1f24668d36ed7bfbd653901a99a1c229582d4d3c Bisecting: 209 revisions left to test after this (roughly 8 steps) [c189229b0b436d5eb1543ef412e9ca4b7cd1e2e3] Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input.git testing commit c189229b0b436d5eb1543ef412e9ca4b7cd1e2e3 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 4cb7d75964aa3010ff9d32988a0d62d048573afe80abf253781635fa2ca2c67c all runs: OK # git bisect good c189229b0b436d5eb1543ef412e9ca4b7cd1e2e3 Bisecting: 106 revisions left to test after this (roughly 7 steps) [3deedc69cb922583c84722288ba1267749f19aa3] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi.git testing commit 3deedc69cb922583c84722288ba1267749f19aa3 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 219c48d9a2757a05e1978e85286446380a5e186f3328cacea150f129a9ecfc98 all runs: crashed: possible deadlock in deactivate_super # git bisect bad 3deedc69cb922583c84722288ba1267749f19aa3 Bisecting: 44 revisions left to test after this (roughly 6 steps) [2aeb4d380497a9ec6236e0a00dcda6a3935dc5d7] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git testing commit 2aeb4d380497a9ec6236e0a00dcda6a3935dc5d7 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 35abb2971e2cff4ca4aeae5586e5175b9c988d642ff7631bdb1989f50f5bd69b all runs: crashed: possible deadlock in deactivate_super # git bisect bad 2aeb4d380497a9ec6236e0a00dcda6a3935dc5d7 Bisecting: 30 revisions left to test after this (roughly 5 steps) [3b255fe79c9eaa84a48e35e8cd6406788ba90d9c] Merge branch 'for-5.18/drivers' into for-next testing commit 3b255fe79c9eaa84a48e35e8cd6406788ba90d9c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 77b79ca91d90b430c8393e7ab3d2d977311e238ed65933fa14b0ad4e116bb03b all runs: OK # git bisect good 3b255fe79c9eaa84a48e35e8cd6406788ba90d9c Bisecting: 15 revisions left to test after this (roughly 4 steps) [14e8dafc7cb8c6d8e69aae4df24b94a950a67247] dm: use state machine for bio polling completion testing commit 14e8dafc7cb8c6d8e69aae4df24b94a950a67247 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e27e5103884b59d77efb218dea72c750313322ba53bd5103b315feeaaec7716b run #0: basic kernel testing failed: BUG: program execution failed: executor NUM: exit status NUM run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good 14e8dafc7cb8c6d8e69aae4df24b94a950a67247 Bisecting: 7 revisions left to test after this (roughly 3 steps) [a4a6f3c8f61c3cfbda4998ad94596059ad7e4332] nvme-multipath: fix hang when disk goes live over reconnect testing commit a4a6f3c8f61c3cfbda4998ad94596059ad7e4332 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 65a38a0fc1770a152950b53fa0912db48ef849b3953f71b8eacf966d6a7ebc2c run #0: basic kernel testing failed: BUG: program execution failed: executor NUM: failed to write control pipe: write |NUM: broken pipe run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good a4a6f3c8f61c3cfbda4998ad94596059ad7e4332 Bisecting: 3 revisions left to test after this (roughly 2 steps) [41a2c7829baa330e56ffa11fc5b20cc351ac8974] dm: conditionally enable branching for less used features testing commit 41a2c7829baa330e56ffa11fc5b20cc351ac8974 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 8dd73390f5b7c16edf89e96e2d661e54c104bcbe0c96766f578673dd80be0c57 all runs: OK # git bisect good 41a2c7829baa330e56ffa11fc5b20cc351ac8974 Bisecting: 1 revision left to test after this (roughly 1 step) [c02b67509585dcd8c5561759dc78165230519d9c] Merge branch 'for-5.18/drivers' into for-next testing commit c02b67509585dcd8c5561759dc78165230519d9c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a2ead86b87fc5b6d8d114aa457527a825a2b0f54e3bfcb2ec2459f5a37ae554e all runs: OK # git bisect good c02b67509585dcd8c5561759dc78165230519d9c Bisecting: 0 revisions left to test after this (roughly 0 steps) [6a711b79aeed5e82c7b60450a8036702972cc48c] Merge branch 'for-next' of git://git.kernel.dk/linux-block.git testing commit 6a711b79aeed5e82c7b60450a8036702972cc48c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 61b3f162a30cb6177df51baaa8979910a078c254fe775f06ea92da4271233425 all runs: crashed: possible deadlock in deactivate_super # git bisect bad 6a711b79aeed5e82c7b60450a8036702972cc48c 6a711b79aeed5e82c7b60450a8036702972cc48c is the first bad commit commit 6a711b79aeed5e82c7b60450a8036702972cc48c Merge: c189229b0b43 c02b67509585 Author: Stephen Rothwell Date: Wed Mar 30 09:43:45 2022 +1100 Merge branch 'for-next' of git://git.kernel.dk/linux-block.git block/blk-cgroup.c | 32 +++++++--- block/blk-ioc.c | 3 +- block/genhd.c | 2 +- drivers/block/n64cart.c | 2 +- drivers/block/xen-blkback/blkback.c | 2 +- drivers/block/xen-blkfront.c | 2 +- drivers/nvme/host/core.c | 38 ++++++++---- drivers/nvme/host/multipath.c | 32 ++++++++-- drivers/nvme/host/nvme.h | 23 +++++++ drivers/nvme/host/pci.c | 7 ++- drivers/nvme/target/admin-cmd.c | 2 +- drivers/nvme/target/configfs.c | 2 +- drivers/nvme/target/core.c | 26 +++++--- drivers/nvme/target/fc.c | 8 +-- drivers/nvme/target/fcloop.c | 16 ++--- drivers/nvme/target/io-cmd-file.c | 6 +- drivers/nvme/target/loop.c | 4 +- drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/passthru.c | 2 +- drivers/nvme/target/rdma.c | 12 ++-- drivers/nvme/target/tcp.c | 10 +-- fs/io_uring.c | 120 ++++++++++++++++++++++++++++++------ include/linux/blk-cgroup.h | 5 +- include/linux/nvme.h | 1 + include/linux/sbitmap.h | 2 +- include/uapi/linux/io_uring.h | 2 - lib/sbitmap.c | 2 +- 27 files changed, 267 insertions(+), 97 deletions(-) revisions tested: 16, total time: 2h58m26.453698039s (build: 1h40m12.363869046s, test: 1h16m33.334020921s) first bad commit: 6a711b79aeed5e82c7b60450a8036702972cc48c Merge branch 'for-next' of git://git.kernel.dk/linux-block.git recipients (to): ["sfr@canb.auug.org.au"] recipients (cc): [] crash: possible deadlock in deactivate_super ====================================================== WARNING: possible circular locking dependency detected 5.17.0-syzkaller #0 Not tainted ------------------------------------------------------ syz-executor234/4062 is trying to acquire lock: ffff88800fc64d38 ((wq_completion)events){+.+.}-{0:0}, at: flush_workqueue+0xca/0x1230 kernel/workqueue.c:2806 but task is already holding lock: ffff888022e3a0e0 (&type->s_umount_key#32){++++}-{3:3}, at: deactivate_super+0x5f/0x80 fs/super.c:362 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #3 (&type->s_umount_key#32){++++}-{3:3}: down_read+0x98/0x440 kernel/locking/rwsem.c:1461 iterate_supers+0xc2/0x200 fs/super.c:692 drop_caches_sysctl_handler+0x55/0x80 fs/drop_caches.c:62 proc_sys_call_handler+0x3c9/0x580 fs/proc/proc_sysctl.c:604 call_write_iter include/linux/fs.h:2080 [inline] do_iter_readv_writev+0x2b4/0x5b0 fs/read_write.c:726 do_iter_write+0x124/0x620 fs/read_write.c:852 iter_file_splice_write+0x598/0xaf0 fs/splice.c:689 do_splice_from fs/splice.c:767 [inline] direct_splice_actor+0xfb/0x1c0 fs/splice.c:936 splice_direct_to_actor+0x2dd/0x7c0 fs/splice.c:891 do_splice_direct+0x148/0x250 fs/splice.c:979 do_sendfile+0x90c/0x1100 fs/read_write.c:1246 __do_sys_sendfile64 fs/read_write.c:1305 [inline] __se_sys_sendfile64 fs/read_write.c:1297 [inline] __x64_sys_sendfile64+0x11a/0x1d0 fs/read_write.c:1297 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0x80 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x44/0xae -> #2 (sb_writers#3){.+.+}-{0:0}: percpu_down_read include/linux/percpu-rwsem.h:51 [inline] __sb_start_write include/linux/fs.h:1728 [inline] sb_start_write include/linux/fs.h:1798 [inline] file_start_write include/linux/fs.h:2815 [inline] kernel_write fs/read_write.c:564 [inline] kernel_write+0x231/0x420 fs/read_write.c:555 p9_fd_write net/9p/trans_fd.c:428 [inline] p9_write_work+0x1ef/0xaf0 net/9p/trans_fd.c:479 process_one_work+0x865/0x13d0 kernel/workqueue.c:2289 worker_thread+0x598/0xec0 kernel/workqueue.c:2436 kthread+0x299/0x340 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298 -> #1 ((work_completion)(&m->wq)){+.+.}-{0:0}: process_one_work+0x800/0x13d0 kernel/workqueue.c:2265 worker_thread+0x598/0xec0 kernel/workqueue.c:2436 kthread+0x299/0x340 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298 -> #0 ((wq_completion)events){+.+.}-{0:0}: check_prev_add kernel/locking/lockdep.c:3065 [inline] check_prevs_add kernel/locking/lockdep.c:3188 [inline] validate_chain kernel/locking/lockdep.c:3803 [inline] __lock_acquire+0x2a44/0x5660 kernel/locking/lockdep.c:5029 lock_acquire kernel/locking/lockdep.c:5641 [inline] lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5606 flush_workqueue+0xed/0x1230 kernel/workqueue.c:2809 flush_scheduled_work include/linux/workqueue.h:592 [inline] ext4_put_super+0x91/0xec0 fs/ext4/super.c:1202 generic_shutdown_super+0x12e/0x3a0 fs/super.c:462 kill_block_super+0x90/0xd0 fs/super.c:1394 deactivate_locked_super+0x7b/0x130 fs/super.c:332 cleanup_mnt+0x324/0x4d0 fs/namespace.c:1186 task_work_run+0xc0/0x160 kernel/task_work.c:164 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline] exit_to_user_mode_loop kernel/entry/common.c:183 [inline] exit_to_user_mode_prepare+0x23c/0x250 kernel/entry/common.c:215 __syscall_exit_to_user_mode_work kernel/entry/common.c:297 [inline] syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:308 do_syscall_64+0x42/0x80 arch/x86/entry/common.c:86 entry_SYSCALL_64_after_hwframe+0x44/0xae other info that might help us debug this: Chain exists of: (wq_completion)events --> sb_writers#3 --> &type->s_umount_key#32 Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&type->s_umount_key#32); lock(sb_writers#3); lock(&type->s_umount_key#32); lock((wq_completion)events); *** DEADLOCK *** 1 lock held by syz-executor234/4062: #0: ffff888022e3a0e0 (&type->s_umount_key#32){++++}-{3:3}, at: deactivate_super+0x5f/0x80 fs/super.c:362 stack backtrace: CPU: 0 PID: 4062 Comm: syz-executor234 Not tainted 5.17.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x57/0x7d lib/dump_stack.c:106 check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2145 check_prev_add kernel/locking/lockdep.c:3065 [inline] check_prevs_add kernel/locking/lockdep.c:3188 [inline] validate_chain kernel/locking/lockdep.c:3803 [inline] __lock_acquire+0x2a44/0x5660 kernel/locking/lockdep.c:5029 lock_acquire kernel/locking/lockdep.c:5641 [inline] lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5606 flush_workqueue+0xed/0x1230 kernel/workqueue.c:2809 flush_scheduled_work include/linux/workqueue.h:592 [inline] ext4_put_super+0x91/0xec0 fs/ext4/super.c:1202 generic_shutdown_super+0x12e/0x3a0 fs/super.c:462 kill_block_super+0x90/0xd0 fs/super.c:1394 deactivate_locked_super+0x7b/0x130 fs/super.c:332 cleanup_mnt+0x324/0x4d0 fs/namespace.c:1186 task_work_run+0xc0/0x160 kernel/task_work.c:164 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline] exit_to_user_mode_loop kernel/entry/common.c:183 [inline] exit_to_user_mode_prepare+0x23c/0x250 kernel/entry/common.c:215 __syscall_exit_to_user_mode_work kernel/entry/common.c:297 [inline] syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:308 do_syscall_64+0x42/0x80 arch/x86/entry/common.c:86 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7f14ab202677 Code: 07 00 48 83 c4 08 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 a6 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007ffe9b654638 EFLAGS: 00000202 ORIG_RAX: 00000000000000a6 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f14ab202677 RDX: 00007ffe9b6546f9 RSI: 000000000000000a RDI: 00007ffe9b6546f0 RBP: 00007ffe9b6546f0 R08: 00000000ffffffff R09: 00007ffe9b6544d0 R10: 0000555556925653 R11: 0000000000000202 R12: 00007ffe9b655760 R13: 00005555569255f0 R14: 00007ffe9b654660 R15: 0000000000000001