bisecting fixing commit since d573e8a79f70404ba08623d1de7ea617d55092ac building syzkaller on eb6b9855e00c6a323f34038df1435b9f75892fe5 testing commit d573e8a79f70404ba08623d1de7ea617d55092ac with gcc (GCC) 8.1.0 kernel signature: 42b28d71666255537316a4fa8c7b30512f26b353 all runs: crashed: possible deadlock in free_ioctx_users testing current HEAD fb683b5e3f53a73e761952735736180939a313df testing commit fb683b5e3f53a73e761952735736180939a313df with gcc (GCC) 8.1.0 kernel signature: e40b1538a52e362093826d63a94f50a12d1b65ef all runs: OK # git bisect start fb683b5e3f53a73e761952735736180939a313df d573e8a79f70404ba08623d1de7ea617d55092ac Bisecting: 1167 revisions left to test after this (roughly 10 steps) [cd554b025c09ab67c278fb8599fd268185a07628] rtl8187: Fix warning generated when strncpy() destination length matches the sixe argument testing commit cd554b025c09ab67c278fb8599fd268185a07628 with gcc (GCC) 8.1.0 kernel signature: f920976a12a4dfc005dbe677819112663b8d02a2 all runs: OK # git bisect bad cd554b025c09ab67c278fb8599fd268185a07628 Bisecting: 583 revisions left to test after this (roughly 9 steps) [a23cd06c2cd2aab5728c1755616d2a1ffb95d6ac] ARM: OMAP2+: Fix missing reset done flag for am3 and am43 testing commit a23cd06c2cd2aab5728c1755616d2a1ffb95d6ac with gcc (GCC) 8.1.0 kernel signature: c8990248b0d05e887c5228de1785d4b6668c21da all runs: OK # git bisect bad a23cd06c2cd2aab5728c1755616d2a1ffb95d6ac Bisecting: 291 revisions left to test after this (roughly 8 steps) [782a77f2eb39207589ef9175a2ceadd0cca12112] drm/amd/display: reprogram VM config when system resume testing commit 782a77f2eb39207589ef9175a2ceadd0cca12112 with gcc (GCC) 8.1.0 kernel signature: b55bdc9efc8c01741412d3ca1e194818adb0400e all runs: OK # git bisect bad 782a77f2eb39207589ef9175a2ceadd0cca12112 Bisecting: 145 revisions left to test after this (roughly 7 steps) [dfaf60580191207627a85739850799bbb13280f4] ARM: dts: imx7-colibri: disable HS400 testing commit dfaf60580191207627a85739850799bbb13280f4 with gcc (GCC) 8.1.0 kernel signature: e932c112f4ec645f38acf019a09f2da69c2986fd all runs: crashed: possible deadlock in free_ioctx_users # git bisect good dfaf60580191207627a85739850799bbb13280f4 Bisecting: 72 revisions left to test after this (roughly 6 steps) [ad6819cd68bef7f37e0a6de8ab65512f59227c12] IB/hfi1: Define variables as unsigned long to fix KASAN warning testing commit ad6819cd68bef7f37e0a6de8ab65512f59227c12 with gcc (GCC) 8.1.0 kernel signature: f7983affe25b0e7fefe2ddf5d3ab3874e5fc0420 all runs: crashed: possible deadlock in free_ioctx_users # git bisect good ad6819cd68bef7f37e0a6de8ab65512f59227c12 Bisecting: 36 revisions left to test after this (roughly 5 steps) [b08344be3546ff71b395a1adac3c075f887edd9f] Btrfs: fix use-after-free when using the tree modification log testing commit b08344be3546ff71b395a1adac3c075f887edd9f with gcc (GCC) 8.1.0 kernel signature: 656e22f2b779200b31880b4cb64b11f028d46d58 all runs: crashed: possible deadlock in free_ioctx_users # git bisect good b08344be3546ff71b395a1adac3c075f887edd9f Bisecting: 18 revisions left to test after this (roughly 4 steps) [5bead06b3443c784637d454c5c64a3bd05752cf4] fuse: fix deadlock with aio poll and fuse_iqueue::waitq.lock testing commit 5bead06b3443c784637d454c5c64a3bd05752cf4 with gcc (GCC) 8.1.0 kernel signature: 427603f59173b7c6e142fc7cd8d180e3f03c3e8b all runs: OK # git bisect bad 5bead06b3443c784637d454c5c64a3bd05752cf4 Bisecting: 8 revisions left to test after this (roughly 3 steps) [caa6926d94f12768706fe580fca211fceb3cfddf] /dev/mem: Bail out upon SIGKILL. testing commit caa6926d94f12768706fe580fca211fceb3cfddf with gcc (GCC) 8.1.0 kernel signature: 6f04f8a888f6c20b8e4c1da6d7b6a99d23be2846 all runs: crashed: possible deadlock in free_ioctx_users # git bisect good caa6926d94f12768706fe580fca211fceb3cfddf Bisecting: 4 revisions left to test after this (roughly 2 steps) [fec38267a2bf563bb1330c7845ffae34a643ad99] hwrng: core - don't wait on add_early_randomness() testing commit fec38267a2bf563bb1330c7845ffae34a643ad99 with gcc (GCC) 8.1.0 kernel signature: 59d2eab0b482f203047953c8dad8520e53095d3c all runs: crashed: possible deadlock in free_ioctx_users # git bisect good fec38267a2bf563bb1330c7845ffae34a643ad99 Bisecting: 2 revisions left to test after this (roughly 1 step) [a3a150895b6f09896f3c3121b2e6cc927f21aba2] CIFS: fix max ea value size testing commit a3a150895b6f09896f3c3121b2e6cc927f21aba2 with gcc (GCC) 8.1.0 kernel signature: 57bc2f7f29851a6c0b7d139418580304a2056c47 all runs: crashed: possible deadlock in free_ioctx_users # git bisect good a3a150895b6f09896f3c3121b2e6cc927f21aba2 Bisecting: 0 revisions left to test after this (roughly 1 step) [bbe3e2056d27c356c8778a2329147a328debc422] md/raid0: avoid RAID0 data corruption due to layout confusion. testing commit bbe3e2056d27c356c8778a2329147a328debc422 with gcc (GCC) 8.1.0 kernel signature: f7bfaef2b4022b1f0e1d482231a5a9eeceecabb0 all runs: crashed: possible deadlock in free_ioctx_users # git bisect good bbe3e2056d27c356c8778a2329147a328debc422 5bead06b3443c784637d454c5c64a3bd05752cf4 is the first bad commit commit 5bead06b3443c784637d454c5c64a3bd05752cf4 Author: Eric Biggers Date: Sun Sep 8 20:15:18 2019 -0700 fuse: fix deadlock with aio poll and fuse_iqueue::waitq.lock [ Upstream commit 76e43c8ccaa35c30d5df853013561145a0f750a5 ] When IOCB_CMD_POLL is used on the FUSE device, aio_poll() disables IRQs and takes kioctx::ctx_lock, then fuse_iqueue::waitq.lock. This may have to wait for fuse_iqueue::waitq.lock to be released by one of many places that take it with IRQs enabled. Since the IRQ handler may take kioctx::ctx_lock, lockdep reports that a deadlock is possible. Fix it by protecting the state of struct fuse_iqueue with a separate spinlock, and only accessing fuse_iqueue::waitq using the versions of the waitqueue functions which do IRQ-safe locking internally. Reproducer: #include #include #include #include #include #include #include int main() { char opts[128]; int fd = open("/dev/fuse", O_RDWR); aio_context_t ctx = 0; struct iocb cb = { .aio_lio_opcode = IOCB_CMD_POLL, .aio_fildes = fd }; struct iocb *cbp = &cb; sprintf(opts, "fd=%d,rootmode=040000,user_id=0,group_id=0", fd); mkdir("mnt", 0700); mount("foo", "mnt", "fuse", 0, opts); syscall(__NR_io_setup, 1, &ctx); syscall(__NR_io_submit, ctx, 1, &cbp); } Beginning of lockdep output: ===================================================== WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected 5.3.0-rc5 #9 Not tainted ----------------------------------------------------- syz_fuse/135 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: 000000003590ceda (&fiq->waitq){+.+.}, at: spin_lock include/linux/spinlock.h:338 [inline] 000000003590ceda (&fiq->waitq){+.+.}, at: aio_poll fs/aio.c:1751 [inline] 000000003590ceda (&fiq->waitq){+.+.}, at: __io_submit_one.constprop.0+0x203/0x5b0 fs/aio.c:1825 and this task is already holding: 0000000075037284 (&(&ctx->ctx_lock)->rlock){..-.}, at: spin_lock_irq include/linux/spinlock.h:363 [inline] 0000000075037284 (&(&ctx->ctx_lock)->rlock){..-.}, at: aio_poll fs/aio.c:1749 [inline] 0000000075037284 (&(&ctx->ctx_lock)->rlock){..-.}, at: __io_submit_one.constprop.0+0x1f4/0x5b0 fs/aio.c:1825 which would create a new lock dependency: (&(&ctx->ctx_lock)->rlock){..-.} -> (&fiq->waitq){+.+.} but this new dependency connects a SOFTIRQ-irq-safe lock: (&(&ctx->ctx_lock)->rlock){..-.} [...] Reported-by: syzbot+af05535bb79520f95431@syzkaller.appspotmail.com Reported-by: syzbot+d86c4426a01f60feddc7@syzkaller.appspotmail.com Fixes: bfe4037e722e ("aio: implement IOCB_CMD_POLL") Cc: # v4.19+ Cc: Christoph Hellwig Signed-off-by: Eric Biggers Signed-off-by: Miklos Szeredi Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman fs/fuse/dev.c | 91 +++++++++++++++++++++++++++++--------------------------- fs/fuse/fuse_i.h | 3 ++ fs/fuse/inode.c | 1 + 3 files changed, 51 insertions(+), 44 deletions(-) culprit signature: 427603f59173b7c6e142fc7cd8d180e3f03c3e8b parent signature: f7bfaef2b4022b1f0e1d482231a5a9eeceecabb0 revisions tested: 13, total time: 3h8m15.889558654s (build: 1h49m13.325363782s, test: 1h17m32.078236293s) first good commit: 5bead06b3443c784637d454c5c64a3bd05752cf4 fuse: fix deadlock with aio poll and fuse_iqueue::waitq.lock cc: ["ebiggers@google.com" "gregkh@linuxfoundation.org" "mszeredi@redhat.com" "sashal@kernel.org"]