bisecting cause commit starting from 1216f02e46a46aad2a9c1c3bb29032f0b42eedac building syzkaller on 7e2b734bac96c22086fedd1b18135da06d5e4054 testing commit 1216f02e46a46aad2a9c1c3bb29032f0b42eedac with gcc (GCC) 10.2.1 20210217 kernel signature: 6fbc532fa2aa34371ab24780ffb2542e4741825bbd50505ed0cb6de2934680e2 all runs: crashed: INFO: task hung in __io_uring_cancel testing release v5.11 testing commit f40ddce88593482919761f74910f42f4b84c004b with gcc (GCC) 10.2.1 20210217 kernel signature: 293e6579f0d65cbda46b2d20e15f30555f6e4ea292502f9356c74a1affb84b2f all runs: crashed: INFO: task hung in io_uring_cancel_task_requests testing release v5.10 testing commit 2c85ebc57b3e1817b6ce1a6b703928e113a90442 with gcc (GCC) 10.2.1 20210217 kernel signature: aa5dbcb7b6350e128566df7b79649d445731318f591bef7c95206222f351bb67 all runs: OK # git bisect start f40ddce88593482919761f74910f42f4b84c004b 2c85ebc57b3e1817b6ce1a6b703928e113a90442 Bisecting: 7761 revisions left to test after this (roughly 13 steps) [538fcf57aaee6ad78a05f52b69a99baa22b33418] Merge branches 'acpi-scan', 'acpi-pnp' and 'acpi-sleep' testing commit 538fcf57aaee6ad78a05f52b69a99baa22b33418 with gcc (GCC) 10.2.1 20210217 kernel signature: ebc8f2aaf2b03ec1539c36d3c5ca281e7405d19cef6fd1fef07ea51a2c41a611 all runs: OK # git bisect good 538fcf57aaee6ad78a05f52b69a99baa22b33418 Bisecting: 3868 revisions left to test after this (roughly 12 steps) [d64c6f96ba86bd8b97ed8d6762a8c8cc1770d214] Merge tag 'net-5.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit d64c6f96ba86bd8b97ed8d6762a8c8cc1770d214 with gcc (GCC) 10.2.1 20210217 kernel signature: a5455bfe63684f38cc3ac5dd1d1d9841418affc50e7a1960709b86ab355d4dca all runs: OK # git bisect good d64c6f96ba86bd8b97ed8d6762a8c8cc1770d214 Bisecting: 1934 revisions left to test after this (roughly 11 steps) [70b6ff35d62050d1573876cc0e1e078acd3e6008] cfg80211/mac80211: fix kernel-doc for SAR APIs testing commit 70b6ff35d62050d1573876cc0e1e078acd3e6008 with gcc (GCC) 10.2.1 20210217 kernel signature: 4151fcbfc12f4102fc682c45053e233ac0c1e543e4c43d40af8fe59601868bd8 all runs: OK # git bisect good 70b6ff35d62050d1573876cc0e1e078acd3e6008 Bisecting: 965 revisions left to test after this (roughly 10 steps) [17b6c49da37f5d57d76bf352d32b0ac498e7c133] Merge tag 'x86_urgent_for_v5.11_rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit 17b6c49da37f5d57d76bf352d32b0ac498e7c133 with gcc (GCC) 10.2.1 20210217 kernel signature: 2ee9c3d5965d44a7e00f7309b0cc27f29960699a384a27d32fecef95cb76cf0a all runs: crashed: WARNING in io_uring_flush # git bisect bad 17b6c49da37f5d57d76bf352d32b0ac498e7c133 Bisecting: 464 revisions left to test after this (roughly 9 steps) [e8c13a6bc8ebbef7bd099ec1061633d1c9c94d5b] Merge tag 'net-5.11-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit e8c13a6bc8ebbef7bd099ec1061633d1c9c94d5b with gcc (GCC) 10.2.1 20210217 kernel signature: e04306b338728f6c1d884b2bb57624199ba963582cd0abe0a82aa95e942a31a3 all runs: crashed: WARNING in io_uring_flush # git bisect bad e8c13a6bc8ebbef7bd099ec1061633d1c9c94d5b Bisecting: 241 revisions left to test after this (roughly 8 steps) [e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e] Merge tag 'char-misc-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc testing commit e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e with gcc (GCC) 10.2.1 20210217 kernel signature: c25dd6a02cc59a10955dc6c3f286098190374ed4a1cda7cf2cafb3946332477e all runs: OK # git bisect good e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e Bisecting: 126 revisions left to test after this (roughly 7 steps) [ea49c88f4071e2bdd55e78987f251ea54aa11004] Merge tag 'mkp-scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi testing commit ea49c88f4071e2bdd55e78987f251ea54aa11004 with gcc (GCC) 10.2.1 20210217 kernel signature: a50e82906e66a64d8d07ac19e544a0c8b873e4e90716c6555a5aba0408385689 all runs: crashed: WARNING in io_uring_flush # git bisect bad ea49c88f4071e2bdd55e78987f251ea54aa11004 Bisecting: 58 revisions left to test after this (roughly 6 steps) [d430adfea8d2c5baa186cabb130235f72fecbd5b] Merge tag 'io_uring-5.11-2021-01-10' of git://git.kernel.dk/linux-block testing commit d430adfea8d2c5baa186cabb130235f72fecbd5b with gcc (GCC) 10.2.1 20210217 kernel signature: 09e7acd4902a24c70ee191e82b89f8a99cf01dbe8044c93e6bc3a293fab18dd8 all runs: crashed: WARNING in io_uring_flush # git bisect bad d430adfea8d2c5baa186cabb130235f72fecbd5b Bisecting: 28 revisions left to test after this (roughly 5 steps) [96ebc9c871d8a28fb22aa758dd9188a4732df482] usb: uas: Add PNY USB Portable SSD to unusual_uas testing commit 96ebc9c871d8a28fb22aa758dd9188a4732df482 with gcc (GCC) 10.2.1 20210217 kernel signature: e7eea6662a46e4a654c849a1cd9b672ad23e11ff75960ab5025e091c5a5a117b all runs: OK # git bisect good 96ebc9c871d8a28fb22aa758dd9188a4732df482 Bisecting: 13 revisions left to test after this (roughly 4 steps) [28318f53503090fcd8fd27c49445396ea2ace44b] Merge tag 'usb-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit 28318f53503090fcd8fd27c49445396ea2ace44b with gcc (GCC) 10.2.1 20210217 kernel signature: cff4578984e6c58fb47f0c8b3c1d3bfb439ad796e914c3e425dcc6a09ca8a167 all runs: OK # git bisect good 28318f53503090fcd8fd27c49445396ea2ace44b Bisecting: 6 revisions left to test after this (roughly 3 steps) [80c18e4ac20c9cde420cb3ffab48c936147cf07d] io_uring: trigger eventfd for IOPOLL testing commit 80c18e4ac20c9cde420cb3ffab48c936147cf07d with gcc (GCC) 10.2.1 20210217 kernel signature: 2ad1b387a01d34c88cc83da9a2059fc77ba0f31954e9ea25deaacd04636efedf all runs: OK # git bisect good 80c18e4ac20c9cde420cb3ffab48c936147cf07d Bisecting: 3 revisions left to test after this (roughly 2 steps) [55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550] io_uring: io_rw_reissue lockdep annotations testing commit 55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550 with gcc (GCC) 10.2.1 20210217 kernel signature: 772e5c027a0d164275d18388a78235486c2813405d763a8ad24669b7795d26b8 all runs: OK # git bisect good 55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550 Bisecting: 1 revision left to test after this (roughly 1 step) [6b5733eb638b7068ab7cb34e663b55a1d1892d85] io_uring: add warn_once for io_uring_flush() testing commit 6b5733eb638b7068ab7cb34e663b55a1d1892d85 with gcc (GCC) 10.2.1 20210217 kernel signature: a51ee75874061ec8ca3d7bd8885b0dd59a92cff7d6023b2f85ee062ef79db8b5 all runs: OK # git bisect good 6b5733eb638b7068ab7cb34e663b55a1d1892d85 Bisecting: 0 revisions left to test after this (roughly 0 steps) [d9d05217cb6990b9a56e13b56e7a1b71e2551f6c] io_uring: stop SQPOLL submit on creator's death testing commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c with gcc (GCC) 10.2.1 20210217 kernel signature: 9440bdca89de99d9b49005708345d1feede5f4401b20f1a6390adfc6f11426b8 all runs: crashed: WARNING in io_uring_flush # git bisect bad d9d05217cb6990b9a56e13b56e7a1b71e2551f6c d9d05217cb6990b9a56e13b56e7a1b71e2551f6c is the first bad commit commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c Author: Pavel Begunkov Date: Fri Jan 8 20:57:25 2021 +0000 io_uring: stop SQPOLL submit on creator's death When the creator of SQPOLL io_uring dies (i.e. sqo_task), we don't want its internals like ->files and ->mm to be poked by the SQPOLL task, it have never been nice and recently got racy. That can happen when the owner undergoes destruction and SQPOLL tasks tries to submit new requests in parallel, and so calls io_sq_thread_acquire*(). That patch halts SQPOLL submissions when sqo_task dies by introducing sqo_dead flag. Once set, the SQPOLL task must not do any submission, which is synchronised by uring_lock as well as the new flag. The tricky part is to make sure that disabling always happens, that means either the ring is discovered by creator's do_exit() -> cancel, or if the final close() happens before it's done by the creator. The last is guaranteed by the fact that for SQPOLL the creator task and only it holds exactly one file note, so either it pins up to do_exit() or removed by the creator on the final put in flush. (see comments in uring_flush() around file->f_count == 2). One more place that can trigger io_sq_thread_acquire_*() is __io_req_task_submit(). Shoot off requests on sqo_dead there, even though actually we don't need to. That's because cancellation of sqo_task should wait for the request before going any further. note 1: io_disable_sqo_submit() does io_ring_set_wakeup_flag() so the caller would enter the ring to get an error, but it still doesn't guarantee that the flag won't be cleared. note 2: if final __userspace__ close happens not from the creator task, the file note will pin the ring until the task dies. Fixed: b1b6b5a30dce8 ("kernel/io_uring: cancel io_uring before task works") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe fs/io_uring.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 9 deletions(-) culprit signature: 9440bdca89de99d9b49005708345d1feede5f4401b20f1a6390adfc6f11426b8 parent signature: a51ee75874061ec8ca3d7bd8885b0dd59a92cff7d6023b2f85ee062ef79db8b5 revisions tested: 17, total time: 4h49m40.525507419s (build: 1h52m44.971376842s, test: 2h54m20.220107092s) first bad commit: d9d05217cb6990b9a56e13b56e7a1b71e2551f6c io_uring: stop SQPOLL submit on creator's death recipients (to): ["asml.silence@gmail.com" "axboe@kernel.dk" "axboe@kernel.dk" "io-uring@vger.kernel.org"] recipients (cc): ["linux-fsdevel@vger.kernel.org" "linux-kernel@vger.kernel.org" "viro@zeniv.linux.org.uk"] crash: WARNING in io_uring_flush ------------[ cut here ]------------ WARNING: CPU: 1 PID: 3465 at fs/io_uring.c:9096 io_uring_flush+0x29d/0x310 fs/io_uring.c:9084 Modules linked in: CPU: 1 PID: 3465 Comm: syz-executor900 Not tainted 5.11.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:io_uring_flush+0x29d/0x310 fs/io_uring.c:9096 Code: ff ff 5b 31 c0 5d 41 5c 41 5d 41 5e 41 5f c3 49 c7 84 24 a0 00 00 00 00 00 00 00 e9 ee fe ff ff e8 f8 85 e2 ff e9 0c fe ff ff <0f> 0b e9 6d ff ff ff e8 f7 85 e2 ff e9 8a fd ff ff 4c 89 e7 e8 ea RSP: 0018:ffffc900051c7af8 EFLAGS: 00010297 RAX: 0000000000000001 RBX: ffff88801969f000 RCX: ffffc900051c79c0 RDX: 1ffff110281e3bba RSI: ffffffff888afd80 RDI: ffffffff88ddbc20 RBP: ffff88804be9a500 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000001 R12: ffff88801969f040 R13: ffff888028f3dc00 R14: ffff88801969f0d0 R15: ffff888050c35600 FS: 0000000000000000(0000) GS:ffff8880b9e00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fa991a7c000 CR3: 0000000010b23000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: filp_close+0x96/0x120 fs/open.c:1280 close_files fs/file.c:401 [inline] put_files_struct fs/file.c:416 [inline] put_files_struct+0x15c/0x2c0 fs/file.c:413 do_exit+0xa5b/0x2570 kernel/exit.c:820 do_group_exit+0xe7/0x290 kernel/exit.c:922 get_signal+0x333/0x1bd0 kernel/signal.c:2770 arch_do_signal_or_restart+0x2a8/0x1eb0 arch/x86/kernel/signal.c:811 handle_signal_work kernel/entry/common.c:147 [inline] exit_to_user_mode_loop kernel/entry/common.c:171 [inline] exit_to_user_mode_prepare+0x148/0x250 kernel/entry/common.c:201 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline] syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:302 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x445929 Code: Unable to access opcode bytes at RIP 0x4458ff. RSP: 002b:00007f68959b8308 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca RAX: fffffffffffffe00 RBX: 00000000004ca408 RCX: 0000000000445929 RDX: 0000000000000000 RSI: 0000000000000080 RDI: 00000000004ca408 RBP: 00000000004ca400 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000004ca40c R13: 0000000000000003 R14: 00007f68959b8400 R15: 0000000000022000