bisecting cause commit starting from 7c53f6b671f4aba70ff15e1b05148b10d58c2837 building syzkaller on 2c1f2513486f21d26b1942ce77ffc782677fbf4e testing commit 7c53f6b671f4aba70ff15e1b05148b10d58c2837 with gcc (GCC) 8.1.0 kernel signature: 179c296627e9f85c27073a52f90ea2da780b4ab2f8c8a163c1f20b7b2301adf5 all runs: crashed: WARNING in io_ring_ctx_wait_and_kill testing release v5.10 testing commit 2c85ebc57b3e1817b6ce1a6b703928e113a90442 with gcc (GCC) 8.1.0 kernel signature: 395a614079c9030036488f72c030c4708fae86b9aba956ca15d15003efd3a593 all runs: OK # git bisect start 7c53f6b671f4aba70ff15e1b05148b10d58c2837 2c85ebc57b3e1817b6ce1a6b703928e113a90442 Bisecting: 6936 revisions left to test after this (roughly 13 steps) [3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9] Merge tag 'staging-5.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging testing commit 3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9 with gcc (GCC) 8.1.0 kernel signature: 6c9809d6712e897dccefdc6b430d7d43d2f2bd116c2c5d44d2bc23ac679aef61 all runs: OK # git bisect good 3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9 Bisecting: 3336 revisions left to test after this (roughly 12 steps) [9805529ec544ea7a82d891d5239a8ebd3dbb2a3e] Merge tag 'arm-soc-dt-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc testing commit 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e with gcc (GCC) 8.1.0 kernel signature: ebc2bf19b9b97d2d465977b4a1a0d8b35c66496a5a60baa86cb7f96369491ecb all runs: OK # git bisect good 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e Bisecting: 1687 revisions left to test after this (roughly 11 steps) [f4a2f7866faaf89ea1595b136e01fcb336b46aab] Merge tag 'rtc-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/abelloni/linux testing commit f4a2f7866faaf89ea1595b136e01fcb336b46aab with gcc (GCC) 8.1.0 kernel signature: 3e99057020720f1d59a48cf9e0120be571492600e5e7772ccddb525281db218b all runs: OK # git bisect good f4a2f7866faaf89ea1595b136e01fcb336b46aab Bisecting: 847 revisions left to test after this (roughly 10 steps) [ef2c8b81b88868f042579b9dd021cc9edbc2d0c6] Merge tag 'drm-next-2020-12-24' of git://anongit.freedesktop.org/drm/drm testing commit ef2c8b81b88868f042579b9dd021cc9edbc2d0c6 with gcc (GCC) 8.1.0 kernel signature: dad45683ffba35a66ea076495223fbcd1cf7908f5e1033f0c8c61273fd244989 all runs: OK # git bisect good ef2c8b81b88868f042579b9dd021cc9edbc2d0c6 Bisecting: 423 revisions left to test after this (roughly 9 steps) [1d011777cdbe7ae38a854a0cbeb6bdfbf724cce0] Merge tag 'sound-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound testing commit 1d011777cdbe7ae38a854a0cbeb6bdfbf724cce0 with gcc (GCC) 8.1.0 kernel signature: 00a9f6ba64942618c55ce35d1866807dfc905c510c57476ff966e5eb1200ba1c all runs: OK # git bisect good 1d011777cdbe7ae38a854a0cbeb6bdfbf724cce0 Bisecting: 226 revisions left to test after this (roughly 8 steps) [2a190b22aa1149cda804527aa603db45f75439c3] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm testing commit 2a190b22aa1149cda804527aa603db45f75439c3 with gcc (GCC) 8.1.0 kernel signature: 4d63dc794e25248e6cad62ad33bf78353b52e114181617630012df0e2f0758a6 all runs: OK # git bisect good 2a190b22aa1149cda804527aa603db45f75439c3 Bisecting: 124 revisions left to test after this (roughly 7 steps) [0653161f0faca68b77b3f36fb4b4b9b8b07050e5] Merge tag 'arc-5.11-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc testing commit 0653161f0faca68b77b3f36fb4b4b9b8b07050e5 with gcc (GCC) 8.1.0 kernel signature: cad5b00276491fcc35dba433894ad3f1f516b62d83863c4c2059a96620061ae6 all runs: OK # git bisect good 0653161f0faca68b77b3f36fb4b4b9b8b07050e5 Bisecting: 53 revisions left to test after this (roughly 6 steps) [28318f53503090fcd8fd27c49445396ea2ace44b] Merge tag 'usb-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit 28318f53503090fcd8fd27c49445396ea2ace44b with gcc (GCC) 8.1.0 kernel signature: 8bbb2b8f794a42555f666fc96cb4bf629c823e979bd2fe5cb17401866f4ffc1a all runs: OK # git bisect good 28318f53503090fcd8fd27c49445396ea2ace44b Bisecting: 26 revisions left to test after this (roughly 5 steps) [5342fd4255021ef0c4ce7be52eea1c4ebda11c63] bcache: set bcache device into read-only mode for BCH_FEATURE_INCOMPAT_OBSO_LARGE_BUCKET testing commit 5342fd4255021ef0c4ce7be52eea1c4ebda11c63 with gcc (GCC) 8.1.0 kernel signature: 5a133f8547e0a38c7bfec75e12bebde1ec6144369c9d884f3f6abe33c7f7ce57 all runs: OK # git bisect good 5342fd4255021ef0c4ce7be52eea1c4ebda11c63 Bisecting: 13 revisions left to test after this (roughly 4 steps) [d9d05217cb6990b9a56e13b56e7a1b71e2551f6c] io_uring: stop SQPOLL submit on creator's death testing commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c with gcc (GCC) 8.1.0 kernel signature: 96f8487b2e16375dfbff1f816c0e6a705604bf23a6d2486d65f3f14e93653179 all runs: crashed: WARNING in io_ring_ctx_wait_and_kill # git bisect bad d9d05217cb6990b9a56e13b56e7a1b71e2551f6c Bisecting: 6 revisions left to test after this (roughly 3 steps) [3e2224c5867fead6c0b94b84727cc676ac6353a3] io_uring: Fix return value from alloc_fixed_file_ref_node testing commit 3e2224c5867fead6c0b94b84727cc676ac6353a3 with gcc (GCC) 8.1.0 kernel signature: c6fb4211ccf58999ed46a63b02cd7c5eac9522f5d459f50c59304c5c8416b86d all runs: OK # git bisect good 3e2224c5867fead6c0b94b84727cc676ac6353a3 Bisecting: 3 revisions left to test after this (roughly 2 steps) [b1445e59cc9a10fdb8f83810ae1f4feb941ab36b] io_uring: synchronise ev_posted() with waitqueues testing commit b1445e59cc9a10fdb8f83810ae1f4feb941ab36b with gcc (GCC) 8.1.0 kernel signature: 0f85de6d453e73fc021874b7eceb4e4d1c374eb0625dd8e7490d20ac18bc994d all runs: OK # git bisect good b1445e59cc9a10fdb8f83810ae1f4feb941ab36b Bisecting: 1 revision left to test after this (roughly 1 step) [4f793dc40bc605b97624fd36baf085b3c35e8bfd] io_uring: inline io_uring_attempt_task_drop() testing commit 4f793dc40bc605b97624fd36baf085b3c35e8bfd with gcc (GCC) 8.1.0 kernel signature: 77049b5c3b7dc3f5a7a6a68a9d20ceca25e111af75cad62055387ea9ee2dee2a all runs: OK # git bisect good 4f793dc40bc605b97624fd36baf085b3c35e8bfd Bisecting: 0 revisions left to test after this (roughly 0 steps) [6b5733eb638b7068ab7cb34e663b55a1d1892d85] io_uring: add warn_once for io_uring_flush() testing commit 6b5733eb638b7068ab7cb34e663b55a1d1892d85 with gcc (GCC) 8.1.0 kernel signature: 80034015ac333611f853691a0621496a068406f2a7c6f7cd51b33ef56c2851d0 all runs: OK # git bisect good 6b5733eb638b7068ab7cb34e663b55a1d1892d85 d9d05217cb6990b9a56e13b56e7a1b71e2551f6c is the first bad commit commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c Author: Pavel Begunkov Date: Fri Jan 8 20:57:25 2021 +0000 io_uring: stop SQPOLL submit on creator's death When the creator of SQPOLL io_uring dies (i.e. sqo_task), we don't want its internals like ->files and ->mm to be poked by the SQPOLL task, it have never been nice and recently got racy. That can happen when the owner undergoes destruction and SQPOLL tasks tries to submit new requests in parallel, and so calls io_sq_thread_acquire*(). That patch halts SQPOLL submissions when sqo_task dies by introducing sqo_dead flag. Once set, the SQPOLL task must not do any submission, which is synchronised by uring_lock as well as the new flag. The tricky part is to make sure that disabling always happens, that means either the ring is discovered by creator's do_exit() -> cancel, or if the final close() happens before it's done by the creator. The last is guaranteed by the fact that for SQPOLL the creator task and only it holds exactly one file note, so either it pins up to do_exit() or removed by the creator on the final put in flush. (see comments in uring_flush() around file->f_count == 2). One more place that can trigger io_sq_thread_acquire_*() is __io_req_task_submit(). Shoot off requests on sqo_dead there, even though actually we don't need to. That's because cancellation of sqo_task should wait for the request before going any further. note 1: io_disable_sqo_submit() does io_ring_set_wakeup_flag() so the caller would enter the ring to get an error, but it still doesn't guarantee that the flag won't be cleared. note 2: if final __userspace__ close happens not from the creator task, the file note will pin the ring until the task dies. Fixed: b1b6b5a30dce8 ("kernel/io_uring: cancel io_uring before task works") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe fs/io_uring.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 9 deletions(-) culprit signature: 96f8487b2e16375dfbff1f816c0e6a705604bf23a6d2486d65f3f14e93653179 parent signature: 80034015ac333611f853691a0621496a068406f2a7c6f7cd51b33ef56c2851d0 revisions tested: 16, total time: 3h21m58.333581889s (build: 1h13m57.609750524s, test: 2h6m31.509205047s) first bad commit: d9d05217cb6990b9a56e13b56e7a1b71e2551f6c io_uring: stop SQPOLL submit on creator's death recipients (to): ["asml.silence@gmail.com" "axboe@kernel.dk" "axboe@kernel.dk" "io-uring@vger.kernel.org"] recipients (cc): ["linux-fsdevel@vger.kernel.org" "linux-kernel@vger.kernel.org" "viro@zeniv.linux.org.uk"] crash: WARNING in io_ring_ctx_wait_and_kill ------------[ cut here ]------------ WARNING: CPU: 1 PID: 10207 at fs/io_uring.c:8717 __mmap_lock_trace_acquire_returned include/linux/mmap_lock.h:36 [inline] WARNING: CPU: 1 PID: 10207 at fs/io_uring.c:8717 mmap_write_lock include/linux/mmap_lock.h:72 [inline] WARNING: CPU: 1 PID: 10207 at fs/io_uring.c:8717 io_unaccount_mem fs/io_uring.c:8211 [inline] WARNING: CPU: 1 PID: 10207 at fs/io_uring.c:8717 io_ring_ctx_wait_and_kill+0x1df/0x1f0 fs/io_uring.c:8741 Modules linked in: CPU: 1 PID: 10207 Comm: syz-executor.1 Not tainted 5.11.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:io_ring_ctx_wait_and_kill+0x1df/0x1f0 fs/io_uring.c:8717 Code: ff be 01 00 00 00 48 89 ef e8 4d 2d ee ff e9 60 ff ff ff ba 01 00 00 00 be 01 00 00 00 48 89 ef e8 76 2b ee ff e9 23 ff ff ff <0f> 0b 80 4b 44 40 e9 50 fe ff ff 66 0f 1f 44 00 00 48 8b be b0 01 RSP: 0018:ffffc90002f6bea0 EFLAGS: 00010246 RAX: 0000000000000002 RBX: ffff88811e5e3800 RCX: 0000000000000001 RDX: 0000000080000001 RSI: ffffffff8490a4c1 RDI: 00000000ffffffff RBP: ffff88811e5e3b80 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000001 R11: 0000000000000002 R12: ffff888141cee040 R13: ffff8881407fe160 R14: ffff888110f2a270 R15: 0000000000000000 FS: 00007fb353526700(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffc72682e27 CR3: 000000011e36e000 CR4: 0000000000350ee0 Call Trace: io_uring_release+0x17/0x20 fs/io_uring.c:8759 __fput+0xa1/0x250 fs/file_table.c:280 task_work_run+0x68/0xb0 kernel/task_work.c:140 tracehook_notify_resume include/linux/tracehook.h:189 [inline] exit_to_user_mode_loop kernel/entry/common.c:174 [inline] exit_to_user_mode_prepare+0x22c/0x250 kernel/entry/common.c:201 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline] syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:302 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x45e219 Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007fb353525be8 EFLAGS: 00000206 ORIG_RAX: 00000000000001a9 RAX: ffffffffffffffe8 RBX: 0000000020000040 RCX: 000000000045e219 RDX: 0000000020ffd000 RSI: 0000000020000040 RDI: 0000000000002094 RBP: 000000000119bfd8 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000206 R12: 0000000020ffd000 R13: 0000000020ffd000 R14: 0000000000000000 R15: 0000000000000000