bisecting cause commit starting from 65f0d2414b7079556fbbcc070b3d1c9f9587606d building syzkaller on 269d24e857a757d09a898086a2fa6fa5d827c3e1 testing commit 65f0d2414b7079556fbbcc070b3d1c9f9587606d with gcc (GCC) 8.1.0 kernel signature: 87a7799bb203f19acdacb4673afefbc2d40e5dd34fca1794a4adfaad69436f02 all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in io_disable_sqo_submit testing release v5.10 testing commit 2c85ebc57b3e1817b6ce1a6b703928e113a90442 with gcc (GCC) 8.1.0 kernel signature: 5035332b8f978c13779dd8be0ba1e4ce5debb9716b94f5b4b7dce31f48ab366b all runs: OK # git bisect start 65f0d2414b7079556fbbcc070b3d1c9f9587606d 2c85ebc57b3e1817b6ce1a6b703928e113a90442 Bisecting: 6983 revisions left to test after this (roughly 13 steps) [ef72cd3c5ce168829c6684ecb2cae047d3493690] ethtool: fix error paths in ethnl_set_channels() testing commit ef72cd3c5ce168829c6684ecb2cae047d3493690 with gcc (GCC) 8.1.0 kernel signature: 65acb08fc412f87186c39ab3f23bd7318b0a74adcc4df5d852bbaabab7c7f2f3 all runs: OK # git bisect good ef72cd3c5ce168829c6684ecb2cae047d3493690 Bisecting: 3383 revisions left to test after this (roughly 12 steps) [9805529ec544ea7a82d891d5239a8ebd3dbb2a3e] Merge tag 'arm-soc-dt-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc testing commit 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e with gcc (GCC) 8.1.0 kernel signature: 65993e7867388edc139d718be1f61798dd2e610add17660c7e442d0d41a2bdcf all runs: OK # git bisect good 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e Bisecting: 1755 revisions left to test after this (roughly 11 steps) [f4a2f7866faaf89ea1595b136e01fcb336b46aab] Merge tag 'rtc-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/abelloni/linux testing commit f4a2f7866faaf89ea1595b136e01fcb336b46aab with gcc (GCC) 8.1.0 kernel signature: 9d78666dee968437ec2f7d54dca391f36fba8aaaac48e19db1c71b9bfe0d7826 all runs: OK # git bisect good f4a2f7866faaf89ea1595b136e01fcb336b46aab Bisecting: 882 revisions left to test after this (roughly 10 steps) [771e7e4161053e606592b9cd056ef7e2ea2316d5] Merge tag 'block-5.11-2020-12-23' of git://git.kernel.dk/linux-block testing commit 771e7e4161053e606592b9cd056ef7e2ea2316d5 with gcc (GCC) 8.1.0 kernel signature: a190664f47e969b8b2bd30b09836e8241cb1a525b591b517e945140cd447d550 all runs: OK # git bisect good 771e7e4161053e606592b9cd056ef7e2ea2316d5 Bisecting: 441 revisions left to test after this (roughly 9 steps) [0b9902c1fcc59ba75268386c0420a554f8844168] s390/qeth: fix deadlock during recovery testing commit 0b9902c1fcc59ba75268386c0420a554f8844168 with gcc (GCC) 8.1.0 kernel signature: 6be5f379c2aebee0bf0b59a3ce0ff11e61163bc813009c84ad8b203c0ba9fc59 all runs: OK # git bisect good 0b9902c1fcc59ba75268386c0420a554f8844168 Bisecting: 226 revisions left to test after this (roughly 8 steps) [caab314792aca89f327abc8b9f730526d3080366] Merge branch 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux testing commit caab314792aca89f327abc8b9f730526d3080366 with gcc (GCC) 8.1.0 kernel signature: 157f48d12cafbb002f52803e59df2cdfbb23507faae340df1f0bdfdd9be4829b all runs: OK # git bisect good caab314792aca89f327abc8b9f730526d3080366 Bisecting: 107 revisions left to test after this (roughly 7 steps) [d430adfea8d2c5baa186cabb130235f72fecbd5b] Merge tag 'io_uring-5.11-2021-01-10' of git://git.kernel.dk/linux-block testing commit d430adfea8d2c5baa186cabb130235f72fecbd5b with gcc (GCC) 8.1.0 kernel signature: b806a1bf7df43197b550bd9f95f437e0d41c76effcce64287defe40116f2699b all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in io_disable_sqo_submit # git bisect bad d430adfea8d2c5baa186cabb130235f72fecbd5b Bisecting: 55 revisions left to test after this (roughly 6 steps) [e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e] Merge tag 'char-misc-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc testing commit e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e with gcc (GCC) 8.1.0 kernel signature: 162bb5418f34bbb77c8f784e3e3ca4042f3a68f0c46e5dba5260e3399927473f all runs: OK # git bisect good e07cd2f3e7e525fa8df334d11beceb4c1bdcc74e Bisecting: 28 revisions left to test after this (roughly 5 steps) [96ebc9c871d8a28fb22aa758dd9188a4732df482] usb: uas: Add PNY USB Portable SSD to unusual_uas testing commit 96ebc9c871d8a28fb22aa758dd9188a4732df482 with gcc (GCC) 8.1.0 kernel signature: 03258d8a83c4c6f677a103652d955b5ac3837e59ee16929a078f8669516d9909 all runs: OK # git bisect good 96ebc9c871d8a28fb22aa758dd9188a4732df482 Bisecting: 13 revisions left to test after this (roughly 4 steps) [28318f53503090fcd8fd27c49445396ea2ace44b] Merge tag 'usb-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit 28318f53503090fcd8fd27c49445396ea2ace44b with gcc (GCC) 8.1.0 kernel signature: ac244629ff8e087543a00a42a7fffdbd100825df824497f4e0816cf1702acf9e all runs: OK # git bisect good 28318f53503090fcd8fd27c49445396ea2ace44b Bisecting: 6 revisions left to test after this (roughly 3 steps) [80c18e4ac20c9cde420cb3ffab48c936147cf07d] io_uring: trigger eventfd for IOPOLL testing commit 80c18e4ac20c9cde420cb3ffab48c936147cf07d with gcc (GCC) 8.1.0 kernel signature: f06b14af5cb0a914ed1bc714d8d118284280dccb10d3bcaf0a20d902b4ae78ee all runs: OK # git bisect good 80c18e4ac20c9cde420cb3ffab48c936147cf07d Bisecting: 3 revisions left to test after this (roughly 2 steps) [55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550] io_uring: io_rw_reissue lockdep annotations testing commit 55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550 with gcc (GCC) 8.1.0 kernel signature: 354b42c0d2890ce29564c1844f005f2d42d65b1876f76b0000f16cc4dc7a67c2 all runs: OK # git bisect good 55e6ac1e1f31c7f678d9f3c8d54c6f102e5f1550 Bisecting: 1 revision left to test after this (roughly 1 step) [6b5733eb638b7068ab7cb34e663b55a1d1892d85] io_uring: add warn_once for io_uring_flush() testing commit 6b5733eb638b7068ab7cb34e663b55a1d1892d85 with gcc (GCC) 8.1.0 kernel signature: 02d922be7abdabd9730a07d6fb50b58eec4884dab1d42893fdee5a7c20d14694 all runs: OK # git bisect good 6b5733eb638b7068ab7cb34e663b55a1d1892d85 Bisecting: 0 revisions left to test after this (roughly 0 steps) [d9d05217cb6990b9a56e13b56e7a1b71e2551f6c] io_uring: stop SQPOLL submit on creator's death testing commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c with gcc (GCC) 8.1.0 kernel signature: 30da954d8dd0aaf91b946d4fd22e394e917647afdc474be090218786484507cb all runs: crashed: BUG: unable to handle kernel NULL pointer dereference in io_disable_sqo_submit # git bisect bad d9d05217cb6990b9a56e13b56e7a1b71e2551f6c d9d05217cb6990b9a56e13b56e7a1b71e2551f6c is the first bad commit commit d9d05217cb6990b9a56e13b56e7a1b71e2551f6c Author: Pavel Begunkov Date: Fri Jan 8 20:57:25 2021 +0000 io_uring: stop SQPOLL submit on creator's death When the creator of SQPOLL io_uring dies (i.e. sqo_task), we don't want its internals like ->files and ->mm to be poked by the SQPOLL task, it have never been nice and recently got racy. That can happen when the owner undergoes destruction and SQPOLL tasks tries to submit new requests in parallel, and so calls io_sq_thread_acquire*(). That patch halts SQPOLL submissions when sqo_task dies by introducing sqo_dead flag. Once set, the SQPOLL task must not do any submission, which is synchronised by uring_lock as well as the new flag. The tricky part is to make sure that disabling always happens, that means either the ring is discovered by creator's do_exit() -> cancel, or if the final close() happens before it's done by the creator. The last is guaranteed by the fact that for SQPOLL the creator task and only it holds exactly one file note, so either it pins up to do_exit() or removed by the creator on the final put in flush. (see comments in uring_flush() around file->f_count == 2). One more place that can trigger io_sq_thread_acquire_*() is __io_req_task_submit(). Shoot off requests on sqo_dead there, even though actually we don't need to. That's because cancellation of sqo_task should wait for the request before going any further. note 1: io_disable_sqo_submit() does io_ring_set_wakeup_flag() so the caller would enter the ring to get an error, but it still doesn't guarantee that the flag won't be cleared. note 2: if final __userspace__ close happens not from the creator task, the file note will pin the ring until the task dies. Fixed: b1b6b5a30dce8 ("kernel/io_uring: cancel io_uring before task works") Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe fs/io_uring.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 53 insertions(+), 9 deletions(-) culprit signature: 30da954d8dd0aaf91b946d4fd22e394e917647afdc474be090218786484507cb parent signature: 02d922be7abdabd9730a07d6fb50b58eec4884dab1d42893fdee5a7c20d14694 revisions tested: 16, total time: 3h19m56.256712084s (build: 1h13m48.081765669s, test: 2h4m42.640491957s) first bad commit: d9d05217cb6990b9a56e13b56e7a1b71e2551f6c io_uring: stop SQPOLL submit on creator's death recipients (to): ["asml.silence@gmail.com" "axboe@kernel.dk" "axboe@kernel.dk" "io-uring@vger.kernel.org"] recipients (cc): ["linux-fsdevel@vger.kernel.org" "linux-kernel@vger.kernel.org" "viro@zeniv.linux.org.uk"] crash: BUG: unable to handle kernel NULL pointer dereference in io_disable_sqo_submit Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007f9daaff0be8 EFLAGS: 00000206 ORIG_RAX: 00000000000001a9 RAX: ffffffffffffffda RBX: 0000000020000200 RCX: 000000000045e219 RDX: 0000000020ffd000 RSI: 0000000020000200 RDI: 0000000000003040 RBP: 00007f9daaff0ca0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000206 R12: 0000000020ffd000 R13: 0000000020ffb000 R14: 0000000000000000 R15: 0000000000000000 BUG: kernel NULL pointer dereference, address: 0000000000000114 #PF: supervisor write access in kernel mode #PF: error_code(0x0002) - not-present page PGD 11f97b067 P4D 11f97b067 PUD 11f97c067 PMD 0 Oops: 0002 [#1] PREEMPT SMP CPU: 0 PID: 10208 Comm: syz-executor.0 Not tainted 5.11.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:io_ring_set_wakeup_flag fs/io_uring.c:6929 [inline] RIP: 0010:io_disable_sqo_submit+0x4d/0x60 fs/io_uring.c:8891 Code: e8 f8 d3 1b 02 48 89 ef 80 4b 44 40 48 8d ab 80 04 00 00 e8 85 bd 1b 02 48 89 ef e8 3d 0e 1c 02 48 8b 83 c0 00 00 00 48 89 ef <83> 88 14 01 00 00 01 5b 5d e9 65 10 1c 02 0f 0b eb b8 90 41 55 65 RSP: 0018:ffffc90002fabe50 EFLAGS: 00010046 RAX: 0000000000000000 RBX: ffff88811fa24000 RCX: ffffffff86b75560 RDX: 0000000000000001 RSI: ffffffff846a9e5c RDI: ffff88811fa24480 RBP: ffff88811fa24480 R08: ffffffff8665f4e0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000020000200 R14: fffffffffffffff4 R15: 0000000000080140 FS: 00007f9daaff1700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000114 CR3: 000000011f97a000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: io_uring_create fs/io_uring.c:9711 [inline] io_uring_setup+0x861/0x1300 fs/io_uring.c:9739 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x45e219 Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007f9daaff0be8 EFLAGS: 00000206 ORIG_RAX: 00000000000001a9 RAX: ffffffffffffffda RBX: 0000000020000200 RCX: 000000000045e219 RDX: 0000000020ffd000 RSI: 0000000020000200 RDI: 0000000000003040 RBP: 00007f9daaff0ca0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000206 R12: 0000000020ffd000 R13: 0000000020ffb000 R14: 0000000000000000 R15: 0000000000000000 Modules linked in: CR2: 0000000000000114 ---[ end trace 4dd1b93aa9c75b0a ]--- RIP: 0010:io_ring_set_wakeup_flag fs/io_uring.c:6929 [inline] RIP: 0010:io_disable_sqo_submit+0x4d/0x60 fs/io_uring.c:8891 Code: e8 f8 d3 1b 02 48 89 ef 80 4b 44 40 48 8d ab 80 04 00 00 e8 85 bd 1b 02 48 89 ef e8 3d 0e 1c 02 48 8b 83 c0 00 00 00 48 89 ef <83> 88 14 01 00 00 01 5b 5d e9 65 10 1c 02 0f 0b eb b8 90 41 55 65 RSP: 0018:ffffc90002fabe50 EFLAGS: 00010046 RAX: 0000000000000000 RBX: ffff88811fa24000 RCX: ffffffff86b75560 RDX: 0000000000000001 RSI: ffffffff846a9e5c RDI: ffff88811fa24480 RBP: ffff88811fa24480 R08: ffffffff8665f4e0 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000020000200 R14: fffffffffffffff4 R15: 0000000000080140 FS: 00007f9daaff1700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000114 CR3: 000000011f97a000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400