bisecting cause commit starting from 7f25f0412c9e2be6811e8aedbd10ef795fff85f2 building syzkaller on 3cd800e43d452c348a66ba475143831d94969a24 testing commit 7f25f0412c9e2be6811e8aedbd10ef795fff85f2 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f8c8dca16f783ff71f4615a4b1f329366ee9f2f7812e63f4c34e44cbd732177a run #0: crashed: WARNING in io_ring_exit_work run #1: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #2: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #3: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #4: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #5: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #6: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #7: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #8: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #9: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #10: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #11: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #12: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #13: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #14: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #15: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #16: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #17: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #18: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #19: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero testing release v5.16 testing commit df0cc57e057f18e44dac8e6c18aba47ab53202f9 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5a962f1df95efeb1654f96bf2029104a7986810c8e745ce0f95463fbcee96f03 all runs: OK # git bisect start 7f25f0412c9e2be6811e8aedbd10ef795fff85f2 df0cc57e057f18e44dac8e6c18aba47ab53202f9 Bisecting: 6680 revisions left to test after this (roughly 13 steps) [42a7b4ed45e7667836fae4fb0e1ac6340588b1b0] Merge tag 'for-5.17/io_uring-2022-01-11' of git://git.kernel.dk/linux-block testing commit 42a7b4ed45e7667836fae4fb0e1ac6340588b1b0 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 4b58f84ac4260948ba861a1f298e5c10a071256ac57740dc7a82517a9d829b74 all runs: OK # git bisect good 42a7b4ed45e7667836fae4fb0e1ac6340588b1b0 Bisecting: 3355 revisions left to test after this (roughly 12 steps) [49ad227d54e842f436ed0122cb7c901d857b86cb] Merge tag '9p-for-5.17-rc1' of git://github.com/martinetd/linux testing commit 49ad227d54e842f436ed0122cb7c901d857b86cb compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 000a1d125380d32342fb884865d088215076b312e8a3de45a1e1b2a24d168492 all runs: OK # git bisect good 49ad227d54e842f436ed0122cb7c901d857b86cb Bisecting: 1678 revisions left to test after this (roughly 11 steps) [3828a76470792aaa5f2de5c0d7fce497187c1e35] zsmalloc: rename zs_stat_type to class_stat_type testing commit 3828a76470792aaa5f2de5c0d7fce497187c1e35 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a4c9aaca0ece4b5763e5ddfe25c87a6bf8381e6d6345e9598c6a882b487c69d9 all runs: OK # git bisect good 3828a76470792aaa5f2de5c0d7fce497187c1e35 Bisecting: 813 revisions left to test after this (roughly 10 steps) [eb2eb5161cdbd4f0acc574ef1c3ce799b980544b] Merge tag 'net-5.17-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit eb2eb5161cdbd4f0acc574ef1c3ce799b980544b compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1e25f607c812d2f065495c5492097612681af1a6c03e2851e2e248790b9645e0 run #0: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #1: crashed: WARNING in io_ring_exit_work run #2: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #3: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #4: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #5: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #6: crashed: WARNING in io_ring_exit_work run #7: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #8: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #9: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero # git bisect bad eb2eb5161cdbd4f0acc574ef1c3ce799b980544b Bisecting: 431 revisions left to test after this (roughly 9 steps) [23a46422c56144939c091c76cf389aa863ce9c18] Merge tag 'net-5.17-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit 23a46422c56144939c091c76cf389aa863ce9c18 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ed6213b2216005d968259efcb6560172d33920e83fe5c4ab91a4ed1990be692a all runs: OK # git bisect good 23a46422c56144939c091c76cf389aa863ce9c18 Bisecting: 221 revisions left to test after this (roughly 8 steps) [44aa31a2bfaab2ad36614f05162cda88ade9ce65] Merge tag 'usb-5.17-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit 44aa31a2bfaab2ad36614f05162cda88ade9ce65 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 45ce5ab37f24a08776cf7db42d1177dbf564fb4d65d25f4bbd3b37bb070af3b6 run #0: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #1: crashed: WARNING: locking bug in io_rsrc_node_ref_zero run #2: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #3: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #4: crashed: WARNING: locking bug in io_rsrc_node_ref_zero run #5: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #6: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #7: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #8: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #9: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero # git bisect bad 44aa31a2bfaab2ad36614f05162cda88ade9ce65 Bisecting: 97 revisions left to test after this (roughly 7 steps) [3cd7cd8a62e6f4b81e8429db7afcb11cc155ea3c] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm testing commit 3cd7cd8a62e6f4b81e8429db7afcb11cc155ea3c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d5fd16d9cd7fc621cd88dbcebc3cf7f191003977de15a97744bc34a34c80c8bb all runs: OK # git bisect good 3cd7cd8a62e6f4b81e8429db7afcb11cc155ea3c Bisecting: 49 revisions left to test after this (roughly 6 steps) [d1e7f0919ea84911e2ab965418cd502ba6a906e1] Merge tag 'fixes-v5.17-lsm-ceph-null' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security testing commit d1e7f0919ea84911e2ab965418cd502ba6a906e1 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d42e238c4a44a805c2d9a6db2efc174a195003ee0500f0c89a5bffe4e0ba8f94 all runs: OK # git bisect good d1e7f0919ea84911e2ab965418cd502ba6a906e1 Bisecting: 27 revisions left to test after this (roughly 5 steps) [3b58e9f3a301e175d2de6f7fa1e834c4605e1c73] Merge tag 'io_uring-5.17-2022-01-28' of git://git.kernel.dk/linux-block testing commit 3b58e9f3a301e175d2de6f7fa1e834c4605e1c73 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 871c4b43d4e80bd7e4ac7ca96ba8f48b9ec41d895e89a6cd08699e0af6c2846f all runs: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero # git bisect bad 3b58e9f3a301e175d2de6f7fa1e834c4605e1c73 Bisecting: 9 revisions left to test after this (roughly 4 steps) [df20597044e59cd383135b3d91c5b131dc333969] Merge tag 'trbe-cortex-a510-errata' of gitolite.kernel.org:pub/scm/linux/kernel/git/coresight/linux into for-next/fixes testing commit df20597044e59cd383135b3d91c5b131dc333969 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 565e0b881059e151e0521f3190a47bf9321a36183c5260341001c4be50741b36 all runs: OK # git bisect good df20597044e59cd383135b3d91c5b131dc333969 Bisecting: 5 revisions left to test after this (roughly 2 steps) [8defc2a5dd8f4c0cb19ecbaca8d3e89ab98524da] powerpc/64s/interrupt: Fix decrementer storm testing commit 8defc2a5dd8f4c0cb19ecbaca8d3e89ab98524da compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 76a7fc76f4128ff0c9c08f607c93b1a49fb64f96eb3690436329bc9d6a0639d8 all runs: OK # git bisect good 8defc2a5dd8f4c0cb19ecbaca8d3e89ab98524da Bisecting: 2 revisions left to test after this (roughly 2 steps) [d66c1e79b9fcbfc6559ea3c5b1243d590fa04179] Merge tag 'powerpc-5.17-3' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux testing commit d66c1e79b9fcbfc6559ea3c5b1243d590fa04179 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: aba23208aab7f64b6172fb2726095acebc41d9002b646aab9028cea07495e5bd all runs: OK # git bisect good d66c1e79b9fcbfc6559ea3c5b1243d590fa04179 Bisecting: 0 revisions left to test after this (roughly 1 step) [f6133fbd373811066c8441737e65f384c8f31974] io_uring: remove unused argument from io_rsrc_node_alloc testing commit f6133fbd373811066c8441737e65f384c8f31974 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f9f9a8279465e1172c10bfa71bcb5e8f20d3f009fe27f0539c7744be051f534a run #0: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #1: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #2: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #3: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #4: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #5: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #6: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #7: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #8: crashed: WARNING in io_ring_exit_work run #9: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero # git bisect bad f6133fbd373811066c8441737e65f384c8f31974 Bisecting: 0 revisions left to test after this (roughly 0 steps) [b36a2050040b2d839bdc044007cdd57101d7f881] io_uring: fix bug in slow unregistering of nodes testing commit b36a2050040b2d839bdc044007cdd57101d7f881 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5225d079227781c0dc01f4059ca3dd6179ac9649edb983404d812f86785818ad run #0: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #1: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #2: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #3: crashed: WARNING in io_ring_exit_work run #4: crashed: WARNING in io_ring_exit_work run #5: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #6: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #7: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #8: crashed: KASAN: use-after-free Read in io_rsrc_node_ref_zero run #9: crashed: WARNING in io_ring_exit_work # git bisect bad b36a2050040b2d839bdc044007cdd57101d7f881 b36a2050040b2d839bdc044007cdd57101d7f881 is the first bad commit commit b36a2050040b2d839bdc044007cdd57101d7f881 Author: Dylan Yudaken Date: Fri Jan 21 04:38:56 2022 -0800 io_uring: fix bug in slow unregistering of nodes In some cases io_rsrc_ref_quiesce will call io_rsrc_node_switch_start, and then immediately flush the delayed work queue &ctx->rsrc_put_work. However the percpu_ref_put does not immediately destroy the node, it will be called asynchronously via RCU. That ends up with io_rsrc_node_ref_zero only being called after rsrc_put_work has been flushed, and so the process ends up sleeping for 1 second unnecessarily. This patch executes the put code immediately if we are busy quiescing. Fixes: 4a38aed2a0a7 ("io_uring: batch reap of dead file registrations") Signed-off-by: Dylan Yudaken Link: https://lore.kernel.org/r/20220121123856.3557884-1-dylany@fb.com Signed-off-by: Jens Axboe fs/io_uring.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) parent commit dd81e1c7d5fb126e5fbc5c9e334d7b3ec29a16a0 wasn't tested testing commit dd81e1c7d5fb126e5fbc5c9e334d7b3ec29a16a0 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: b7cbc6b5f4b3c205e76c2a2b939b871b206366a98de3f635d567739c6ead12f4 culprit signature: 5225d079227781c0dc01f4059ca3dd6179ac9649edb983404d812f86785818ad parent signature: b7cbc6b5f4b3c205e76c2a2b939b871b206366a98de3f635d567739c6ead12f4 revisions tested: 16, total time: 2h57m42.021953048s (build: 1h47m28.618534025s, test: 1h8m39.790403852s) first bad commit: b36a2050040b2d839bdc044007cdd57101d7f881 io_uring: fix bug in slow unregistering of nodes recipients (to): ["axboe@kernel.dk" "axboe@kernel.dk" "dylany@fb.com" "io-uring@vger.kernel.org"] recipients (cc): ["asml.silence@gmail.com" "linux-kernel@vger.kernel.org"] crash: WARNING in io_ring_exit_work ------------[ cut here ]------------ WARNING: CPU: 1 PID: 1026 at fs/io_uring.c:9459 io_ring_ctx_free fs/io_uring.c:9459 [inline] WARNING: CPU: 1 PID: 1026 at fs/io_uring.c:9459 io_ring_exit_work+0x79d/0xa01 fs/io_uring.c:9623 Modules linked in: CPU: 1 PID: 1026 Comm: kworker/u4:5 Not tainted 5.17.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: events_unbound io_ring_exit_work RIP: 0010:io_ring_ctx_free fs/io_uring.c:9459 [inline] RIP: 0010:io_ring_exit_work+0x79d/0xa01 fs/io_uring.c:9623 Code: 4c 89 e2 b8 ff ff 37 00 48 c1 ea 03 48 c1 e0 2a 80 3c 02 00 74 08 4c 89 e7 e8 d4 06 6b f9 48 8b 83 e0 fe ff ff 49 39 c4 74 02 <0f> 0b 48 8d bb d8 fe ff ff b8 ff ff 37 00 48 89 fa 48 c1 e0 2a 48 RSP: 0018:ffffc9000463fbd0 EFLAGS: 00010202 RAX: ffff888019aee510 RBX: ffff8880782d8688 RCX: ffffc9000463f9b0 RDX: 1ffff1100f05b0ad RSI: ffffffff88cb6dc0 RDI: ffffffff8921a5e0 RBP: ffff8880782d8000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000001 R11: 0000000000000000 R12: ffff8880782d8568 R13: ffff8880782d8010 R14: ffffc9000463fc28 R15: 0000000000000005 FS: 0000000000000000(0000) GS:ffff8880b9f00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020000140 CR3: 000000007e751000 CR4: 00000000003506e0