bisecting cause commit starting from 139c2d13c258bacc545fc2a4091f7fb0a6fb08fd building syzkaller on 25bb509e5964da8203766c4039e4fef25e4689b1 testing commit 139c2d13c258bacc545fc2a4091f7fb0a6fb08fd with gcc (GCC) 8.1.0 run #0: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #1: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #2: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #3: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #4: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #5: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #6: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #7: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #8: crashed: INFO: task hung in mpage_prepare_extent_to_map run #9: crashed: INFO: task hung in mpage_prepare_extent_to_map testing release v5.3 testing commit 4d856f72c10ecb060868ed10ff1b1453943fc6c8 with gcc (GCC) 8.1.0 all runs: OK # git bisect start 139c2d13c258bacc545fc2a4091f7fb0a6fb08fd v5.3 Bisecting: 10645 revisions left to test after this (roughly 13 steps) [a703d279c57e1bfe2b6536c3a17c1c498b416d24] Merge tag 'clk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux testing commit a703d279c57e1bfe2b6536c3a17c1c498b416d24 with gcc (GCC) 8.1.0 run #0: OK run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: boot failed: WARNING: workqueue cpumask: online intersect > possible intersect # git bisect good a703d279c57e1bfe2b6536c3a17c1c498b416d24 Bisecting: 5352 revisions left to test after this (roughly 12 steps) [88a557a701a38bc49be84c7e19d42a04f48e6983] Merge remote-tracking branch 'nfsd/nfsd-next' testing commit 88a557a701a38bc49be84c7e19d42a04f48e6983 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 88a557a701a38bc49be84c7e19d42a04f48e6983 Bisecting: 2855 revisions left to test after this (roughly 11 steps) [6c1b0a2961995d2c635f22e8063fcd29fb80b5d4] Merge remote-tracking branch 'drm/drm-next' testing commit 6c1b0a2961995d2c635f22e8063fcd29fb80b5d4 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 6c1b0a2961995d2c635f22e8063fcd29fb80b5d4 Bisecting: 1495 revisions left to test after this (roughly 11 steps) [76ba0f534b74f04da4006886cd96cd75c9b7c2cf] Merge remote-tracking branch 'spi/for-next' testing commit 76ba0f534b74f04da4006886cd96cd75c9b7c2cf with gcc (GCC) 8.1.0 run #0: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #1: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #2: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #3: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #4: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #5: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #6: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #7: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #8: crashed: INFO: task hung in io_wq_destroy run #9: crashed: INFO: task hung in io_wq_destroy # git bisect bad 76ba0f534b74f04da4006886cd96cd75c9b7c2cf Bisecting: 652 revisions left to test after this (roughly 9 steps) [e053201c0c34141bb32d834f35e3ac4179e2e519] Merge remote-tracking branch 'drm-tegra/drm/tegra/for-next' testing commit e053201c0c34141bb32d834f35e3ac4179e2e519 with gcc (GCC) 8.1.0 all runs: OK # git bisect good e053201c0c34141bb32d834f35e3ac4179e2e519 Bisecting: 334 revisions left to test after this (roughly 8 steps) [444e2093136ef89943421ea2c672c4bc9aa8ff27] Merge remote-tracking branch 'block/for-next' testing commit 444e2093136ef89943421ea2c672c4bc9aa8ff27 with gcc (GCC) 8.1.0 run #0: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #1: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #2: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #3: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #4: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #5: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #6: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #7: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #8: crashed: INFO: task hung in io_wq_destroy run #9: crashed: INFO: task hung in io_wq_destroy # git bisect bad 444e2093136ef89943421ea2c672c4bc9aa8ff27 Bisecting: 158 revisions left to test after this (roughly 7 steps) [b2d6ee75312649d55b41386d1d80cdbca48e3cf0] ASOC: adau7118: Change regulators id testing commit b2d6ee75312649d55b41386d1d80cdbca48e3cf0 with gcc (GCC) 8.1.0 all runs: OK # git bisect good b2d6ee75312649d55b41386d1d80cdbca48e3cf0 Bisecting: 70 revisions left to test after this (roughly 6 steps) [20d71346dfad85412a99ed72fecf5353942af55a] Merge remote-tracking branch 'sound/for-next' testing commit 20d71346dfad85412a99ed72fecf5353942af55a with gcc (GCC) 8.1.0 all runs: OK # git bisect good 20d71346dfad85412a99ed72fecf5353942af55a Bisecting: 32 revisions left to test after this (roughly 5 steps) [f015479bab2cce2c304ff77801f1542292efb7f2] Merge branch 'for-5.5/drivers' into for-next testing commit f015479bab2cce2c304ff77801f1542292efb7f2 with gcc (GCC) 8.1.0 run #0: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #1: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #2: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #3: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #4: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #5: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #6: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #7: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #8: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #9: crashed: INFO: task hung in io_wq_destroy # git bisect bad f015479bab2cce2c304ff77801f1542292efb7f2 Bisecting: 21 revisions left to test after this (roughly 4 steps) [3f982fff29b4ad339b36e9cf43422d1039f9917a] Merge branch 'for-5.5/drivers' into for-next testing commit 3f982fff29b4ad339b36e9cf43422d1039f9917a with gcc (GCC) 8.1.0 all runs: OK # git bisect good 3f982fff29b4ad339b36e9cf43422d1039f9917a Bisecting: 10 revisions left to test after this (roughly 4 steps) [6ccfabc9b7f40f3775b390aa9bf3e4d31880f6bf] io_uring: add support for canceling timeout requests testing commit 6ccfabc9b7f40f3775b390aa9bf3e4d31880f6bf with gcc (GCC) 8.1.0 all runs: OK # git bisect good 6ccfabc9b7f40f3775b390aa9bf3e4d31880f6bf Bisecting: 5 revisions left to test after this (roughly 3 steps) [0c4434ce7b762220ebc98f9b61037f70172c739e] Merge branch 'for-5.5/io_uring-wq' into for-next testing commit 0c4434ce7b762220ebc98f9b61037f70172c739e with gcc (GCC) 8.1.0 all runs: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all # git bisect bad 0c4434ce7b762220ebc98f9b61037f70172c739e Bisecting: 2 revisions left to test after this (roughly 1 step) [46134db8fdc5977e9dc80d1f9d2746521570b56e] io-wq: small threadpool implementation for io_uring testing commit 46134db8fdc5977e9dc80d1f9d2746521570b56e with gcc (GCC) 8.1.0 all runs: OK # git bisect good 46134db8fdc5977e9dc80d1f9d2746521570b56e Bisecting: 1 revision left to test after this (roughly 1 step) [d5f773aba1186142d52aef8242a426310a39fa86] io_uring: replace workqueue usage with io-wq testing commit d5f773aba1186142d52aef8242a426310a39fa86 with gcc (GCC) 8.1.0 run #0: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #1: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #2: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #3: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #4: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #5: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #6: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #7: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #8: crashed: KASAN: null-ptr-deref Write in io_wq_cancel_all run #9: crashed: BUG: unable to handle kernel NULL pointer dereference in io_wq_cancel_all # git bisect bad d5f773aba1186142d52aef8242a426310a39fa86 d5f773aba1186142d52aef8242a426310a39fa86 is the first bad commit commit d5f773aba1186142d52aef8242a426310a39fa86 Author: Jens Axboe Date: Thu Oct 24 07:25:42 2019 -0600 io_uring: replace workqueue usage with io-wq Drop various work-arounds we have for workqueues: - We no longer need the async_list for tracking sequential IO. - We don't have to maintain our own mm tracking/setting. - We don't need a separate workqueue for buffered writes. This didn't even work that well to begin with, as it was suboptimal for multiple buffered writers on multiple files. - We can properly cancel pending interruptible work. This fixes deadlocks with particularly socket IO, where we cannot cancel them when the io_uring is closed. Hence the ring will wait forever for these requests to complete, which may never happen. This is different from disk IO where we know requests will complete in a finite amount of time. - Due to being able to cancel work interruptible work that is already running, we can implement file table support for work. We need that for supporting system calls that add to a process file table. - It gets us one step closer to adding async support for any system call. Signed-off-by: Jens Axboe :040000 040000 4c5834f8ccde36bbdb76cf27654d792b825ede68 b514746517a7af6c4802a9f2020463027ace2033 M fs :040000 040000 157ea496cd6121360830b53a6ccc905ca19e1f34 85b4c49736bfc58a78a7516a110eb319d46900d7 M include :040000 040000 ceda7e59c359fb11616071c1af447af9c94b7a85 85a16f73da800a0b58036ca1f8094ed06f3c62ea M init revisions tested: 16, total time: 3h56m23.855882485s (build: 1h35m56.149927274s, test: 2h15m3.774984608s) first bad commit: d5f773aba1186142d52aef8242a426310a39fa86 io_uring: replace workqueue usage with io-wq cc: ["akpm@linux-foundation.org" "axboe@kernel.dk" "dan.j.williams@intel.com" "dhowells@redhat.com" "gregkh@linuxfoundation.org" "hannes@cmpxchg.org" "joel@joelfernandes.org" "linux-block@vger.kernel.org" "linux-fsdevel@vger.kernel.org" "linux-kernel@vger.kernel.org" "mchehab+samsung@kernel.org" "mingo@redhat.com" "patrick.bellasi@arm.com" "rgb@redhat.com" "rostedt@goodmis.org" "viro@zeniv.linux.org.uk" "yamada.masahiro@socionext.com"] crash: BUG: unable to handle kernel NULL pointer dereference in io_wq_cancel_all BUG: kernel NULL pointer dereference, address: 0000000000000004 #PF: supervisor write access in kernel mode #PF: error_code(0x0002) - not-present page PGD 0 P4D 0 Oops: 0002 [#1] PREEMPT SMP KASAN CPU: 1 PID: 19757 Comm: syz-executor.1 Not tainted 5.4.0-rc4+ #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:arch_set_bit arch/x86/include/asm/bitops.h:55 [inline] RIP: 0010:set_bit include/asm-generic/bitops-instrumented.h:29 [inline] RIP: 0010:io_wq_cancel_all+0x22/0x210 fs/io-wq.c:574 Code: e9 3a ff ff ff 0f 1f 00 55 be 08 00 00 00 48 89 e5 41 57 49 89 ff 48 83 c7 08 41 56 41 55 41 54 53 48 83 ec 08 e8 3e 66 e1 ff 41 80 4f 08 02 e8 63 99 9d ff 31 d2 45 31 c9 45 31 c0 31 f6 68 RSP: 0018:ffff888081f179a8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff88809cc67000 RCX: ffffffff81b9e932 RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000004 RBP: ffff888081f179d8 R08: 0000000000000006 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff88809cc673d0 R13: ffff88809cc67078 R14: ffff88809cc67380 R15: fffffffffffffffc FS: 0000000000000000(0000) GS:ffff8880aeb00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000004 CR3: 000000000846d000 CR4: 00000000001406e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: io_ring_ctx_wait_and_kill+0x1a4/0x610 fs/io_uring.c:3679 io_uring_release+0x3d/0x50 fs/io_uring.c:3691 __fput+0x25a/0x770 fs/file_table.c:280 ____fput+0x9/0x10 fs/file_table.c:313 task_work_run+0x108/0x180 kernel/task_work.c:113 exit_task_work include/linux/task_work.h:22 [inline] do_exit+0x9d4/0x2c70 kernel/exit.c:817 do_group_exit+0xf4/0x2e0 kernel/exit.c:921 get_signal+0x36c/0x1d50 kernel/signal.c:2734 do_signal+0x87/0x1710 arch/x86/kernel/signal.c:815 exit_to_usermode_loop+0x114/0x2e0 arch/x86/entry/common.c:159 prepare_exit_to_usermode arch/x86/entry/common.c:194 [inline] syscall_return_slowpath+0x369/0x410 arch/x86/entry/common.c:274 ret_from_fork+0x15/0x30 arch/x86/entry/entry_64.S:344 RIP: 0033:0x45c909 Code: Bad RIP value. RSP: 002b:00007f5d80ca1db0 EFLAGS: 00000202 ORIG_RAX: 0000000000000038 RAX: 0000000000000000 RBX: 00007f5d80ca2700 RCX: 000000000045c909 RDX: 00007f5d80ca29d0 RSI: 00007f5d80ca1db0 RDI: 00000000003d0f00 RBP: 00007ffc07e9f750 R08: 00007f5d80ca2700 R09: 00007f5d80ca2700 R10: 00007f5d80ca29d0 R11: 0000000000000202 R12: 0000000000000000 R13: 00007ffc07e9f5ef R14: 00007f5d80ca29c0 R15: 000000000075bfd4 Modules linked in: CR2: 0000000000000004 ---[ end trace 8467466421cbe904 ]--- RIP: 0010:arch_set_bit arch/x86/include/asm/bitops.h:55 [inline] RIP: 0010:set_bit include/asm-generic/bitops-instrumented.h:29 [inline] RIP: 0010:io_wq_cancel_all+0x22/0x210 fs/io-wq.c:574 Code: e9 3a ff ff ff 0f 1f 00 55 be 08 00 00 00 48 89 e5 41 57 49 89 ff 48 83 c7 08 41 56 41 55 41 54 53 48 83 ec 08 e8 3e 66 e1 ff 41 80 4f 08 02 e8 63 99 9d ff 31 d2 45 31 c9 45 31 c0 31 f6 68 RSP: 0018:ffff888081f179a8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffff88809cc67000 RCX: ffffffff81b9e932 RDX: 0000000000000001 RSI: 0000000000000008 RDI: 0000000000000004 RBP: ffff888081f179d8 R08: 0000000000000006 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff88809cc673d0 R13: ffff88809cc67078 R14: ffff88809cc67380 R15: fffffffffffffffc FS: 0000000000000000(0000) GS:ffff8880aeb00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000000000045c8df CR3: 000000000846d000 CR4: 00000000001406e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400