bisecting cause commit starting from b145b0eb2031a620ca010174240963e4d2c6ce26 building syzkaller on f3f7d9c8c3dfa1406a5f5a5f5f45a351f533139f testing commit b145b0eb2031a620ca010174240963e4d2c6ce26 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue testing release v5.3 testing commit 4d856f72c10ecb060868ed10ff1b1453943fc6c8 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue testing release v5.2 testing commit 0ecfebd2b52404ae0c54a878c872bb93363ada36 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue testing release v5.1 testing commit e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue testing release v5.0 testing commit 1c163f4c7b3f621efff9b28a47abb36f7378d783 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue testing release v4.20 testing commit 8fe28cb58bcb235034b64cbbb7550a8a43fd88be with gcc (GCC) 8.1.0 all runs: OK # git bisect start v5.0 v4.20 Bisecting: 7011 revisions left to test after this (roughly 13 steps) [af7ddd8a627c62a835524b3f5b471edbbbcce025] Merge tag 'dma-mapping-4.21' of git://git.infradead.org/users/hch/dma-mapping testing commit af7ddd8a627c62a835524b3f5b471edbbbcce025 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad af7ddd8a627c62a835524b3f5b471edbbbcce025 Bisecting: 3448 revisions left to test after this (roughly 12 steps) [792bf4d871dea8b69be2aaabdd320d7c6ed15985] Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit 792bf4d871dea8b69be2aaabdd320d7c6ed15985 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 792bf4d871dea8b69be2aaabdd320d7c6ed15985 Bisecting: 1729 revisions left to test after this (roughly 11 steps) [aa9d6e0f33aea8a1879e7e53fe0e436943f9ce0c] linux/netlink.h: drop unnecessary extern prefix testing commit aa9d6e0f33aea8a1879e7e53fe0e436943f9ce0c with gcc (GCC) 8.1.0 all runs: OK # git bisect good aa9d6e0f33aea8a1879e7e53fe0e436943f9ce0c Bisecting: 835 revisions left to test after this (roughly 10 steps) [e0c38a4d1f196a4b17d2eba36afff8f656a4f1de] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next testing commit e0c38a4d1f196a4b17d2eba36afff8f656a4f1de with gcc (GCC) 8.1.0 all runs: OK # git bisect good e0c38a4d1f196a4b17d2eba36afff8f656a4f1de Bisecting: 460 revisions left to test after this (roughly 9 steps) [b12a9124eeb71d766a3e3eb594ebbb3fefc66902] Merge tag 'y2038-for-4.21' of ssh://gitolite.kernel.org:/pub/scm/linux/kernel/git/arnd/playground testing commit b12a9124eeb71d766a3e3eb594ebbb3fefc66902 with gcc (GCC) 8.1.0 all runs: OK # git bisect good b12a9124eeb71d766a3e3eb594ebbb3fefc66902 Bisecting: 230 revisions left to test after this (roughly 8 steps) [03198c4d9fc8e72ba8a5ad74959b61de7f2780a6] nvmet: mark nvmet_genctr static testing commit 03198c4d9fc8e72ba8a5ad74959b61de7f2780a6 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad 03198c4d9fc8e72ba8a5ad74959b61de7f2780a6 Bisecting: 114 revisions left to test after this (roughly 7 steps) [8295a69bdc3cb8707e645f9b2de6f3019a521882] drbd: don't override the queue_lock testing commit 8295a69bdc3cb8707e645f9b2de6f3019a521882 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad 8295a69bdc3cb8707e645f9b2de6f3019a521882 Bisecting: 57 revisions left to test after this (roughly 6 steps) [a13165441d58b216adbd50252a9cc829d78a6bce] loop: Push lo_ctl_mutex down into individual ioctls testing commit a13165441d58b216adbd50252a9cc829d78a6bce with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad a13165441d58b216adbd50252a9cc829d78a6bce Bisecting: 28 revisions left to test after this (roughly 5 steps) [db6d995235606191fa9db0c717e9d843200b71ea] block: remove request_list code testing commit db6d995235606191fa9db0c717e9d843200b71ea with gcc (GCC) 8.1.0 all runs: OK # git bisect good db6d995235606191fa9db0c717e9d843200b71ea Bisecting: 14 revisions left to test after this (roughly 4 steps) [b3c661b15d5ab11d982e58bee23e05c1780528a1] blk-mq: support multiple hctx maps testing commit b3c661b15d5ab11d982e58bee23e05c1780528a1 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad b3c661b15d5ab11d982e58bee23e05c1780528a1 Bisecting: 6 revisions left to test after this (roughly 3 steps) [a0fedc857dff457e123aeaa2039d28ac90e543df] Merge branch 'irq/for-block' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into for-4.21/block testing commit a0fedc857dff457e123aeaa2039d28ac90e543df with gcc (GCC) 8.1.0 all runs: OK # git bisect good a0fedc857dff457e123aeaa2039d28ac90e543df Bisecting: 3 revisions left to test after this (roughly 2 steps) [ff2c56609d9b1f0739ae3a3bfdb78191d01e4192] blk-mq: provide dummy blk_mq_map_queue_type() helper testing commit ff2c56609d9b1f0739ae3a3bfdb78191d01e4192 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad ff2c56609d9b1f0739ae3a3bfdb78191d01e4192 Bisecting: 0 revisions left to test after this (roughly 1 step) [ed76e329d74a4b15ac0f5fd3adbd52ec0178a134] blk-mq: abstract out queue map testing commit ed76e329d74a4b15ac0f5fd3adbd52ec0178a134 with gcc (GCC) 8.1.0 all runs: crashed: WARNING in __blk_mq_delay_run_hw_queue # git bisect bad ed76e329d74a4b15ac0f5fd3adbd52ec0178a134 Bisecting: 0 revisions left to test after this (roughly 0 steps) [a8908939af569ce2419f43fd56eeaf003bc3d85d] blk-mq: kill q->mq_map testing commit a8908939af569ce2419f43fd56eeaf003bc3d85d with gcc (GCC) 8.1.0 all runs: OK # git bisect good a8908939af569ce2419f43fd56eeaf003bc3d85d ed76e329d74a4b15ac0f5fd3adbd52ec0178a134 is the first bad commit commit ed76e329d74a4b15ac0f5fd3adbd52ec0178a134 Author: Jens Axboe Date: Mon Oct 29 13:06:14 2018 -0600 blk-mq: abstract out queue map This is in preparation for allowing multiple sets of maps per queue, if so desired. Reviewed-by: Hannes Reinecke Reviewed-by: Bart Van Assche Reviewed-by: Keith Busch Signed-off-by: Jens Axboe :040000 040000 68fa4cfd44f843990d9bd7da369b2ddb4960d977 9443e15faf10bb9097414c6367f978c176690dc7 M block :040000 040000 a9eb902652eb99e7ae28f2a3ecab5b2bd3f8a9e4 7098c3be6f338d0021625db5b664ff0af034a0d3 M drivers :040000 040000 d3656fb18f2e910bb2afd203c568ca49d7e0264c d45ac928e9643cf11de316929f170afe6b18a8a5 M include revisions tested: 20, total time: 3h56m30.573345057s (build: 1h49m3.8195792s, test: 2h1m25.842369659s) first bad commit: ed76e329d74a4b15ac0f5fd3adbd52ec0178a134 blk-mq: abstract out queue map cc: ["axboe@kernel.dk" "bvanassche@acm.org" "hare@suse.com" "keith.busch@intel.com"] crash: WARNING in __blk_mq_delay_run_hw_queue block nbd3: Receive control failed (result -22) block nbd2: Receive control failed (result -22) block nbd1: Send control failed (result -89) block nbd1: Request send failed, requeueing block nbd1: Receive control failed (result -32) WARNING: CPU: 0 PID: 2435 at include/linux/cpumask.h:121 blk_mq_hctx_next_cpu block/blk-mq.c:1331 [inline] WARNING: CPU: 0 PID: 2435 at include/linux/cpumask.h:121 __blk_mq_delay_run_hw_queue+0x349/0x4e0 block/blk-mq.c:1382 block nbd0: Receive control failed (result -22) Kernel panic - not syncing: panic_on_warn set ... CPU: 0 PID: 2435 Comm: kworker/0:1H Not tainted 4.20.0-rc1+ #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: kblockd blk_mq_requeue_work Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x16b/0x224 lib/dump_stack.c:113 panic+0x212/0x3b7 kernel/panic.c:188 kobject: 'nbd0' (00000000315fdc87): fill_kobj_path: path = '/devices/virtual/block/nbd0' __warn.cold.8+0x1b/0x3c kernel/panic.c:540 report_bug+0x1a4/0x200 lib/bug.c:186 fixup_bug arch/x86/kernel/traps.c:178 [inline] do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:271 do_invalid_op+0x36/0x40 arch/x86/kernel/traps.c:290 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:969 RIP: 0010:cpu_max_bits_warn include/linux/cpumask.h:121 [inline] RIP: 0010:cpumask_check include/linux/cpumask.h:128 [inline] RIP: 0010:cpumask_test_cpu include/linux/cpumask.h:344 [inline] RIP: 0010:blk_mq_hctx_next_cpu block/blk-mq.c:1346 [inline] RIP: 0010:__blk_mq_delay_run_hw_queue+0x349/0x4e0 block/blk-mq.c:1382 Code: 38 d0 7c 08 84 d2 0f 85 a0 01 00 00 c7 83 24 01 00 00 01 00 00 00 b8 40 00 00 00 e9 17 ff ff ff b8 40 00 00 00 e9 0d ff ff ff <0f> 0b e9 10 fe ff ff 48 8b 75 c8 48 c7 c2 e0 b2 af 88 bf ff ff ff RSP: 0018:ffff8800a3597b50 EFLAGS: 00010212 RAX: 0000000000000040 RBX: ffff8800964eacc0 RCX: 1ffff10012c9d5bb RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8800964eadd8 RBP: ffff8800a3597b90 R08: 0000000000000007 R09: 0000000000000000 R10: 0000000000000050 R11: ffff8800a3586180 R12: 0000000000000040 R13: ffff8800964eade4 R14: ffff8800964eade0 R15: 0000000000000000 blk_mq_run_hw_queue+0x148/0x2f0 block/blk-mq.c:1411 blk_mq_run_hw_queues+0xc5/0x140 block/blk-mq.c:1428 blk_mq_requeue_work+0x410/0x700 block/blk-mq.c:727 process_one_work+0x830/0x1670 kernel/workqueue.c:2153 worker_thread+0x85/0xb60 kernel/workqueue.c:2296 kthread+0x324/0x3e0 kernel/kthread.c:246 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 Kernel Offset: disabled Rebooting in 86400 seconds..