bisecting cause commit starting from 3cd52c1e32fe7dfee09815ced702db9ee9f84ec9 building syzkaller on 6a81331a1d4c744da9204d02ec88d558f7eea9c9 testing commit 3cd52c1e32fe7dfee09815ced702db9ee9f84ec9 with gcc (GCC) 10.2.1 20210217 kernel signature: 40387863cc17e70a9dc0c66c4e29acde32129991bfd761c44adb98ab2c1087c6 all runs: crashed: divide error in __tcp_select_window testing release v5.11 testing commit f40ddce88593482919761f74910f42f4b84c004b with gcc (GCC) 10.2.1 20210217 kernel signature: 0f3c1837f4dfe64969c4bc941acbd990cc700391e4774b889e5ff798e68c350c run #0: basic kernel testing failed: BUG: program execution failed: executor 0: failed to write control pipe: write |1: broken pipe run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect start 3cd52c1e32fe7dfee09815ced702db9ee9f84ec9 f40ddce88593482919761f74910f42f4b84c004b Bisecting: 7279 revisions left to test after this (roughly 13 steps) [d99676af540c2dc829999928fb81c58c80a1dce4] Merge tag 'drm-next-2021-02-19' of git://anongit.freedesktop.org/drm/drm testing commit d99676af540c2dc829999928fb81c58c80a1dce4 with gcc (GCC) 10.2.1 20210217 kernel signature: f54c0a1422d049199285718662bca4c29b30196cf70be135e68dba1db7c77269 run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: crashed: WARNING in kvm_wait run #4: OK run #5: OK run #6: OK run #7: OK run #8: boot failed: WARNING in kvm_wait run #9: boot failed: WARNING in kvm_wait # git bisect bad d99676af540c2dc829999928fb81c58c80a1dce4 Bisecting: 3717 revisions left to test after this (roughly 12 steps) [f9d58de23152f2c16f326d7e014cfa2933b00304] Merge tag 'affs-for-5.12-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux testing commit f9d58de23152f2c16f326d7e014cfa2933b00304 with gcc (GCC) 10.2.1 20210217 kernel signature: c2a1f40d9b4e66b7f256ddf36469510ac8851efd1dcfc40e956082586074ded4 all runs: OK # git bisect good f9d58de23152f2c16f326d7e014cfa2933b00304 Bisecting: 1854 revisions left to test after this (roughly 11 steps) [de1617578849acab8e16c9ffdce39b91fb50639d] Merge tag 'media/v5.12-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media testing commit de1617578849acab8e16c9ffdce39b91fb50639d with gcc (GCC) 10.2.1 20210217 kernel signature: 9ca65779ad27903475cc5fc958b0f043c31b5b95472e1bc82c70a0165483e5ff run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: boot failed: WARNING in kvm_wait reproducer seems to be flaky # git bisect bad de1617578849acab8e16c9ffdce39b91fb50639d Bisecting: 929 revisions left to test after this (roughly 10 steps) [4a037ad5d115b2cc79a5071a7854475f365476fa] Merge tag 'for-linus-5.12-rc1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip testing commit 4a037ad5d115b2cc79a5071a7854475f365476fa with gcc (GCC) 10.2.1 20210217 kernel signature: bd9b229895ae6da8a5c6155f9f296291bf26abf43715b503ef586c5cfc2eba56 run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: crashed: WARNING in kvm_wait run #4: crashed: WARNING in kvm_wait run #5: crashed: WARNING in kvm_wait run #6: crashed: WARNING in kvm_wait run #7: crashed: WARNING in kvm_wait run #8: crashed: WARNING in kvm_wait run #9: OK run #10: OK run #11: OK run #12: OK run #13: crashed: WARNING in kvm_wait run #14: OK run #15: OK run #16: boot failed: WARNING in kvm_wait run #17: boot failed: WARNING in kvm_wait run #18: boot failed: WARNING in kvm_wait run #19: boot failed: WARNING in kvm_wait # git bisect bad 4a037ad5d115b2cc79a5071a7854475f365476fa Bisecting: 451 revisions left to test after this (roughly 9 steps) [582cd91f69de8e44857cb610ebca661dac8656b7] Merge tag 'for-5.12/block-2021-02-17' of git://git.kernel.dk/linux-block testing commit 582cd91f69de8e44857cb610ebca661dac8656b7 with gcc (GCC) 10.2.1 20210217 kernel signature: 0b267425421ace61f9c20ce2d978aa9f1e72e37b32ca4af88927307eafcbc1c5 all runs: OK # git bisect good 582cd91f69de8e44857cb610ebca661dac8656b7 Bisecting: 247 revisions left to test after this (roughly 8 steps) [3f6ec19f2d05d800bbc42d95dece433da7697864] Merge tag 'timers-core-2021-02-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit 3f6ec19f2d05d800bbc42d95dece433da7697864 with gcc (GCC) 10.2.1 20210217 kernel signature: f71826e2c3fa2a5f8fd6778a299b1c332ec4740339e5fedc67ae9f672a3dca24 all runs: OK # git bisect good 3f6ec19f2d05d800bbc42d95dece433da7697864 Bisecting: 125 revisions left to test after this (roughly 7 steps) [85e853c5ec8486117182baab10c98b321daa6d47] Merge branch 'for-mingo-rcu' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu testing commit 85e853c5ec8486117182baab10c98b321daa6d47 with gcc (GCC) 10.2.1 20210217 kernel signature: 453bc83e99d3fee5232331818c91fabe79025b7af3c30aa16eb8e1e4d5e11042 all runs: OK # git bisect good 85e853c5ec8486117182baab10c98b321daa6d47 Bisecting: 73 revisions left to test after this (roughly 6 steps) [c5e6fc08feb2b88dc5dac2f3c817e1c2a4cafda4] sched,x86: Allow !PREEMPT_DYNAMIC testing commit c5e6fc08feb2b88dc5dac2f3c817e1c2a4cafda4 with gcc (GCC) 10.2.1 20210217 kernel signature: 351ba92e3c1e57b050532824ddfe7f8703fb92d5ef2bb313a8f1291c5089f28d all runs: OK # git bisect good c5e6fc08feb2b88dc5dac2f3c817e1c2a4cafda4 Bisecting: 33 revisions left to test after this (roughly 5 steps) [9eef02334505411667a7b51a8f349f8c6c4f3b66] Merge tag 'locking-core-2021-02-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit 9eef02334505411667a7b51a8f349f8c6c4f3b66 with gcc (GCC) 10.2.1 20210217 kernel signature: acc3d22a3259ecc5da54baeae76aeb496266c4d7d27b2392fc1f0c4a206e5224 run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: crashed: WARNING in kvm_wait run #4: crashed: WARNING in kvm_wait run #5: crashed: WARNING in kvm_wait run #6: crashed: WARNING in kvm_wait run #7: crashed: WARNING in kvm_wait run #8: crashed: WARNING in kvm_wait run #9: crashed: WARNING in kvm_wait run #10: crashed: WARNING in kvm_wait run #11: OK run #12: crashed: WARNING in kvm_wait run #13: OK run #14: OK run #15: OK run #16: crashed: WARNING in kvm_wait run #17: OK run #18: boot failed: WARNING in kvm_wait run #19: boot failed: WARNING in kvm_wait # git bisect bad 9eef02334505411667a7b51a8f349f8c6c4f3b66 Bisecting: 19 revisions left to test after this (roughly 4 steps) [62137364e3e8afcc745846c5c67cacf943149073] Merge branch 'linus' into locking/core, to pick up upstream fixes testing commit 62137364e3e8afcc745846c5c67cacf943149073 with gcc (GCC) 10.2.1 20210217 kernel signature: 4d830c0c5bdd245f8a956306162c12ed750e680b1c050b389512331989e9f835 run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: crashed: WARNING in kvm_wait run #4: crashed: WARNING in kvm_wait run #5: crashed: WARNING in kvm_wait run #6: crashed: WARNING in kvm_wait run #7: crashed: WARNING in kvm_wait run #8: crashed: WARNING in kvm_wait run #9: crashed: WARNING in kvm_wait run #10: crashed: WARNING in kvm_wait run #11: crashed: WARNING in kvm_wait run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: boot failed: WARNING in kvm_wait run #19: boot failed: WARNING in kvm_wait # git bisect bad 62137364e3e8afcc745846c5c67cacf943149073 Bisecting: 9 revisions left to test after this (roughly 3 steps) [997acaf6b4b59c6a9c259740312a69ea549cc684] lockdep: report broken irq restoration testing commit 997acaf6b4b59c6a9c259740312a69ea549cc684 with gcc (GCC) 10.2.1 20210217 kernel signature: 9c77a7501bcb690507ec6130c65ef312892734e7414471c506d27784d9b60b4b run #0: crashed: WARNING in kvm_wait run #1: crashed: WARNING in kvm_wait run #2: crashed: WARNING in kvm_wait run #3: crashed: WARNING in kvm_wait run #4: crashed: WARNING in kvm_wait run #5: crashed: WARNING in kvm_wait run #6: crashed: WARNING in kvm_wait run #7: crashed: WARNING in kvm_wait run #8: crashed: WARNING in kvm_wait run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: crashed: WARNING in corrupted run #15: OK run #16: OK run #17: OK run #18: OK run #19: boot failed: WARNING in kvm_wait # git bisect bad 997acaf6b4b59c6a9c259740312a69ea549cc684 Bisecting: 4 revisions left to test after this (roughly 2 steps) [175b1a60e8805617d74aefe17ce0d3a32eceb55c] locking/lockdep: Clean up check_redundant() a bit testing commit 175b1a60e8805617d74aefe17ce0d3a32eceb55c with gcc (GCC) 10.2.1 20210217 kernel signature: 35de4eb55759b0040422a8f2f3c7b99784c05823216135d1d42efda391cd67c5 all runs: OK # git bisect good 175b1a60e8805617d74aefe17ce0d3a32eceb55c Bisecting: 2 revisions left to test after this (roughly 1 step) [7e923e6a3ceb877497dd9ee70d71fa33b94f332b] locking/selftests: Add local_lock inversion tests testing commit 7e923e6a3ceb877497dd9ee70d71fa33b94f332b with gcc (GCC) 10.2.1 20210217 kernel signature: 94aba241cea574bb1c828060613e2038bb278f412d2b7c102dadd89619d2f3dd all runs: OK # git bisect good 7e923e6a3ceb877497dd9ee70d71fa33b94f332b Bisecting: 0 revisions left to test after this (roughly 1 step) [2f0df49c89acaa58571d509830bc481250699885] jump_label: Do not profile branch annotations testing commit 2f0df49c89acaa58571d509830bc481250699885 with gcc (GCC) 10.2.1 20210217 kernel signature: 94aba241cea574bb1c828060613e2038bb278f412d2b7c102dadd89619d2f3dd all runs: OK # git bisect good 2f0df49c89acaa58571d509830bc481250699885 997acaf6b4b59c6a9c259740312a69ea549cc684 is the first bad commit commit 997acaf6b4b59c6a9c259740312a69ea549cc684 Author: Mark Rutland Date: Mon Jan 11 15:37:07 2021 +0000 lockdep: report broken irq restoration We generally expect local_irq_save() and local_irq_restore() to be paired and sanely nested, and so local_irq_restore() expects to be called with irqs disabled. Thus, within local_irq_restore() we only trace irq flag changes when unmasking irqs. This means that a sequence such as: | local_irq_disable(); | local_irq_save(flags); | local_irq_enable(); | local_irq_restore(flags); ... is liable to break things, as the local_irq_restore() would mask irqs without tracing this change. Similar problems may exist for architectures whose arch_irq_restore() function depends on being called with irqs disabled. We don't consider such sequences to be a good idea, so let's define those as forbidden, and add tooling to detect such broken cases. This patch adds debug code to WARN() when raw_local_irq_restore() is called with irqs enabled. As raw_local_irq_restore() is expected to pair with raw_local_irq_save(), it should never be called with irqs enabled. To avoid the possibility of circular header dependencies between irqflags.h and bug.h, the warning is handled in a separate C file. The new code is all conditional on a new CONFIG_DEBUG_IRQFLAGS symbol which is independent of CONFIG_TRACE_IRQFLAGS. As noted above such cases will confuse lockdep, so CONFIG_DEBUG_LOCKDEP now selects CONFIG_DEBUG_IRQFLAGS. Signed-off-by: Mark Rutland Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20210111153707.10071-1-mark.rutland@arm.com include/linux/irqflags.h | 12 ++++++++++++ kernel/locking/Makefile | 1 + kernel/locking/irqflag-debug.c | 11 +++++++++++ lib/Kconfig.debug | 8 ++++++++ 4 files changed, 32 insertions(+) create mode 100644 kernel/locking/irqflag-debug.c culprit signature: 9c77a7501bcb690507ec6130c65ef312892734e7414471c506d27784d9b60b4b parent signature: 94aba241cea574bb1c828060613e2038bb278f412d2b7c102dadd89619d2f3dd Reproducer flagged being flaky revisions tested: 16, total time: 4h24m57.72000912s (build: 1h50m26.75249432s, test: 2h32m1.025092683s) first bad commit: 997acaf6b4b59c6a9c259740312a69ea549cc684 lockdep: report broken irq restoration recipients (to): ["linux-kernel@vger.kernel.org" "mark.rutland@arm.com" "mingo@redhat.com" "peterz@infradead.org" "peterz@infradead.org" "will@kernel.org"] recipients (cc): ["akpm@linux-foundation.org" "masahiroy@kernel.org" "rafael.j.wysocki@intel.com" "rostedt@goodmis.org" "tglx@linutronix.de"] crash: WARNING in corrupted ------------[ cut here ]------------ raw_local_irq_restore() called with IRQs enabled WARNING: CPU: 0 PID: 3503 at kernel/locking/irqflag-debug.c:9 warn_bogus_irq_restore kernel/locking/irqflag-debug.c:9 [inline] WARNING: CPU: 0 PID: 3503 at kernel/locking/irqflag-debug.c:9 warn_bogus_irq_restore+0x1d/0x20 kernel/locking/irqflag-debug.c:7 Modules linked in: CPU: 0 PID: 3503 Comm: syz-executor.4 Not tainted 5.11.0-rc2-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:warn_bogus_irq_restore kernel/locking/irqflag-debug.c:9 [inline] RIP: 0010:warn_bogus_irq_restore+0x1d/0x20 kernel/locking/irqflag-debug.c:7 Code: 51 00 e9 3f fe ff ff cc cc cc cc cc cc 80 3d e0 b4 ce 0a 00 74 01 c3 48 c7 c7 60 f5 8a 88 c6 05 cf b4 ce 0a 01 e8 17 01 a4 06 <0f> 0b c3 48 c7 c0 a0 46 4d 8e 53 48 89 fb 48 ba 00 00 00 00 00 fc RSP: 0018:ffffc900034ff9e0 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff88802165a040 RCX: 0000000000000000 RDX: 0000000000000002 RSI: ffffffff88ddbd00 RDI: fffff5200069ff2e RBP: 0000000000000246 R08: 0000000000000001 R09: ffff8880b9e30827 R10: ffffed10173c6104 R11: 0000000000000001 R12: 0000000000000003 R13: ffffed10042cb408 R14: 0000000000000001 R15: ffff8880b9e359c0 FS: 000000000337d400(0000) GS:ffff8880b9e00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000004e4a61 CR3: 0000