bisecting cause commit starting from 03bd4773d898783fe3bc321287e4838e515fea92 building syzkaller on fc17ba4941e5e2cae9663b84e13627981c29d381 testing commit 03bd4773d898783fe3bc321287e4838e515fea92 with gcc (GCC) 8.1.0 all runs: crashed: general protection fault in ip6_tnl_ioctl testing release v5.3 testing commit 4d856f72c10ecb060868ed10ff1b1453943fc6c8 with gcc (GCC) 8.1.0 all runs: OK # git bisect start 03bd4773d898783fe3bc321287e4838e515fea92 v5.3 Bisecting: 5830 revisions left to test after this (roughly 13 steps) [81160dda9a7aad13c04e78bb2cfd3c4630e3afab] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next testing commit 81160dda9a7aad13c04e78bb2cfd3c4630e3afab with gcc (GCC) 8.1.0 all runs: OK # git bisect good 81160dda9a7aad13c04e78bb2cfd3c4630e3afab Bisecting: 2922 revisions left to test after this (roughly 12 steps) [8c2b418c3f95a488f5226870eee68574d323f0f8] Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux testing commit 8c2b418c3f95a488f5226870eee68574d323f0f8 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 8c2b418c3f95a488f5226870eee68574d323f0f8 Bisecting: 1461 revisions left to test after this (roughly 11 steps) [3e4d890a26d5411d0b64e5e8ecfdcdb435c1d3f8] modules: make MODULE_IMPORT_NS() work even when modular builds are disabled testing commit 3e4d890a26d5411d0b64e5e8ecfdcdb435c1d3f8 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 3e4d890a26d5411d0b64e5e8ecfdcdb435c1d3f8 Bisecting: 725 revisions left to test after this (roughly 10 steps) [cbafe18c71028d5e0ee1626b4776fea5d5824a78] Merge branch 'akpm' (patches from Andrew) testing commit cbafe18c71028d5e0ee1626b4776fea5d5824a78 with gcc (GCC) 8.1.0 all runs: OK # git bisect good cbafe18c71028d5e0ee1626b4776fea5d5824a78 Bisecting: 359 revisions left to test after this (roughly 9 steps) [e37e3bc7e265d05d00f14079767537699cf6bd46] Merge tag 'pwm/for-5.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/thierry.reding/linux-pwm testing commit e37e3bc7e265d05d00f14079767537699cf6bd46 with gcc (GCC) 8.1.0 run #0: crashed: WARNING: ODEBUG bug in netdev_freemem run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad e37e3bc7e265d05d00f14079767537699cf6bd46 Bisecting: 166 revisions left to test after this (roughly 8 steps) [a7b7b772bb4abaa4b2d9df67b50bf7208203da82] Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit a7b7b772bb4abaa4b2d9df67b50bf7208203da82 with gcc (GCC) 8.1.0 all runs: OK # git bisect good a7b7b772bb4abaa4b2d9df67b50bf7208203da82 Bisecting: 67 revisions left to test after this (roughly 6 steps) [7bccb9f10c8f36ee791769b531ed4d28f6379aae] Merge tag 'linux-watchdog-5.4-rc1' of git://www.linux-watchdog.org/linux-watchdog testing commit 7bccb9f10c8f36ee791769b531ed4d28f6379aae with gcc (GCC) 8.1.0 all runs: OK # git bisect good 7bccb9f10c8f36ee791769b531ed4d28f6379aae Bisecting: 33 revisions left to test after this (roughly 5 steps) [da635e7abe3f4ec9a8270ca4f5ba946d1a43f678] MAINTAINERS: Add myself as reviewer for the PWM subsystem testing commit da635e7abe3f4ec9a8270ca4f5ba946d1a43f678 with gcc (GCC) 8.1.0 all runs: boot failed: WARNING: workqueue cpumask: online intersect > possible intersect # git bisect skip da635e7abe3f4ec9a8270ca4f5ba946d1a43f678 Bisecting: 33 revisions left to test after this (roughly 5 steps) [c9675829ba4b0e95c613f6d6d83d2b5cb9c5371c] pwm: fsl-ftm: Don't update the state for the caller of pwm_apply_state() testing commit c9675829ba4b0e95c613f6d6d83d2b5cb9c5371c with gcc (GCC) 8.1.0 all runs: boot failed: WARNING: workqueue cpumask: online intersect > possible intersect # git bisect skip c9675829ba4b0e95c613f6d6d83d2b5cb9c5371c Bisecting: 33 revisions left to test after this (roughly 5 steps) [f6abac0379b8368519f28016c8c8821b8bd17f5e] pwm: sifive: Remove redundant error message testing commit f6abac0379b8368519f28016c8c8821b8bd17f5e with gcc (GCC) 8.1.0 all runs: boot failed: WARNING: workqueue cpumask: online intersect > possible intersect # git bisect skip f6abac0379b8368519f28016c8c8821b8bd17f5e Bisecting: 33 revisions left to test after this (roughly 5 steps) [c91e3234c6035baf5a79763cb4fcd5d23ce75c2b] pwm: stm32-lp: Add check in case requested period cannot be achieved testing commit c91e3234c6035baf5a79763cb4fcd5d23ce75c2b with gcc (GCC) 8.1.0 all runs: boot failed: WARNING: workqueue cpumask: online intersect > possible intersect # git bisect skip c91e3234c6035baf5a79763cb4fcd5d23ce75c2b Bisecting: 33 revisions left to test after this (roughly 5 steps) [b89f625e28d44552083f43752f62d8621ded0a04] block: don't release queue's sysfs lock during switching elevator testing commit b89f625e28d44552083f43752f62d8621ded0a04 with gcc (GCC) 8.1.0 run #0: crashed: WARNING: ODEBUG bug in netdev_freemem run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad b89f625e28d44552083f43752f62d8621ded0a04 Bisecting: 0 revisions left to test after this (roughly 0 steps) [284b94be1925dbe035ce5218d8b5c197321262c7] blk-mq: move lockdep_assert_held() into elevator_exit testing commit 284b94be1925dbe035ce5218d8b5c197321262c7 with gcc (GCC) 8.1.0 all runs: OK # git bisect good 284b94be1925dbe035ce5218d8b5c197321262c7 b89f625e28d44552083f43752f62d8621ded0a04 is the first bad commit commit b89f625e28d44552083f43752f62d8621ded0a04 Author: Ming Lei Date: Mon Sep 23 23:12:09 2019 +0800 block: don't release queue's sysfs lock during switching elevator cecf5d87ff20 ("block: split .sysfs_lock into two locks") starts to release & acquire sysfs_lock before registering/un-registering elevator queue during switching elevator for avoiding potential deadlock from showing & storing 'queue/iosched' attributes and removing elevator's kobject. Turns out there isn't such deadlock because 'q->sysfs_lock' isn't required in .show & .store of queue/iosched's attributes, and just elevator's sysfs lock is acquired in elv_iosched_store() and elv_iosched_show(). So it is safe to hold queue's sysfs lock when registering/un-registering elevator queue. The biggest issue is that commit cecf5d87ff20 assumes that concurrent write on 'queue/scheduler' can't happen. However, this assumption isn't true, because kernfs_fop_write() only guarantees that concurrent write aren't called on the same open file, but the write could be from different open on the file. So we can't release & re-acquire queue's sysfs lock during switching elevator, otherwise use-after-free on elevator could be triggered. Fixes the issue by not releasing queue's sysfs lock during switching elevator. Fixes: cecf5d87ff20 ("block: split .sysfs_lock into two locks") Cc: Christoph Hellwig Cc: Hannes Reinecke Cc: Greg KH Cc: Mike Snitzer Reviewed-by: Bart Van Assche Signed-off-by: Ming Lei Signed-off-by: Jens Axboe :040000 040000 597d8ae667e5dbe1bbe7d6fe2ee5c06d79edc269 7087493adb6bd53e9a7ca304499d539a85b090a9 M block revisions tested: 15, total time: 4h20m16.945622617s (build: 1h29m38.440629155s, test: 2h45m22.353435973s) first bad commit: b89f625e28d44552083f43752f62d8621ded0a04 block: don't release queue's sysfs lock during switching elevator cc: ["axboe@kernel.dk" "bvanassche@acm.org" "gregkh@linuxfoundation.org" "hare@suse.com" "hch@infradead.org" "ming.lei@redhat.com" "snitzer@redhat.com"] crash: WARNING: ODEBUG bug in netdev_freemem ------------[ cut here ]------------ ODEBUG: free active (active state 0) object type: timer_list hint: delayed_work_timer_fn+0x0/0x90 arch/x86/include/asm/paravirt.h:756 WARNING: CPU: 1 PID: 34 at lib/debugobjects.c:484 debug_print_object+0x168/0x210 lib/debugobjects.c:481 Kernel panic - not syncing: panic_on_warn set ... CPU: 1 PID: 34 Comm: kworker/u4:2 Not tainted 5.3.0+ #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: netns cleanup_net Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x113/0x167 lib/dump_stack.c:113 panic+0x223/0x4dc kernel/panic.c:219 __warn.cold.10+0x1b/0x37 kernel/panic.c:576 report_bug+0x1a4/0x200 lib/bug.c:186 fixup_bug arch/x86/kernel/traps.c:179 [inline] do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:272 do_invalid_op+0x36/0x40 arch/x86/kernel/traps.c:291 invalid_op+0x23/0x30 arch/x86/entry/entry_64.S:1028 RIP: 0010:debug_print_object+0x168/0x210 lib/debugobjects.c:481 Code: 63 87 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 92 00 00 00 48 8b 14 dd c0 ac 63 87 4c 89 fe 48 c7 c7 c0 a1 63 87 e8 3a 66 34 fe <0f> 0b 83 05 03 de 12 06 01 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f RSP: 0018:ffff8880a9aff850 EFLAGS: 00010086 RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffffffff89fe00e0 RBP: ffff8880a9aff890 R08: ffffed1015d64101 R09: ffffed1015d64101 R10: ffffed1015d64100 R11: ffff8880aeb20807 R12: 0000000000000001 R13: ffffffff885ae900 R14: ffffffff81596a70 R15: ffffffff8763a920 __debug_check_no_obj_freed lib/debugobjects.c:963 [inline] debug_check_no_obj_freed+0x2db/0x436 lib/debugobjects.c:994 kfree+0xf6/0x2c0 mm/slab.c:3755 kvfree+0x2c/0x30 mm/util.c:593 netdev_freemem+0x47/0x60 net/core/dev.c:9148 netdev_release+0x6c/0x90 net/core/net-sysfs.c:1635 device_release+0x6a/0x1c0 drivers/base/core.c:1100 kobject_cleanup lib/kobject.c:693 [inline] kobject_release lib/kobject.c:722 [inline] kref_put include/linux/kref.h:65 [inline] kobject_put.cold.8+0x229/0x27c lib/kobject.c:739 netdev_run_todo+0x459/0x6a0 net/core/dev.c:9053 rtnl_unlock+0x9/0x10 net/core/rtnetlink.c:112 default_device_exit_batch+0x2e9/0x3d0 net/core/dev.c:9836 ops_exit_list.isra.6+0xd3/0x120 net/core/net_namespace.c:175 cleanup_net+0x430/0x950 net/core/net_namespace.c:594 process_one_work+0x856/0x1630 kernel/workqueue.c:2269 worker_thread+0x85/0xb60 kernel/workqueue.c:2415 kthread+0x331/0x3f0 kernel/kthread.c:255 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 Kernel Offset: disabled Rebooting in 86400 seconds..