bisecting cause commit starting from c49243e8898233de18edfaaa5b7b261ea457f221 building syzkaller on a6c52263888c2ac5e9c267ec8f1d77664649536e testing commit c49243e8898233de18edfaaa5b7b261ea457f221 with gcc (GCC) 8.1.0 kernel signature: 7eb0fa8a53518aa3e015880573a71e35ef4bfb51b31bd55a4213543f4e270a16 all runs: crashed: kernel BUG at net/core/dev.c:NUM! testing release v5.10 testing commit 2c85ebc57b3e1817b6ce1a6b703928e113a90442 with gcc (GCC) 8.1.0 kernel signature: 395a614079c9030036488f72c030c4708fae86b9aba956ca15d15003efd3a593 all runs: OK # git bisect start c49243e8898233de18edfaaa5b7b261ea457f221 2c85ebc57b3e1817b6ce1a6b703928e113a90442 Bisecting: 6644 revisions left to test after this (roughly 13 steps) [3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9] Merge tag 'staging-5.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging testing commit 3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9 with gcc (GCC) 8.1.0 kernel signature: 6c9809d6712e897dccefdc6b430d7d43d2f2bd116c2c5d44d2bc23ac679aef61 all runs: OK # git bisect good 3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9 Bisecting: 3044 revisions left to test after this (roughly 12 steps) [9805529ec544ea7a82d891d5239a8ebd3dbb2a3e] Merge tag 'arm-soc-dt-5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc testing commit 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e with gcc (GCC) 8.1.0 kernel signature: c63f876bd71d42da5e2474fe33a7bc3455dc565406fb77305f5ade4292ce9150 all runs: OK # git bisect good 9805529ec544ea7a82d891d5239a8ebd3dbb2a3e Bisecting: 1509 revisions left to test after this (roughly 11 steps) [d56154c7e8ba090126a5a2cb76098628bc2216a2] Merge tag 'pwm/for-5.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/thierry.reding/linux-pwm testing commit d56154c7e8ba090126a5a2cb76098628bc2216a2 with gcc (GCC) 8.1.0 kernel signature: a413f42c6944ac7fc146865abb1fa011acdd4694fec00a67b311e8a15f99e6db all runs: OK # git bisect good d56154c7e8ba090126a5a2cb76098628bc2216a2 Bisecting: 745 revisions left to test after this (roughly 10 steps) [7b95f0563ab5a8f75195cdd4b2c3325c0c1df319] Merge tag 'kbuild-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild testing commit 7b95f0563ab5a8f75195cdd4b2c3325c0c1df319 with gcc (GCC) 8.1.0 kernel signature: 5aa46be2cc24612e6d09fac5db86d0df74a62aacd67e5c9e2ffed575a3a26bdb all runs: OK # git bisect good 7b95f0563ab5a8f75195cdd4b2c3325c0c1df319 Bisecting: 372 revisions left to test after this (roughly 9 steps) [5814bc2d4cc241c1a603fac2b5bf1bd4daa108fc] Merge tag 'perf-tools-2020-12-24' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux testing commit 5814bc2d4cc241c1a603fac2b5bf1bd4daa108fc with gcc (GCC) 8.1.0 kernel signature: bba3e4bf9cb102aa36265394437acb7b8bbbdbb2a1cd49b00a0960980538747b all runs: OK # git bisect good 5814bc2d4cc241c1a603fac2b5bf1bd4daa108fc Bisecting: 150 revisions left to test after this (roughly 8 steps) [aa35e45cd42aa249562c65e440c8d69fb84945d9] Merge tag 'net-5.11-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit aa35e45cd42aa249562c65e440c8d69fb84945d9 with gcc (GCC) 8.1.0 kernel signature: 91c055f07f410c3f608ae766e7449abd25257b102a19a29148b4060393c3dafc all runs: OK # git bisect good aa35e45cd42aa249562c65e440c8d69fb84945d9 Bisecting: 77 revisions left to test after this (roughly 6 steps) [3545454c7801e391b0d966f82c98614d45394770] net: dsa: lantiq_gswip: Exclude RMII from modes that report 1 GbE testing commit 3545454c7801e391b0d966f82c98614d45394770 with gcc (GCC) 8.1.0 kernel signature: 56397ea6ea1667a12f077cf4e75aa7af703ec19b95464b7cc766f4f91b0c68c2 all runs: OK # git bisect good 3545454c7801e391b0d966f82c98614d45394770 Bisecting: 38 revisions left to test after this (roughly 5 steps) [a1a7b4f32433e91f0fff32cde534eadc67242298] Merge tag 'regulator-fix-v5.11-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator testing commit a1a7b4f32433e91f0fff32cde534eadc67242298 with gcc (GCC) 8.1.0 kernel signature: 5eebb9879a4d0b1d55930cfc869b55574cd67217f2f0a0c31f98b0254fccfbb6 all runs: OK # git bisect good a1a7b4f32433e91f0fff32cde534eadc67242298 Bisecting: 20 revisions left to test after this (roughly 4 steps) [ea1c87c156d94dd78b4f5267ec40c403b2da7e14] Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 testing commit ea1c87c156d94dd78b4f5267ec40c403b2da7e14 with gcc (GCC) 8.1.0 kernel signature: 3aec7f847a59d28882d429693f3ecec0aa4925d847684d4d424a505e1baea1ad all runs: OK # git bisect good ea1c87c156d94dd78b4f5267ec40c403b2da7e14 Bisecting: 10 revisions left to test after this (roughly 3 steps) [7a6eb072a9548492ead086f3e820e9aac71c7138] net/mlx5e: Fix two double free cases testing commit 7a6eb072a9548492ead086f3e820e9aac71c7138 with gcc (GCC) 8.1.0 kernel signature: 5b9c9f28596b0bfef1f872beb4ba4e922bb1cde7d5d0068c4ff5062c758e3161 all runs: OK # git bisect good 7a6eb072a9548492ead086f3e820e9aac71c7138 Bisecting: 5 revisions left to test after this (roughly 3 steps) [53475c5dd856212e91538a9501162e821cc1f791] net: fix use-after-free when UDP GRO with shared fraglist testing commit 53475c5dd856212e91538a9501162e821cc1f791 with gcc (GCC) 8.1.0 kernel signature: 6a24b27aa00a7bc26539aacb38e5c390151fc2f66d345546c4eda264488c414b all runs: OK # git bisect good 53475c5dd856212e91538a9501162e821cc1f791 Bisecting: 2 revisions left to test after this (roughly 2 steps) [2b446e650b418f9a9e75f99852e2f2560cabfa17] docs: net: explain struct net_device lifetime testing commit 2b446e650b418f9a9e75f99852e2f2560cabfa17 with gcc (GCC) 8.1.0 kernel signature: eb71ab3068918f7a1e4d990adfcaf9acf39d08a342b82bd5a3753c2d4fc1f1da all runs: OK # git bisect good 2b446e650b418f9a9e75f99852e2f2560cabfa17 Bisecting: 0 revisions left to test after this (roughly 1 step) [766b0515d5bec4b780750773ed3009b148df8c0a] net: make sure devices go through netdev_wait_all_refs testing commit 766b0515d5bec4b780750773ed3009b148df8c0a with gcc (GCC) 8.1.0 kernel signature: 7eb0fa8a53518aa3e015880573a71e35ef4bfb51b31bd55a4213543f4e270a16 all runs: crashed: kernel BUG at net/core/dev.c:NUM! # git bisect bad 766b0515d5bec4b780750773ed3009b148df8c0a Bisecting: 0 revisions left to test after this (roughly 0 steps) [c269a24ce057abfc31130960e96ab197ef6ab196] net: make free_netdev() more lenient with unregistering devices testing commit c269a24ce057abfc31130960e96ab197ef6ab196 with gcc (GCC) 8.1.0 kernel signature: af39ea327280afc4e651ecff941f5ede3401b2e510205304438cf60b399ba168 all runs: crashed: kernel BUG at net/core/dev.c:NUM! # git bisect bad c269a24ce057abfc31130960e96ab197ef6ab196 c269a24ce057abfc31130960e96ab197ef6ab196 is the first bad commit commit c269a24ce057abfc31130960e96ab197ef6ab196 Author: Jakub Kicinski Date: Wed Jan 6 10:40:06 2021 -0800 net: make free_netdev() more lenient with unregistering devices There are two flavors of handling netdev registration: - ones called without holding rtnl_lock: register_netdev() and unregister_netdev(); and - those called with rtnl_lock held: register_netdevice() and unregister_netdevice(). While the semantics of the former are pretty clear, the same can't be said about the latter. The netdev_todo mechanism is utilized to perform some of the device unregistering tasks and it hooks into rtnl_unlock() so the locked variants can't actually finish the work. In general free_netdev() does not mix well with locked calls. Most drivers operating under rtnl_lock set dev->needs_free_netdev to true and expect core to make the free_netdev() call some time later. The part where this becomes most problematic is error paths. There is no way to unwind the state cleanly after a call to register_netdevice(), since unreg can't be performed fully without dropping locks. Make free_netdev() more lenient, and defer the freeing if device is being unregistered. This allows error paths to simply call free_netdev() both after register_netdevice() failed, and after a call to unregister_netdevice() but before dropping rtnl_lock. Simplify the error paths which are currently doing gymnastics around free_netdev() handling. Signed-off-by: Jakub Kicinski net/8021q/vlan.c | 4 +--- net/core/dev.c | 11 +++++++++++ net/core/rtnetlink.c | 23 ++++++----------------- 3 files changed, 18 insertions(+), 20 deletions(-) culprit signature: af39ea327280afc4e651ecff941f5ede3401b2e510205304438cf60b399ba168 parent signature: eb71ab3068918f7a1e4d990adfcaf9acf39d08a342b82bd5a3753c2d4fc1f1da revisions tested: 16, total time: 3h19m24.553211056s (build: 1h15m56.422331953s, test: 2h2m1.642577251s) first bad commit: c269a24ce057abfc31130960e96ab197ef6ab196 net: make free_netdev() more lenient with unregistering devices recipients (to): ["davem@davemloft.net" "kuba@kernel.org" "kuba@kernel.org" "netdev@vger.kernel.org"] recipients (cc): ["andriin@fb.com" "edumazet@google.com" "f.fainelli@gmail.com" "linux-kernel@vger.kernel.org" "roopa@cumulusnetworks.com"] crash: kernel BUG at net/core/dev.c:NUM! ------------[ cut here ]------------ kernel BUG at net/core/dev.c:10667! invalid opcode: 0000 [#1] PREEMPT SMP CPU: 0 PID: 10201 Comm: syz-executor.1 Not tainted 5.11.0-rc2-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:free_netdev+0x157/0x190 net/core/dev.c:10667 Code: 9f 85 01 00 85 c0 74 24 5b c6 85 84 05 00 00 01 5d 41 5c c3 0f b7 85 32 02 00 00 48 89 ef 5b 5d 41 5c 48 29 c7 e9 e9 42 72 fe <0f> 0b 80 3d a6 cf 6a 02 00 75 d3 ba 90 29 00 00 48 c7 c6 97 d0 98 RSP: 0018:ffffc90002e7b708 EFLAGS: 00010293 RAX: 0000000000000001 RBX: ffff88811f033ef8 RCX: 0000000000000001 RDX: 0000000080000001 RSI: ffffffff848f3d10 RDI: 0000000000000000 RBP: ffff88811f034000 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000001 R11: 0000000000000004 R12: ffff88811f034058 R13: ffffffff8548b560 R14: ffffc90002e7bbb0 R15: 0000000000000000 FS: 00007f0640c10700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000004e35b0 CR3: 000000011e7a9000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: __rtnl_newlink+0x835/0x890 net/core/rtnetlink.c:3447 rtnl_newlink+0x3e/0x60 net/core/rtnetlink.c:3491 rtnetlink_rcv_msg+0x173/0x480 net/core/rtnetlink.c:5553 netlink_rcv_skb+0x49/0xf0 net/netlink/af_netlink.c:2494 netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline] netlink_unicast+0x19a/0x270 net/netlink/af_netlink.c:1330 netlink_sendmsg+0x248/0x480 net/netlink/af_netlink.c:1919 sock_sendmsg_nosec net/socket.c:652 [inline] sock_sendmsg+0x2b/0x40 net/socket.c:672 ____sys_sendmsg+0x1ed/0x230 net/socket.c:2345 ___sys_sendmsg+0x77/0xb0 net/socket.c:2399 __sys_sendmsg+0x52/0xa0 net/socket.c:2432 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x45e219 Code: 0d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007f0640c0fc68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 000000000045e219 RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000003 RBP: 000000000119bfc0 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000119bf8c R13: 00007ffc9581f15f R14: 00007f0640c109c0 R15: 000000000119bf8c Modules linked in: ---[ end trace 3196ff58052153d5 ]--- RIP: 0010:free_netdev+0x157/0x190 net/core/dev.c:10667 Code: 9f 85 01 00 85 c0 74 24 5b c6 85 84 05 00 00 01 5d 41 5c c3 0f b7 85 32 02 00 00 48 89 ef 5b 5d 41 5c 48 29 c7 e9 e9 42 72 fe <0f> 0b 80 3d a6 cf 6a 02 00 75 d3 ba 90 29 00 00 48 c7 c6 97 d0 98 RSP: 0018:ffffc90002e7b708 EFLAGS: 00010293 RAX: 0000000000000001 RBX: ffff88811f033ef8 RCX: 0000000000000001 RDX: 0000000080000001 RSI: ffffffff848f3d10 RDI: 0000000000000000 RBP: ffff88811f034000 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000001 R11: 0000000000000004 R12: ffff88811f034058 R13: ffffffff8548b560 R14: ffffc90002e7bbb0 R15: 0000000000000000 FS: 00007f0640c10700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000004e35b0 CR3: 000000011e7a9000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400