bisecting cause commit starting from 6992ca0dd017ebaa2bf8e9dcc49f1c3b7033b082 building syzkaller on 662cf49ae315772e243d80a1c87dcdee1a304196 testing commit 6992ca0dd017ebaa2bf8e9dcc49f1c3b7033b082 with gcc (GCC) 8.1.0 kernel signature: 33cb2909e07303ad2bec4275b8e3f25f2cfa6c32d9cdcddd11b015fd867cbac5 all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id testing release v5.5 testing commit d5226fa6dbae0569ee43ecfc08bdcd6770fc4755 with gcc (GCC) 8.1.0 kernel signature: 0bb226147777f6a201864b7837f3108dc95cd405fe637e84cceccaf0f8cd0e94 all runs: OK # git bisect start 6992ca0dd017ebaa2bf8e9dcc49f1c3b7033b082 d5226fa6dbae0569ee43ecfc08bdcd6770fc4755 Bisecting: 4529 revisions left to test after this (roughly 12 steps) [7ba31c3f2f1ee095d8126f4d3757fc3b2bc3c838] Merge tag 'staging-5.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging testing commit 7ba31c3f2f1ee095d8126f4d3757fc3b2bc3c838 with gcc (GCC) 8.1.0 kernel signature: ca158a04caca976373b2bd5ce400569e24a59ecd2aaaac7f67ef40b17fb1cb9b all runs: OK # git bisect good 7ba31c3f2f1ee095d8126f4d3757fc3b2bc3c838 Bisecting: 2499 revisions left to test after this (roughly 11 steps) [b45f1b3b585e195a7daead16d914e164310b1df6] Merge branch 'ttm-prot-fix' of git://people.freedesktop.org/~thomash/linux into drm-next testing commit b45f1b3b585e195a7daead16d914e164310b1df6 with gcc (GCC) 8.1.0 kernel signature: 22a32e510a5fa197f2caad0de3dd1148ec8ed486be6791d8fc17ba96f0536ce5 all runs: OK # git bisect good b45f1b3b585e195a7daead16d914e164310b1df6 Bisecting: 1217 revisions left to test after this (roughly 10 steps) [7eec11d3a784a283f916590e5aa30b855c2ccfd7] Merge branch 'akpm' (patches from Andrew) testing commit 7eec11d3a784a283f916590e5aa30b855c2ccfd7 with gcc (GCC) 8.1.0 kernel signature: 91c6cee0c3f52a691628ddf47229caa2a69a2b808213247c4b64e231fc2b1687 all runs: OK # git bisect good 7eec11d3a784a283f916590e5aa30b855c2ccfd7 Bisecting: 605 revisions left to test after this (roughly 9 steps) [e17ac02b18c61f0d5f85c6ec9e49f3ff00b2b3cd] Merge tag 'kgdb-5.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/danielt/linux testing commit e17ac02b18c61f0d5f85c6ec9e49f3ff00b2b3cd with gcc (GCC) 8.1.0 kernel signature: 937bb49909030457f46846e5ad2435722f080050277a8739d08d9ab95e708bf1 all runs: OK # git bisect good e17ac02b18c61f0d5f85c6ec9e49f3ff00b2b3cd Bisecting: 333 revisions left to test after this (roughly 8 steps) [9717c1cea16e3eae81ca226f4c3670bb799b61ad] Merge tag 'drm-next-2020-02-04' of git://anongit.freedesktop.org/drm/drm testing commit 9717c1cea16e3eae81ca226f4c3670bb799b61ad with gcc (GCC) 8.1.0 kernel signature: b6691d602cc75b5b3554dbb359a214996caa97da11a854c165f0a70f6303a404 all runs: OK # git bisect good 9717c1cea16e3eae81ca226f4c3670bb799b61ad Bisecting: 201 revisions left to test after this (roughly 7 steps) [4c25df5640ae6e4491ee2c50d3f70c1559ef037d] Merge branch 'topic/user-access-begin' into next testing commit 4c25df5640ae6e4491ee2c50d3f70c1559ef037d with gcc (GCC) 8.1.0 kernel signature: 54cae613169c85b82b4faa5e72ecacd1da3bc55ec74a639573eabd9c39b6b61e all runs: OK # git bisect good 4c25df5640ae6e4491ee2c50d3f70c1559ef037d Bisecting: 101 revisions left to test after this (roughly 7 steps) [a45ad71e8995eed2b95c6ef0f4c442da0c4f6677] Merge tag 'rproc-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/andersson/remoteproc testing commit a45ad71e8995eed2b95c6ef0f4c442da0c4f6677 with gcc (GCC) 8.1.0 kernel signature: 2c07f19ed7d758c52e6253205c5f2971ff6472c046002e83966f6ca4337c1ae9 all runs: OK # git bisect good a45ad71e8995eed2b95c6ef0f4c442da0c4f6677 Bisecting: 50 revisions left to test after this (roughly 6 steps) [0d0d9a388a858e271bb70e71e99e7fe2a6fd6f64] l2tp: Allow duplicate session creation with UDP testing commit 0d0d9a388a858e271bb70e71e99e7fe2a6fd6f64 with gcc (GCC) 8.1.0 kernel signature: 1921f3e56c90d878ddd1b49bc56bf1a467c79b5ba98c7007c186de7dff8a3966 all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id # git bisect bad 0d0d9a388a858e271bb70e71e99e7fe2a6fd6f64 Bisecting: 26 revisions left to test after this (roughly 5 steps) [83d0585f91da441a0b11bc5ff93f4cda56de6703] Merge branch 'Fix-reconnection-latency-caused-by-FIN-ACK-handling-race' testing commit 83d0585f91da441a0b11bc5ff93f4cda56de6703 with gcc (GCC) 8.1.0 kernel signature: 4abb760fdb8e855c66ab86ae6b8d1e89149c1a86f06f8111a107aa5f11be1108 all runs: OK # git bisect good 83d0585f91da441a0b11bc5ff93f4cda56de6703 Bisecting: 13 revisions left to test after this (roughly 4 steps) [6ab63366e1ec4ec1900f253aa64727b4b5f4ee73] netdevsim: disable devlink reload when resources are being used testing commit 6ab63366e1ec4ec1900f253aa64727b4b5f4ee73 with gcc (GCC) 8.1.0 kernel signature: c594f37c57449af1606573b11d0e5b087e3cf04a27d8f962975634780c1b88c8 all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id # git bisect bad 6ab63366e1ec4ec1900f253aa64727b4b5f4ee73 Bisecting: 6 revisions left to test after this (roughly 3 steps) [14b41a2959fbaa50932699d32ceefd6643abacc6] net: stmmac: Delete txtimer in suspend() testing commit 14b41a2959fbaa50932699d32ceefd6643abacc6 with gcc (GCC) 8.1.0 kernel signature: 28e60ca0460f8438f1f7b89b4884a4cc8a0d5e5b37cde819fadc5a091b56b7d5 all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id # git bisect bad 14b41a2959fbaa50932699d32ceefd6643abacc6 Bisecting: 2 revisions left to test after this (roughly 2 steps) [04d36d748fac349b068ef621611f454010054c58] rxrpc: Fix missing active use pinning of rxrpc_local object testing commit 04d36d748fac349b068ef621611f454010054c58 with gcc (GCC) 8.1.0 kernel signature: d86e714c5dce4f86df2464b3f202a716c87d7eb669f3a8997190aa5367b70d5a run #0: OK run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: boot failed: can't ssh into the instance # git bisect good 04d36d748fac349b068ef621611f454010054c58 Bisecting: 0 revisions left to test after this (roughly 1 step) [3d80c653f99658af6af3ac1b74f70bf9a069e71f] Merge tag 'rxrpc-fixes-20200203' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs testing commit 3d80c653f99658af6af3ac1b74f70bf9a069e71f with gcc (GCC) 8.1.0 kernel signature: fe4c7145733bd310bb03be2e6a46e9a33a4e16eae247dbd6a9a3e4a13a9aaf46 all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id # git bisect bad 3d80c653f99658af6af3ac1b74f70bf9a069e71f Bisecting: 0 revisions left to test after this (roughly 0 steps) [5273a191dca65a675dc0bcf3909e59c6933e2831] rxrpc: Fix NULL pointer deref due to call->conn being cleared on disconnect testing commit 5273a191dca65a675dc0bcf3909e59c6933e2831 with gcc (GCC) 8.1.0 kernel signature: aa64a845ea986d9aa9ddc3db3302b8f65bac107112b3cfd976f3a22e1315a54f all runs: crashed: inconsistent lock state in rxrpc_put_client_connection_id # git bisect bad 5273a191dca65a675dc0bcf3909e59c6933e2831 5273a191dca65a675dc0bcf3909e59c6933e2831 is the first bad commit commit 5273a191dca65a675dc0bcf3909e59c6933e2831 Author: David Howells Date: Thu Jan 30 21:50:36 2020 +0000 rxrpc: Fix NULL pointer deref due to call->conn being cleared on disconnect When a call is disconnected, the connection pointer from the call is cleared to make sure it isn't used again and to prevent further attempted transmission for the call. Unfortunately, there might be a daemon trying to use it at the same time to transmit a packet. Fix this by keeping call->conn set, but setting a flag on the call to indicate disconnection instead. Remove also the bits in the transmission functions where the conn pointer is checked and a ref taken under spinlock as this is now redundant. Fixes: 8d94aa381dab ("rxrpc: Calls shouldn't hold socket refs") Signed-off-by: David Howells net/rxrpc/ar-internal.h | 1 + net/rxrpc/call_object.c | 4 ++-- net/rxrpc/conn_client.c | 3 +-- net/rxrpc/conn_object.c | 4 ++-- net/rxrpc/output.c | 27 +++++++++------------------ 5 files changed, 15 insertions(+), 24 deletions(-) culprit signature: aa64a845ea986d9aa9ddc3db3302b8f65bac107112b3cfd976f3a22e1315a54f parent signature: d86e714c5dce4f86df2464b3f202a716c87d7eb669f3a8997190aa5367b70d5a revisions tested: 16, total time: 4h1m12.247976592s (build: 1h52m10.752672925s, test: 2h7m43.287058292s) first bad commit: 5273a191dca65a675dc0bcf3909e59c6933e2831 rxrpc: Fix NULL pointer deref due to call->conn being cleared on disconnect cc: ["davem@davemloft.net" "dhowells@redhat.com" "kuba@kernel.org" "linux-afs@lists.infradead.org" "linux-kernel@vger.kernel.org" "netdev@vger.kernel.org"] crash: inconsistent lock state in rxrpc_put_client_connection_id ================================ WARNING: inconsistent lock state 5.5.0-syzkaller #0 Not tainted -------------------------------- inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. syz-executor.3/8398 [HC0[0]:SC1[1]:HE1:SE0] takes: ffffffff89c01a18 (rxrpc_conn_id_lock){+.?.}, at: spin_lock include/linux/spinlock.h:338 [inline] ffffffff89c01a18 (rxrpc_conn_id_lock){+.?.}, at: rxrpc_put_client_connection_id+0x51/0xb0 net/rxrpc/conn_client.c:138 {SOFTIRQ-ON-W} state was registered at: lock_acquire+0x194/0x410 kernel/locking/lockdep.c:4484 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] rxrpc_get_client_connection_id net/rxrpc/conn_client.c:109 [inline] rxrpc_alloc_client_connection net/rxrpc/conn_client.c:193 [inline] rxrpc_get_client_conn net/rxrpc/conn_client.c:340 [inline] rxrpc_connect_call+0x7ae/0x4700 net/rxrpc/conn_client.c:701 rxrpc_new_client_call+0x82c/0x1670 net/rxrpc/call_object.c:290 rxrpc_new_client_call_for_sendmsg net/rxrpc/sendmsg.c:595 [inline] rxrpc_do_sendmsg+0x886/0x186d net/rxrpc/sendmsg.c:652 rxrpc_sendmsg+0x47e/0x5d0 net/rxrpc/af_rxrpc.c:586 sock_sendmsg_nosec net/socket.c:652 [inline] sock_sendmsg+0xb5/0xf0 net/socket.c:672 ____sys_sendmsg+0x3b0/0x950 net/socket.c:2343 ___sys_sendmsg+0xe4/0x160 net/socket.c:2397 __sys_sendmmsg+0x160/0x370 net/socket.c:2487 __do_sys_sendmmsg net/socket.c:2516 [inline] __se_sys_sendmmsg net/socket.c:2513 [inline] __x64_sys_sendmmsg+0x98/0x100 net/socket.c:2513 do_syscall_64+0xca/0x5f0 arch/x86/entry/common.c:294 entry_SYSCALL_64_after_hwframe+0x49/0xbe irq event stamp: 140490 hardirqs last enabled at (140490): [] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:160 [inline] hardirqs last enabled at (140490): [] _raw_spin_unlock_irqrestore+0x7d/0xd0 kernel/locking/spinlock.c:191 hardirqs last disabled at (140489): [] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline] hardirqs last disabled at (140489): [] _raw_spin_lock_irqsave+0x74/0xd0 kernel/locking/spinlock.c:159 softirqs last enabled at (140240): [] sock_net include/net/sock.h:2459 [inline] softirqs last enabled at (140240): [] unix_create1+0x425/0x500 net/unix/af_unix.c:811 softirqs last disabled at (140277): [] invoke_softirq kernel/softirq.c:373 [inline] softirqs last disabled at (140277): [] irq_exit+0x19a/0x1d0 kernel/softirq.c:413 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(rxrpc_conn_id_lock); lock(rxrpc_conn_id_lock); *** DEADLOCK *** 1 lock held by syz-executor.3/8398: #0: ffffffff88fa3780 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2176 [inline] #0: ffffffff88fa3780 (rcu_callback){....}, at: rcu_core+0x53e/0x1340 kernel/rcu/tree.c:2410 stack backtrace: CPU: 0 PID: 8398 Comm: syz-executor.3 Not tainted 5.5.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x12d/0x187 lib/dump_stack.c:118 print_usage_bug.cold.63+0x31b/0x36c kernel/locking/lockdep.c:3100 valid_state kernel/locking/lockdep.c:3111 [inline] mark_lock_irq kernel/locking/lockdep.c:3308 [inline] mark_lock+0xb2c/0x11d0 kernel/locking/lockdep.c:3665 mark_usage kernel/locking/lockdep.c:3565 [inline] __lock_acquire+0x1974/0x4ef0 kernel/locking/lockdep.c:3908 lock_acquire+0x194/0x410 kernel/locking/lockdep.c:4484 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] rxrpc_put_client_connection_id+0x51/0xb0 net/rxrpc/conn_client.c:138 rxrpc_put_one_client_conn net/rxrpc/conn_client.c:955 [inline] rxrpc_put_client_conn+0x1a0/0x8f0 net/rxrpc/conn_client.c:1001 rxrpc_put_connection net/rxrpc/ar-internal.h:965 [inline] rxrpc_rcu_destroy_call+0x9b/0x1a0 net/rxrpc/call_object.c:572 rcu_do_batch kernel/rcu/tree.c:2186 [inline] rcu_core+0x5bd/0x1340 kernel/rcu/tree.c:2410 rcu_core_si+0x9/0x10 kernel/rcu/tree.c:2419 __do_softirq+0x262/0x9a8 kernel/softirq.c:292 invoke_softirq kernel/softirq.c:373 [inline] irq_exit+0x19a/0x1d0 kernel/softirq.c:413 exiting_irq arch/x86/include/asm/apic.h:536 [inline] smp_apic_timer_interrupt+0x1a6/0x5f0 arch/x86/kernel/apic/apic.c:1137 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829 RIP: 0010:__might_fault+0xb6/0x1b0 mm/memory.c:4595 Code: e8 3f 34 b7 ff 48 b8 00 00 00 00 00 fc ff df 65 48 8b 1c 25 c0 1e 02 00 48 8d bb 28 04 00 00 48 89 fa 48 c1 ea 03 80 3c 02 00 <0f> 85 ca 00 00 00 48 83 bb 28 04 00 00 00 74 b8 65 48 8b 1c 25 c0 RSP: 0018:ffffc900072bfa58 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: dffffc0000000000 RBX: ffff888097d7a680 RCX: 0000000000000000 RDX: 1ffff11012faf555 RSI: 00000000ffffffff RDI: ffff888097d7aaa8 RBP: ffffc900072bfa70 R08: 0000000000000001 R09: ffff888097d7af10 R10: fffffbfff13ae938 R11: ffffffff89d749c7 R12: ffffffff87ed4ca0 R13: ffffc900072bfb60 R14: ffff8880980761c0 R15: ffffc900072bfb60 _copy_from_user+0x25/0x110 lib/usercopy.c:11 copy_from_user include/linux/uaccess.h:144 [inline] sock_do_ioctl+0x153/0x230 net/socket.c:1074 sock_ioctl+0x484/0x600 net/socket.c:1204 vfs_ioctl fs/ioctl.c:47 [inline] file_ioctl fs/ioctl.c:545 [inline] do_vfs_ioctl+0x196/0x1190 fs/ioctl.c:732 ksys_ioctl+0x62/0x90 fs/ioctl.c:749 __do_sys_ioctl fs/ioctl.c:756 [inline] __se_sys_ioctl fs/ioctl.c:754 [inline] __x64_sys_ioctl+0x6e/0xb0 fs/ioctl.c:754 do_syscall_64+0xca/0x5f0 arch/x86/entry/common.c:294 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x45b207 Code: 48 83 c4 08 48 89 d8 5b 5d c3 66 0f 1f 84 00 00 00 00 00 48 89 e8 48 f7 d8 48 39 c3 0f 92 c0 eb 92 66 90 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 0f 83 0d b8 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007ffcfe9a3de8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 000000000045b207 RDX: 00007ffcfe9a3df0 RSI: 0000000000008933 RDI: 0000000000000004 RBP: 00000000004c08c7 R08: 0000000000000005 R09: 000000000000001e R10: 0000000000000004 R11: 0000000000000246 R12: 0000000000000003 R13: 0000000000000000 R14: 0000000000000000 R15: 00007ffcfe9a4370