bisecting cause commit starting from f5499c67477eb640e794428da0502c5e4c723119 building syzkaller on 0ea7a8875d3f20d3bdf6323495521b62ef71878c testing commit f5499c67477eb640e794428da0502c5e4c723119 with gcc (GCC) 8.1.0 kernel signature: e10cd8e14f0a4bf6e0be0f63601f8caa0e5e575a4845d836db9a078941f53bb4 all runs: crashed: kernel BUG at net/rxrpc/conn_object.c:LINE! testing release v5.8 testing commit bcf876870b95592b52519ed4aafcf9d95999bc9c with gcc (GCC) 8.1.0 kernel signature: 127c0ca33d5176d534e86a9f6f5274dd346d4667ef9d47387b7b1facae2ab484 all runs: OK # git bisect start f5499c67477eb640e794428da0502c5e4c723119 bcf876870b95592b52519ed4aafcf9d95999bc9c Bisecting: 7073 revisions left to test after this (roughly 13 steps) [47ec5303d73ea344e84f46660fff693c57641386] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next testing commit 47ec5303d73ea344e84f46660fff693c57641386 with gcc (GCC) 8.1.0 kernel signature: b9390d600267a843a663de79d8cccbf4a0d1096aa910bca0d6029c4c2435d172 all runs: OK # git bisect good 47ec5303d73ea344e84f46660fff693c57641386 Bisecting: 3557 revisions left to test after this (roughly 12 steps) [05487215e6b9732cc4ad0e83e465b33182200ad5] KVM: x86: Don't attempt to load PDPTRs when 64-bit mode is enabled testing commit 05487215e6b9732cc4ad0e83e465b33182200ad5 with gcc (GCC) 8.1.0 kernel signature: 8701d34c5a4518c4d8643b272e2bfb9b768d091bc2b971c19c6aa5b9e59b74a9 all runs: OK # git bisect good 05487215e6b9732cc4ad0e83e465b33182200ad5 Bisecting: 1789 revisions left to test after this (roughly 11 steps) [6ffdcde4ee9a20beda096dec664da89002610d7d] Merge tag 'edac_updates_for_5.9_pt2' of git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras testing commit 6ffdcde4ee9a20beda096dec664da89002610d7d with gcc (GCC) 8.1.0 kernel signature: 1319106487f2a6ba0ed0446d0ddf1208ac430a9475fd6ace57ec1f51111ae4ba all runs: OK # git bisect good 6ffdcde4ee9a20beda096dec664da89002610d7d Bisecting: 887 revisions left to test after this (roughly 10 steps) [d2283cdc18d3378587f9d05be4fd1818059a757a] Merge tag 'irq-urgent-2020-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit d2283cdc18d3378587f9d05be4fd1818059a757a with gcc (GCC) 8.1.0 kernel signature: a2eaa659da15e9bf2ec9d75fafdc95d36ba03a6d3f06f4da7b652d695299c2f1 all runs: OK # git bisect good d2283cdc18d3378587f9d05be4fd1818059a757a Bisecting: 443 revisions left to test after this (roughly 9 steps) [753c66ef4386d2fa5da429e90f0684f0557568cf] net: phy: dp83867: Fix various styling and space issues testing commit 753c66ef4386d2fa5da429e90f0684f0557568cf with gcc (GCC) 8.1.0 kernel signature: aca5dcb91e21d8d4e616a2c351d3fd11650d0c45654ab77dd2829d6521887fbe all runs: OK # git bisect good 753c66ef4386d2fa5da429e90f0684f0557568cf Bisecting: 217 revisions left to test after this (roughly 8 steps) [3e8d3bdc2a757cc6be5470297947799a7df445cc] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit 3e8d3bdc2a757cc6be5470297947799a7df445cc with gcc (GCC) 8.1.0 kernel signature: 6a433bff4d7b9a389d95632926a458e90f3a2f58b352d252a892dad46b567774 all runs: OK # git bisect good 3e8d3bdc2a757cc6be5470297947799a7df445cc Bisecting: 108 revisions left to test after this (roughly 7 steps) [d824e0809ce3c9e935f3aa37381cda7fd4184f12] Merge tag 'libata-5.9-2020-09-04' of git://git.kernel.dk/linux-block testing commit d824e0809ce3c9e935f3aa37381cda7fd4184f12 with gcc (GCC) 8.1.0 kernel signature: 2733b0d9d9982c7e9178d3676ed8732b4665600ccf21cf2cecfb112b989815a2 all runs: OK # git bisect good d824e0809ce3c9e935f3aa37381cda7fd4184f12 Bisecting: 54 revisions left to test after this (roughly 6 steps) [6ec0d0ee66473469ea85a2da204eefc14c465c5e] net: bridge: mdb: arrange internal structs so fast-path fields are close testing commit 6ec0d0ee66473469ea85a2da204eefc14c465c5e with gcc (GCC) 8.1.0 kernel signature: 7546ef0699968cd96298bc22f74ec87c3351632912f4e63937e677e1315e391e all runs: OK # git bisect good 6ec0d0ee66473469ea85a2da204eefc14c465c5e Bisecting: 27 revisions left to test after this (roughly 5 steps) [14e9e262119e5517d4815b9b92860172aba6a969] Merge branch 'sfc-ethtool-for-EF100-and-related-improvements' testing commit 14e9e262119e5517d4815b9b92860172aba6a969 with gcc (GCC) 8.1.0 kernel signature: 4834ba80b6946c6d9ba201086e5a54c2a9ea2805a4d80252f8a0aa4025ee6642 all runs: OK # git bisect good 14e9e262119e5517d4815b9b92860172aba6a969 Bisecting: 13 revisions left to test after this (roughly 4 steps) [652b4987ba1aaa25bd7d76a563e957eddf8ae1c0] net: smsc911x: Remove unused variables testing commit 652b4987ba1aaa25bd7d76a563e957eddf8ae1c0 with gcc (GCC) 8.1.0 kernel signature: 94964719b97851691b4b86f4c81cbbc7c58daa40f8cd01cd07c3f5aaf0a6f5f9 all runs: OK # git bisect good 652b4987ba1aaa25bd7d76a563e957eddf8ae1c0 Bisecting: 6 revisions left to test after this (roughly 3 steps) [f33a7251c825b718ffd79b3b7b74e55e0c5b790a] hippi: switch from 'pci_' to 'dma_' API testing commit f33a7251c825b718ffd79b3b7b74e55e0c5b790a with gcc (GCC) 8.1.0 kernel signature: 56f06479ef5cbf2a0328e53d09dea94517ace19769abbb2915da34de12fff7ec all runs: OK # git bisect good f33a7251c825b718ffd79b3b7b74e55e0c5b790a Bisecting: 3 revisions left to test after this (roughly 2 steps) [288827d53e8edcca94caf6a507105fcbd20f419d] rxrpc: Allow multiple client connections to the same peer testing commit 288827d53e8edcca94caf6a507105fcbd20f419d with gcc (GCC) 8.1.0 kernel signature: a232da1f48dd3f932e97a1acb5f9e02969e67662963c7e8cf90a204052f00ce8 all runs: crashed: kernel BUG at net/rxrpc/conn_object.c:LINE! # git bisect bad 288827d53e8edcca94caf6a507105fcbd20f419d Bisecting: 0 revisions left to test after this (roughly 1 step) [245500d853e9f20036cec7df4f6984ece4c6bf26] rxrpc: Rewrite the client connection manager testing commit 245500d853e9f20036cec7df4f6984ece4c6bf26 with gcc (GCC) 8.1.0 kernel signature: d1141ef3207fa2fe500a3f55e1c1cd594c18c8eb83e37c3fe544fcac881d7557 all runs: crashed: kernel BUG at net/rxrpc/conn_object.c:LINE! # git bisect bad 245500d853e9f20036cec7df4f6984ece4c6bf26 Bisecting: 0 revisions left to test after this (roughly 0 steps) [b7a7d67408032843c14071711d6259aada9392f0] rxrpc: Impose a maximum number of client calls testing commit b7a7d67408032843c14071711d6259aada9392f0 with gcc (GCC) 8.1.0 kernel signature: 7d4f201d509e22c6fd3257373ed049b255235bc723b89ca40910a7f172848811 all runs: OK # git bisect good b7a7d67408032843c14071711d6259aada9392f0 245500d853e9f20036cec7df4f6984ece4c6bf26 is the first bad commit commit 245500d853e9f20036cec7df4f6984ece4c6bf26 Author: David Howells Date: Wed Jul 1 11:15:32 2020 +0100 rxrpc: Rewrite the client connection manager Rewrite the rxrpc client connection manager so that it can support multiple connections for a given security key to a peer. The following changes are made: (1) For each open socket, the code currently maintains an rbtree with the connections placed into it, keyed by communications parameters. This is tricky to maintain as connections can be culled from the tree or replaced within it. Connections can require replacement for a number of reasons, e.g. their IDs span too great a range for the IDR data type to represent efficiently, the call ID numbers on that conn would overflow or the conn got aborted. This is changed so that there's now a connection bundle object placed in the tree, keyed on the same parameters. The bundle, however, does not need to be replaced. (2) An rxrpc_bundle object can now manage the available channels for a set of parallel connections. The lock that manages this is moved there from the rxrpc_connection struct (channel_lock). (3) There'a a dummy bundle for all incoming connections to share so that they have a channel_lock too. It might be better to give each incoming connection its own bundle. This bundle is not needed to manage which channels incoming calls are made on because that's the solely at whim of the client. (4) The restrictions on how many client connections are around are removed. Instead, a previous patch limits the number of client calls that can be allocated. Ordinarily, client connections are reaped after 2 minutes on the idle queue, but when more than a certain number of connections are in existence, the reaper starts reaping them after 2s of idleness instead to get the numbers back down. It could also be made such that new call allocations are forced to wait until the number of outstanding connections subsides. Signed-off-by: David Howells include/trace/events/rxrpc.h | 33 +- net/rxrpc/ar-internal.h | 68 ++- net/rxrpc/conn_client.c | 1081 +++++++++++++++++++----------------------- net/rxrpc/conn_event.c | 14 +- net/rxrpc/conn_object.c | 12 +- net/rxrpc/conn_service.c | 7 + net/rxrpc/local_object.c | 4 +- net/rxrpc/net_ns.c | 5 +- net/rxrpc/output.c | 6 + net/rxrpc/proc.c | 2 +- net/rxrpc/rxkad.c | 8 +- net/rxrpc/sysctl.c | 10 +- 12 files changed, 559 insertions(+), 691 deletions(-) culprit signature: d1141ef3207fa2fe500a3f55e1c1cd594c18c8eb83e37c3fe544fcac881d7557 parent signature: 7d4f201d509e22c6fd3257373ed049b255235bc723b89ca40910a7f172848811 revisions tested: 16, total time: 3h50m51.797840634s (build: 1h19m14.368666627s, test: 2h30m5.359148363s) first bad commit: 245500d853e9f20036cec7df4f6984ece4c6bf26 rxrpc: Rewrite the client connection manager recipients (to): ["davem@davemloft.net" "dhowells@redhat.com" "kuba@kernel.org" "mingo@redhat.com" "netdev@vger.kernel.org" "rostedt@goodmis.org"] recipients (cc): ["dhowells@redhat.com" "linux-afs@lists.infradead.org" "linux-kernel@vger.kernel.org"] crash: kernel BUG at net/rxrpc/conn_object.c:LINE! rxrpc: Assertion failed ------------[ cut here ]------------ kernel BUG at net/rxrpc/conn_object.c:481! invalid opcode: 0000 [#1] PREEMPT SMP CPU: 1 PID: 1169 Comm: kworker/u4:6 Not tainted 5.9.0-rc3-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: netns cleanup_net RIP: 0010:rxrpc_destroy_all_connections.cold.17+0x59/0x69 net/rxrpc/conn_object.c:481 Code: 83 c0 01 00 00 48 89 de 4c 39 e0 48 8d 9a 40 fe ff ff 75 d8 48 89 ef e8 7e 14 31 00 0f 0b 48 c7 c7 8a 52 08 84 e8 79 db 4f fe <0f> 0b cc cc cc cc cc cc cc cc cc cc cc cc cc cc 41 57 41 56 45 31 RSP: 0018:ffffc9000243fd48 EFLAGS: 00010246 RAX: 0000000000000017 RBX: ffff88810a388000 RCX: 0000000000000001 RDX: 0000000000000000 RSI: ffffffff84021f41 RDI: 00000000ffffffff RBP: ffff88810a388088 R08: 0000000000000001 R09: 0000000000000001 R10: ffff88812ab1e080 R11: 79ca7834c8b2ce98 R12: ffff88810a388078 R13: ffff88810a388064 R14: ffff88810da6a000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff88812c100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055e42caeb160 CR3: 000000012078d000 CR4: 00000000001506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: rxrpc_exit_net+0xc9/0x180 net/rxrpc/net_ns.c:119 ops_exit_list.isra.9+0x31/0x60 net/core/net_namespace.c:186 cleanup_net+0x273/0x3d0 net/core/net_namespace.c:603 process_one_work+0x26a/0x5f0 kernel/workqueue.c:2269 worker_thread+0x38/0x380 kernel/workqueue.c:2415 kthread+0x148/0x170 kernel/kthread.c:292 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294 Modules linked in: ---[ end trace e41165d8381974f5 ]--- RIP: 0010:rxrpc_destroy_all_connections.cold.17+0x59/0x69 net/rxrpc/conn_object.c:481 Code: 83 c0 01 00 00 48 89 de 4c 39 e0 48 8d 9a 40 fe ff ff 75 d8 48 89 ef e8 7e 14 31 00 0f 0b 48 c7 c7 8a 52 08 84 e8 79 db 4f fe <0f> 0b cc cc cc cc cc cc cc cc cc cc cc cc cc cc 41 57 41 56 45 31 RSP: 0018:ffffc9000243fd48 EFLAGS: 00010246 RAX: 0000000000000017 RBX: ffff88810a388000 RCX: 0000000000000001 RDX: 0000000000000000 RSI: ffffffff84021f41 RDI: 00000000ffffffff RBP: ffff88810a388088 R08: 0000000000000001 R09: 0000000000000001 R10: ffff88812ab1e080 R11: 79ca7834c8b2ce98 R12: ffff88810a388078 R13: ffff88810a388064 R14: ffff88810da6a000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff88812c100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f209c03d048 CR3: 000000012078d000 CR4: 00000000001506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400