ci starts bisection 2023-01-05 00:20:41.587454056 +0000 UTC m=+113623.943644331 bisecting cause commit starting from c76083fac3bae1a87ae3d005b5cb1cbc761e31d5 building syzkaller on ab32d50881df9f96f2af301aadca62ad00b7e099 ensuring issue is reproducible on original commit c76083fac3bae1a87ae3d005b5cb1cbc761e31d5 testing commit c76083fac3bae1a87ae3d005b5cb1cbc761e31d5 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 6bfad69dedfc4de43d77034f110735de94823e5138a2425ef40963269215dc32 all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick testing release v6.1 testing commit 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 659e299569b8ed39235c28f47eda91e15f5f37f01630ad79640866549c3a541e all runs: OK # git bisect start c76083fac3bae1a87ae3d005b5cb1cbc761e31d5 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 Bisecting: 8330 revisions left to test after this (roughly 13 steps) [1ca06f1c1acecbe02124f14a37cce347b8c1a90c] Merge tag 'xtensa-20221213' of https://github.com/jcmvbkbc/linux-xtensa testing commit 1ca06f1c1acecbe02124f14a37cce347b8c1a90c gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5ee2d2df652d015c00e94315dd3fcbbb9a6189759b4615b69b3cedade0792290 all runs: OK # git bisect good 1ca06f1c1acecbe02124f14a37cce347b8c1a90c Bisecting: 4144 revisions left to test after this (roughly 12 steps) [ec9187ecea142593c54cf7a73ef2e1a3d517495a] Merge tag 'i2c-for-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux testing commit ec9187ecea142593c54cf7a73ef2e1a3d517495a gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5c07a47b59155bb1dd23fad485eb4ccc760558db0b62be361af0c6ca99dda956 all runs: OK # git bisect good ec9187ecea142593c54cf7a73ef2e1a3d517495a Bisecting: 2078 revisions left to test after this (roughly 11 steps) [e79041113b19b8c7b8410d862d4a3630debded58] Merge tag 'phy-for-6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/phy/linux-phy testing commit e79041113b19b8c7b8410d862d4a3630debded58 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d03db4dd7272066eff1c8a8cf11eb6ff560f2b978b55802393c738ccef703d1b all runs: OK # git bisect good e79041113b19b8c7b8410d862d4a3630debded58 Bisecting: 1039 revisions left to test after this (roughly 10 steps) [e9f669ce91edf531dcb427c5b64c900184a07055] Merge branch 'asahi-soc/for-next' of https://github.com/AsahiLinux/linux.git testing commit e9f669ce91edf531dcb427c5b64c900184a07055 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 16a6ece031ca44ecd41f7789ebe2fbe7f9f9eac876f30429c131ac12b61eeac9 all runs: OK # git bisect good e9f669ce91edf531dcb427c5b64c900184a07055 Bisecting: 522 revisions left to test after this (roughly 9 steps) [3d47812ecb29fca14ed2e2e0412f26c6696bfae7] Merge branch 'msm-next-lumag' of https://gitlab.freedesktop.org/lumag/msm.git testing commit 3d47812ecb29fca14ed2e2e0412f26c6696bfae7 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 435451d4f26d3b70405e0f9a0a8d23c144a4428ef9eb6c1d30d204d4a4a58aca all runs: OK # git bisect good 3d47812ecb29fca14ed2e2e0412f26c6696bfae7 Bisecting: 279 revisions left to test after this (roughly 8 steps) [1dbb40da154717740301fd0e9a218b6cf1f9d717] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace.git testing commit 1dbb40da154717740301fd0e9a218b6cf1f9d717 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 68ceecaa09f6e8235740e898c49d651e2adbd8de4b8966e70c54aa2ccfdb1f91 all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick # git bisect bad 1dbb40da154717740301fd0e9a218b6cf1f9d717 Bisecting: 119 revisions left to test after this (roughly 7 steps) [fcbfcb41569d0c5c3a326c29fb7045fefe7737a8] Merge branch 'habanalabs-next' of git://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/linux.git testing commit fcbfcb41569d0c5c3a326c29fb7045fefe7737a8 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: fa8e8baa16b32dfaa5ffb01d23e74b69995e9ddad6c57a2016847bb67a0c0144 all runs: OK # git bisect good fcbfcb41569d0c5c3a326c29fb7045fefe7737a8 Bisecting: 59 revisions left to test after this (roughly 6 steps) [98dd6b2ef50d6f7876606a86c8d8a767c9fef6f5] virtio_blk: mark all zone fields LE testing commit 98dd6b2ef50d6f7876606a86c8d8a767c9fef6f5 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 7eeb223345b099d360a31632d67951d559645fa81a5de3725b9b4475950a6c38 all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick # git bisect bad 98dd6b2ef50d6f7876606a86c8d8a767c9fef6f5 Bisecting: 29 revisions left to test after this (roughly 5 steps) [e044cefc19994ba9fb083a45dfe3bc023d2151c6] vDPA/ifcvf: implement features provisioning testing commit e044cefc19994ba9fb083a45dfe3bc023d2151c6 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5fa3927021f0804b55b0b0116e321c2a27e1724672f06989a066210d307a8d2b all runs: OK # git bisect good e044cefc19994ba9fb083a45dfe3bc023d2151c6 Bisecting: 14 revisions left to test after this (roughly 4 steps) [8b17634ac576618aa64c2636bff18ecf4e2faa1e] vduse: Validate vq_num in vduse_validate_config() testing commit 8b17634ac576618aa64c2636bff18ecf4e2faa1e gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a42bf4fe98cbe14d4226b6b6d0d1c570cff40b7157616194f7d905fd0aa8dd40 all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick # git bisect bad 8b17634ac576618aa64c2636bff18ecf4e2faa1e Bisecting: 7 revisions left to test after this (roughly 3 steps) [9e3aa950927da695551d043344fb9a46ac395a96] tools/virtio: Variable type completion testing commit 9e3aa950927da695551d043344fb9a46ac395a96 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 209d9ec61606cce8e5bb1e47d513573b793db28fe68d9f00240ab38de93921bd all runs: OK # git bisect good 9e3aa950927da695551d043344fb9a46ac395a96 Bisecting: 3 revisions left to test after this (roughly 2 steps) [5385023e41d2a492f1232da1727b608963ddb801] virtio: Implementing attribute show with sysfs_emit testing commit 5385023e41d2a492f1232da1727b608963ddb801 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 068d71035ad8e691137ab7350c621621d0071d35d65035ea2fa2b4883da26dfe all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick # git bisect bad 5385023e41d2a492f1232da1727b608963ddb801 Bisecting: 1 revision left to test after this (roughly 1 step) [71bd45c2acfeadcefca601f130857ef85983cf5b] vdpa_sim_net: Offer VIRTIO_NET_F_STATUS testing commit 71bd45c2acfeadcefca601f130857ef85983cf5b gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: c8771eae4583dbb2042c11e8af6d7cfb667c094a403d9041fb62bf155f818f2e all runs: OK # git bisect good 71bd45c2acfeadcefca601f130857ef85983cf5b Bisecting: 0 revisions left to test after this (roughly 0 steps) [b68396fad17ff7fae3bb5b594d703f7195caebb9] virtio/vsock: replace virtio_vsock_pkt with sk_buff testing commit b68396fad17ff7fae3bb5b594d703f7195caebb9 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 3907ab9bcea2f8b8dd623fae6f48690ed6e0e18140559973a1a705cd2dbd87f9 all runs: crashed: kernel BUG in vhost_vsock_handle_tx_kick # git bisect bad b68396fad17ff7fae3bb5b594d703f7195caebb9 b68396fad17ff7fae3bb5b594d703f7195caebb9 is the first bad commit commit b68396fad17ff7fae3bb5b594d703f7195caebb9 Author: Bobby Eshleman Date: Thu Dec 15 04:36:44 2022 +0000 virtio/vsock: replace virtio_vsock_pkt with sk_buff This commit changes virtio/vsock to use sk_buff instead of virtio_vsock_pkt. Beyond better conforming to other net code, using sk_buff allows vsock to use sk_buff-dependent features in the future (such as sockmap) and improves throughput. This patch introduces the following performance changes: Tool/Config: uperf w/ 64 threads, SOCK_STREAM Test Runs: 5, mean of results Before: commit 95ec6bce2a0b ("Merge branch 'net-ipa-more-endpoints'") Test: 64KB, g2h Before: 21.63 Gb/s After: 25.59 Gb/s (+18%) Test: 16B, g2h Before: 11.86 Mb/s After: 17.41 Mb/s (+46%) Test: 64KB, h2g Before: 2.15 Gb/s After: 3.6 Gb/s (+67%) Test: 16B, h2g Before: 14.38 Mb/s After: 18.43 Mb/s (+28%) Signed-off-by: Bobby Eshleman Message-Id: <20221215043645.3545127-1-bobby.eshleman@bytedance.com> Signed-off-by: Michael S. Tsirkin Reviewed-by: Stefano Garzarella Acked-by: Paolo Abeni drivers/vhost/vsock.c | 213 +++++++--------- include/linux/virtio_vsock.h | 126 ++++++++-- net/vmw_vsock/virtio_transport.c | 149 ++++------- net/vmw_vsock/virtio_transport_common.c | 422 ++++++++++++++++++-------------- net/vmw_vsock/vsock_loopback.c | 51 ++-- 5 files changed, 495 insertions(+), 466 deletions(-) culprit signature: 3907ab9bcea2f8b8dd623fae6f48690ed6e0e18140559973a1a705cd2dbd87f9 parent signature: c8771eae4583dbb2042c11e8af6d7cfb667c094a403d9041fb62bf155f818f2e revisions tested: 16, total time: 4h30m4.819073221s (build: 2h37m41.841292065s, test: 1h49m9.926598081s) first bad commit: b68396fad17ff7fae3bb5b594d703f7195caebb9 virtio/vsock: replace virtio_vsock_pkt with sk_buff recipients (to): ["bobby.eshleman@bytedance.com" "mst@redhat.com" "pabeni@redhat.com" "sgarzare@redhat.com"] recipients (cc): [] crash: kernel BUG in vhost_vsock_handle_tx_kick skbuff: skb_over_panic: text:ffffffff86b67f23 len:25109 put:25109 head:ffff88806b895800 data:ffff88806b89582c tail:0x6241 end:0xc0 dev: ------------[ cut here ]------------ kernel BUG at net/core/skbuff.c:120! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 1 PID: 4166 Comm: vhost-4165 Not tainted 6.1.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022 RIP: 0010:skb_panic+0x15b/0x15d net/core/skbuff.c:120 Code: a8 f8 4c 8b 4c 24 10 8b 4b 70 55 45 89 f0 4c 89 ea 41 57 4c 89 e6 48 c7 c7 00 c6 69 8a ff 74 24 10 ff 74 24 20 e8 51 1b c2 ff <0f> 0b 4c 8b 64 24 20 89 34 24 e8 4d 6a a8 f8 8b 34 24 48 c7 c1 a0 RSP: 0018:ffffc90004ae7cb8 EFLAGS: 00010286 RAX: 000000000000008d RBX: ffff88801c818000 RCX: 0000000000000000 RDX: 0000000000000001 RSI: ffffffff89c1ee40 RDI: fffff5200095cf8a RBP: ffffffff8a69c5c0 R08: 000000000000008d R09: ffffc90004ae7ae7 R10: fffff5200095cf5c R11: 0000000000000000 R12: ffffffff8a69d1e0 R13: ffffffff86b67f23 R14: 0000000000006215 R15: 00000000000000c0 FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020085040 CR3: 000000007ad3c000 CR4: 00000000003506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: skb_over_panic net/core/skbuff.c:125 [inline] skb_put.cold+0x24/0x24 net/core/skbuff.c:2152 virtio_vsock_skb_rx_put include/linux/virtio_vsock.h:56 [inline] vhost_vsock_alloc_skb drivers/vhost/vsock.c:373 [inline] vhost_vsock_handle_tx_kick+0x7e3/0x990 drivers/vhost/vsock.c:508 vhost_worker+0x1ac/0x320 drivers/vhost/vhost.c:364 kthread+0x298/0x340 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306 Modules linked in: ---[ end trace 0000000000000000 ]--- RIP: 0010:skb_panic+0x15b/0x15d net/core/skbuff.c:120 Code: a8 f8 4c 8b 4c 24 10 8b 4b 70 55 45 89 f0 4c 89 ea 41 57 4c 89 e6 48 c7 c7 00 c6 69 8a ff 74 24 10 ff 74 24 20 e8 51 1b c2 ff <0f> 0b 4c 8b 64 24 20 89 34 24 e8 4d 6a a8 f8 8b 34 24 48 c7 c1 a0 RSP: 0018:ffffc90004ae7cb8 EFLAGS: 00010286 RAX: 000000000000008d RBX: ffff88801c818000 RCX: 0000000000000000 RDX: 0000000000000001 RSI: ffffffff89c1ee40 RDI: fffff5200095cf8a RBP: ffffffff8a69c5c0 R08: 000000000000008d R09: ffffc90004ae7ae7 R10: fffff5200095cf5c R11: 0000000000000000 R12: ffffffff8a69d1e0 R13: ffffffff86b67f23 R14: 0000000000006215 R15: 00000000000000c0 FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020085040 CR3: 000000007ad3c000 CR4: 00000000003506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400