ci starts bisection 2022-07-19 17:23:07.35267466 +0000 UTC m=+18859.453295611 bisecting cause commit starting from e22c88799f2629088504e1357384f2ec3798da46 building syzkaller on 72a3cc0c8dfd0116bb3ecf36eb5983ec3cae691c testing commit e22c88799f2629088504e1357384f2ec3798da46 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f2733d547d347fe454cedbde9d0cf85f7a54dbb3ea997a297986a8de88eaea74 all runs: crashed: INFO: task hung in lock_sock_nested testing release v5.18 testing commit 4b0986a3613c92f4ec1bdc7f60ec66fea135991f compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f31ab05c81740655f89f08801ac270ec6d2871b6c7909f722a617b3c0afa63c6 all runs: OK # git bisect start e22c88799f2629088504e1357384f2ec3798da46 4b0986a3613c92f4ec1bdc7f60ec66fea135991f Bisecting: 8278 revisions left to test after this (roughly 13 steps) [ae862183285cbb2ef9032770d98ffa9becffe9d5] Merge tag 'arm-dt-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc testing commit ae862183285cbb2ef9032770d98ffa9becffe9d5 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: aa364fb0483e1bf12e0ff62f86007fbbcf8d3419a1a87364a50606413cc75d5a all runs: OK # git bisect good ae862183285cbb2ef9032770d98ffa9becffe9d5 Bisecting: 4210 revisions left to test after this (roughly 12 steps) [4ad680f083ec360e0991c453e18a38ed9ae500d7] Merge tag 'staging-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging testing commit 4ad680f083ec360e0991c453e18a38ed9ae500d7 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 803a9baf8332f983f586d781a9d46bbe6191f2ad365561eebd437255fd03a0e7 run #0: basic kernel testing failed: BUG: program execution failed: executor NUM: failed to write control pipe: write |NUM: broken pipe run #1: boot failed: INFO: task hung in add_early_randomness run #2: boot failed: INFO: task hung in add_early_randomness run #3: boot failed: INFO: task hung in add_early_randomness run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good 4ad680f083ec360e0991c453e18a38ed9ae500d7 Bisecting: 2100 revisions left to test after this (roughly 11 steps) [c24eb8d6a5b2da4cbef6a053f58ea9818c6dd659] Merge tag 'usb-5.19-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb testing commit c24eb8d6a5b2da4cbef6a053f58ea9818c6dd659 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 97ca0b07e8fd2031dad8d7226a036d354fdd6a212dd58b8503826e9e13360c11 failed: failed to create VM pool: failed to create GCE image: create image operation failed: &{Code:PERMISSIONS_ERROR Location: Message:Required 'read' permission for 'disks/ci-upstream-net-kasan-gce-bisect-job-bisect-job-image.tar.gz' ForceSendFields:[] NullFields:[]}. # git bisect skip c24eb8d6a5b2da4cbef6a053f58ea9818c6dd659 Bisecting: 2100 revisions left to test after this (roughly 11 steps) [316f92a705a4c2bf4712135180d56f3cca09243a] iommu/vt-d: Fix PCI bus rescan device hot add testing commit 316f92a705a4c2bf4712135180d56f3cca09243a compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ab3a4772dcfa5f36b4ce279aa8e41c53bf973ac6f5f5b8604c72810877ffb0a2 all runs: OK # git bisect good 316f92a705a4c2bf4712135180d56f3cca09243a Bisecting: 773 revisions left to test after this (roughly 10 steps) [83ec88d81aa8762d4fb75f95365da6b73a38efe9] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit 83ec88d81aa8762d4fb75f95365da6b73a38efe9 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f29b8afedf8958f5510ef990d2844fbda598669a17d322c716cb82085cbfb71e all runs: OK # git bisect good 83ec88d81aa8762d4fb75f95365da6b73a38efe9 Bisecting: 410 revisions left to test after this (roughly 9 steps) [db886979683a8360ced9b24ab1125ad0c4d2cf76] x86/speculation: Use DECLARE_PER_CPU for x86_spec_ctrl_current testing commit db886979683a8360ced9b24ab1125ad0c4d2cf76 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 84324efdb80317b9dd91f674b398ffee728404b23d9bd76411e90d141eccf540 all runs: OK # git bisect good db886979683a8360ced9b24ab1125ad0c4d2cf76 Bisecting: 205 revisions left to test after this (roughly 8 steps) [1abfb265f0accc8635254cf1da03154ac7a5fb1b] net: devlink: fix unlocked vs locked functions descriptions testing commit 1abfb265f0accc8635254cf1da03154ac7a5fb1b compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 7386aa9ab6128112ac817a2200f4b64c4e2053bc0ca1ea99b85ee0e6766efddb all runs: OK # git bisect good 1abfb265f0accc8635254cf1da03154ac7a5fb1b Bisecting: 102 revisions left to test after this (roughly 7 steps) [206bbcf76121664e95a42e1c014c3fe168d07a3d] wifi: nl80211: hold wdev mutex for tid config testing commit 206bbcf76121664e95a42e1c014c3fe168d07a3d compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 71a476eff04c75f236d6580afc5b9928c785deb193834e37ef6e74aab22025e2 run #0: boot failed: INFO: task hung in add_early_randomness run #1: boot failed: INFO: task hung in add_early_randomness run #2: boot failed: INFO: task hung in add_early_randomness run #3: boot failed: INFO: task hung in add_early_randomness run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good 206bbcf76121664e95a42e1c014c3fe168d07a3d Bisecting: 55 revisions left to test after this (roughly 6 steps) [1a55048674379f9b093e0a6dbef586d28b55f9ae] net/mlx5e: Remove the duplicating check for striding RQ when enabling LRO testing commit 1a55048674379f9b093e0a6dbef586d28b55f9ae compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 33b0e8d2e5f27cff4466e9c9d3fb1c304f80071a3ead1a6bb916d2dd5a1a7594 all runs: OK # git bisect good 1a55048674379f9b093e0a6dbef586d28b55f9ae Bisecting: 27 revisions left to test after this (roughly 5 steps) [008141de8557479289936e21e7ed6bad50d09443] tls: rx: don't try to keep the skbs always on the list testing commit 008141de8557479289936e21e7ed6bad50d09443 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 76b8e743b7afbf9304003ffdcc091201b0bc025956d4067b5692a45dfb10ba29 all runs: crashed: INFO: task hung in lock_sock_nested # git bisect bad 008141de8557479289936e21e7ed6bad50d09443 Bisecting: 13 revisions left to test after this (roughly 4 steps) [da92e03c7fbf4dc6e1a4a030433c8c30946e6aa0] octeontx2-af: Fixes static warnings testing commit da92e03c7fbf4dc6e1a4a030433c8c30946e6aa0 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 56951576098e6ac6c79dd90737e04864e3eefc1bd3eaba7dd4557a400420b510 all runs: OK # git bisect good da92e03c7fbf4dc6e1a4a030433c8c30946e6aa0 Bisecting: 6 revisions left to test after this (roughly 3 steps) [0ef69e788411cba2af017db731a9fc62d255e9ac] net/smc: optimize for smc_sndbuf_sync_sg_for_device and smc_rmb_sync_sg_for_cpu testing commit 0ef69e788411cba2af017db731a9fc62d255e9ac compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 98dc7f5478ebb6ea31ffe69faddcb679be7c4eb051f602efa7c2f30bdfa8f893 all runs: OK # git bisect good 0ef69e788411cba2af017db731a9fc62d255e9ac Bisecting: 3 revisions left to test after this (roughly 2 steps) [b8d199451c99b3796b840c350eb74b830c5c869b] net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R testing commit b8d199451c99b3796b840c350eb74b830c5c869b compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 2ab426839874ef850f282e2cb5484f655cd5614b28efc4b2df915115b8129d13 all runs: OK # git bisect good b8d199451c99b3796b840c350eb74b830c5c869b Bisecting: 1 revision left to test after this (roughly 1 step) [3898f52cd4f92a007d71a52edf70aafc7c8ae193] Merge branch 'net-smc-virt-contig-buffers' testing commit 3898f52cd4f92a007d71a52edf70aafc7c8ae193 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e85d76ae3f73eb338e2842a526540dd56f7938d2dfdd5e27a747130c1168ee00 run #0: basic kernel testing failed: BUG: program execution failed: executor NUM: exit status NUM run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect good 3898f52cd4f92a007d71a52edf70aafc7c8ae193 Bisecting: 0 revisions left to test after this (roughly 0 steps) [4cbc325ed6b4dce4910be06d9d6940a8b919c59b] tls: rx: allow only one reader at a time testing commit 4cbc325ed6b4dce4910be06d9d6940a8b919c59b compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: c496ba355d38dd6c4c56e5561ea33d89c5100a3684baaab59bb3f36e48b60f9d all runs: crashed: INFO: task hung in lock_sock_nested # git bisect bad 4cbc325ed6b4dce4910be06d9d6940a8b919c59b 4cbc325ed6b4dce4910be06d9d6940a8b919c59b is the first bad commit commit 4cbc325ed6b4dce4910be06d9d6940a8b919c59b Author: Jakub Kicinski Date: Thu Jul 14 22:22:25 2022 -0700 tls: rx: allow only one reader at a time recvmsg() in TLS gets data from the skb list (rx_list) or fresh skbs we read from TCP via strparser. The former holds skbs which were already decrypted for peek or decrypted and partially consumed. tls_wait_data() only notices appearance of fresh skbs coming out of TCP (or psock). It is possible, if there is a concurrent call to peek() and recv() that the peek() will move the data from input to rx_list without recv() noticing. recv() will then read data out of order or never wake up. This is not a practical use case/concern, but it makes the self tests less reliable. This patch solves the problem by allowing only one reader in. Because having multiple processes calling read()/peek() is not normal avoid adding a lock and try to fast-path the single reader case. Signed-off-by: Jakub Kicinski Signed-off-by: David S. Miller include/net/tls.h | 3 +++ net/tls/tls_sw.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 57 insertions(+), 7 deletions(-) culprit signature: c496ba355d38dd6c4c56e5561ea33d89c5100a3684baaab59bb3f36e48b60f9d parent signature: e85d76ae3f73eb338e2842a526540dd56f7938d2dfdd5e27a747130c1168ee00 revisions tested: 17, total time: 4h41m33.954729452s (build: 1h57m35.38798394s, test: 2h42m13.848795057s) first bad commit: 4cbc325ed6b4dce4910be06d9d6940a8b919c59b tls: rx: allow only one reader at a time recipients (to): ["borisp@nvidia.com" "daniel@iogearbox.net" "davem@davemloft.net" "davem@davemloft.net" "edumazet@google.com" "john.fastabend@gmail.com" "kuba@kernel.org" "kuba@kernel.org" "netdev@vger.kernel.org" "pabeni@redhat.com"] recipients (cc): ["linux-kernel@vger.kernel.org"] crash: INFO: task hung in lock_sock_nested INFO: task syz-executor.0:4125 blocked for more than 143 seconds. Not tainted 5.19.0-rc6-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:syz-executor.0 state:D stack:28520 pid: 4125 ppid: 4093 flags:0x00004006 Call Trace: context_switch kernel/sched/core.c:5146 [inline] __schedule+0x9cd/0x4b20 kernel/sched/core.c:6458 schedule+0xd2/0x1f0 kernel/sched/core.c:6530 __lock_sock+0x118/0x1f0 net/core/sock.c:2826 lock_sock_nested+0xb3/0xd0 net/core/sock.c:3396 lock_sock include/net/sock.h:1659 [inline] tls_sk_proto_close+0xf9/0xa70 net/tls/tls_main.c:311 inet_release+0xef/0x210 net/ipv4/af_inet.c:428 __sock_release+0xbb/0x270 net/socket.c:650 sock_close+0xf/0x20 net/socket.c:1365 __fput+0x1f5/0x8c0 fs/file_table.c:317 task_work_run+0xc0/0x160 kernel/task_work.c:177 get_signal+0x19b/0x2030 kernel/signal.c:2634 arch_do_signal_or_restart+0x82/0x2300 arch/x86/kernel/signal.c:869 exit_to_user_mode_loop kernel/entry/common.c:166 [inline] exit_to_user_mode_prepare+0x15f/0x250 kernel/entry/common.c:201 __syscall_exit_to_user_mode_work kernel/entry/common.c:283 [inline] syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:294 do_syscall_64+0x42/0xb0 arch/x86/entry/common.c:86 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7fae89a89199 RSP: 002b:00007fae8acb6168 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: fffffffffffffe00 RBX: 00007fae89b9c030 RCX: 00007fae89a89199 RDX: 00000000000000c1 RSI: 0000000020000080 RDI: 0000000000000003 RBP: 00007fae89ae313b R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffecb97ea8f R14: 00007fae8acb6300 R15: 0000000000022000 INFO: lockdep is turned off. NMI backtrace for cpu 1 CPU: 1 PID: 28 Comm: khungtaskd Not tainted 5.19.0-rc6-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/29/2022 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x57/0x7d lib/dump_stack.c:106 nmi_cpu_backtrace.cold+0x30/0xc0 lib/nmi_backtrace.c:111 nmi_trigger_cpumask_backtrace+0x140/0x170 lib/nmi_backtrace.c:62 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:212 [inline] watchdog+0x88c/0xbf0 kernel/hung_task.c:369 kthread+0x299/0x340 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306 Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 CPU: 0 PID: 601 Comm: kworker/u4:3 Not tainted 5.19.0-rc6-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/29/2022 Workqueue: phy5 ieee80211_iface_work RIP: 0010:exc_int3+0x0/0x80 arch/x86/kernel/traps.c:811 Code: e0 a5 05 89 48 c7 c2 40 a6 05 89 be 5c 00 00 00 48 8d 7c 24 0c e8 d0 50 25 fb 48 8b 0c 24 e9 08 ff ff ff e8 02 2e 00 00 66 90 <41> 54 55 48 89 fd e8 b5 0d 00 00 85 c0 75 46 f6 85 88 00 00 00 03 RSP: 0018:ffffc90003b8f830 EFLAGS: 00000097 RAX: ffffc90003b8f838 RBX: 0000000000000000 RCX: ffffffff88c00f06 RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffc90003b8f838 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055a1aa8ad680 CR3: 000000000ae8e000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: asm_exc_int3+0x35/0x40 arch/x86/include/asm/idtentry.h:569 RIP: 0010:arch_static_branch arch/x86/include/asm/jump_label.h:27 [inline] RIP: 0010:kfence_alloc include/linux/kfence.h:120 [inline] RIP: 0010:slab_alloc_node mm/slub.c:3161 [inline] RIP: 0010:slab_alloc mm/slub.c:3251 [inline] RIP: 0010:kmem_cache_alloc_trace+0x120/0x3f0 mm/slub.c:3282 Code: 00 e8 94 b2 ac ff 48 c7 c6 97 37 b0 81 48 c7 c7 60 d0 17 8b e8 21 9a a5 ff 45 31 e4 e9 c1 00 00 00 45 31 e4 48 85 ed 74 f0 cc 00 00 00 48 c7 44 24 08 00 00 00 00 48 8b 4d 00 65 48 03 0d ff RSP: 0018:ffffc90003b8f910 EFLAGS: 00000286 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 RDX: 1ffffffff165ae14 RSI: 0000000000000400 RDI: ffff88801c53b284 RBP: ffff888010c41dc0 R08: 0000000000000000 R09: ffff888011bc4060 R10: fffffbfff19d7fea R11: ffff888024599b18 R12: 0000000000000000 R13: 0000000000000b20 R14: 0000000000000220 R15: 0000000000000b20 kmalloc include/linux/slab.h:600 [inline] kzalloc include/linux/slab.h:733 [inline] ieee802_11_parse_elems_crc+0xcb/0xf70 net/mac80211/util.c:1505 ieee802_11_parse_elems net/mac80211/ieee80211_i.h:2119 [inline] ieee80211_rx_mgmt_probe_beacon net/mac80211/ibss.c:1606 [inline] ieee80211_ibss_rx_queued_mgmt+0xabd/0x3030 net/mac80211/ibss.c:1640 ieee80211_iface_process_skb net/mac80211/iface.c:1596 [inline] ieee80211_iface_work+0x749/0x990 net/mac80211/iface.c:1650 process_one_work+0x865/0x13d0 kernel/workqueue.c:2289 worker_thread+0x598/0xec0 kernel/workqueue.c:2436 kthread+0x299/0x340 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306