ci2 starts bisection 2026-01-09 22:11:26.974960316 +0000 UTC m=+3832899.152083373 bisecting fixing commit since 22c0b7236c43dfcea893a99fc11dc7d35a518341 building syzkaller on c0460fcde7051a8d07612ec2a17718d3c3019bb0 ensuring issue is reproducible on original commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: c80b6452039ca523acd512df7f26aae603a916bae8379c03672386a92135634b run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: soft lockup in smp_call_function run #5: crashed: BUG: soft lockup in smp_call_function run #6: crashed: BUG: soft lockup in smp_call_function run #7: crashed: BUG: soft lockup in smp_call_function run #8: crashed: BUG: soft lockup in smp_call_function run #9: crashed: BUG: soft lockup in smp_call_function run #10: crashed: BUG: soft lockup in smp_call_function run #11: crashed: BUG: soft lockup in smp_call_function run #12: crashed: BUG: soft lockup in smp_call_function run #13: crashed: BUG: soft lockup in smp_call_function run #14: crashed: BUG: soft lockup in smp_call_function run #15: crashed: BUG: soft lockup in smp_call_function run #16: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor1763631178" "root@10.128.10.24:./syz-executor1763631178"]: exit status 255 run #17: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor3577520834" "root@10.128.10.28:./syz-executor3577520834"]: exit status 255 run #18: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor3512085023" "root@10.128.10.38:./syz-executor3512085023"]: exit status 255 run #19: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor3681791339" "root@10.128.0.161:./syz-executor3681791339"]: exit status 255 representative crash: BUG: soft lockup in smp_call_function, types: [HANG] check whether we can drop unnecessary instrumentation disabling configs for [locking atomic_sleep memleak ubsan bug_or_warning kasan], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 9ecc6f8f5ced527f96e6be68a996e81d3826252044a8d999a170323ceab16ce2 run #0: crashed: BUG: soft lockup in smp_call_function run #1: crashed: no output from test machine run #2: crashed: BUG: soft lockup in smp_call_function run #3: crashed: BUG: soft lockup in smp_call_function run #4: crashed: BUG: soft lockup in smp_call_function run #5: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor4079492320" "root@10.128.10.12:./syz-executor4079492320"]: exit status 255 run #6: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor698861810" "root@10.128.1.155:./syz-executor698861810"]: exit status 255 run #7: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor334427424" "root@10.128.1.159:./syz-executor334427424"]: exit status 255 run #8: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor2611257441" "root@10.128.1.97:./syz-executor2611257441"]: exit status 255 run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor1992042206" "root@10.128.10.16:./syz-executor1992042206"]: exit status 255 representative crash: BUG: soft lockup in smp_call_function, types: [HANG] the bug reproduces without the instrumentation disabling configs for [locking atomic_sleep memleak ubsan bug_or_warning kasan], they are not needed kconfig minimization: base=5186 full=6556 leaves diff=265 split chunks (needed=false): <265> split chunk #0 of len 265 into 5 parts testing without sub-chunk 1/5 disabling configs for [memleak ubsan bug_or_warning kasan locking atomic_sleep], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 1c90a93ead0f57f116de5a6bb5fff002e0544d191701ba9493dbc4e72fbe3ee2 run #0: crashed: BUG: soft lockup in ip_rcv run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: soft lockup in smp_call_function run #4: crashed: BUG: soft lockup in sock_read_iter run #5: crashed: BUG: soft lockup in smp_call_function run #6: crashed: BUG: soft lockup in smp_call_function run #7: crashed: BUG: soft lockup in smp_call_function run #8: crashed: BUG: soft lockup in smp_call_function run #9: crashed: BUG: soft lockup in smp_call_function representative crash: BUG: soft lockup in ip_rcv, types: [HANG] the chunk can be dropped testing without sub-chunk 2/5 disabling configs for [memleak ubsan bug_or_warning kasan locking atomic_sleep], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 3030ceca4b19f9ce7eb08ec5fddfdeead0d305ffd9bc6dca6870fb7a62b5fc40 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: soft lockup in smp_call_function run #3: crashed: BUG: soft lockup in ip_rcv run #4: crashed: BUG: soft lockup in smp_call_function run #5: crashed: BUG: soft lockup in smp_call_function run #6: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor970085595" "root@10.128.1.43:./syz-executor970085595"]: exit status 255 run #7: crashed: BUG: soft lockup in smp_call_function run #8: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor3381687733" "root@10.128.10.24:./syz-executor3381687733"]: exit status 255 run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor155487875" "root@10.128.1.136:./syz-executor155487875"]: exit status 255 representative crash: BUG: soft lockup in smp_call_function, types: [HANG UNKNOWN] the chunk can be dropped testing without sub-chunk 3/5 disabling configs for [atomic_sleep memleak ubsan bug_or_warning kasan locking], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 56dbbbb76d82a69c39384fc02cd762fd4b020cfd7d43aafe537dc1677f199fe1 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: soft lockup in sock_read_iter run #6: crashed: no output from test machine run #7: crashed: no output from test machine run #8: crashed: BUG: soft lockup in smp_call_function run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor1658278033" "root@10.128.10.45:./syz-executor1658278033"]: exit status 255 representative crash: BUG: workqueue lockup, types: [UNKNOWN] the chunk can be dropped testing without sub-chunk 4/5 disabling configs for [atomic_sleep memleak ubsan bug_or_warning kasan locking], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 11ed4290c5568f6c0c74be85beed9ee6fb3bcfb4236fb0d3c17f75e9f3087583 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: soft lockup in smp_call_function run #2: crashed: BUG: soft lockup in smp_call_function run #3: crashed: BUG: soft lockup in smp_call_function run #4: crashed: BUG: soft lockup in smp_call_function run #5: crashed: BUG: soft lockup in smp_call_function run #6: crashed: BUG: soft lockup in smp_call_function run #7: crashed: BUG: soft lockup in smp_call_function run #8: crashed: BUG: soft lockup in smp_call_function run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor1466961101" "root@10.128.1.231:./syz-executor1466961101"]: exit status 255 representative crash: BUG: soft lockup in smp_call_function, types: [HANG] the chunk can be dropped testing without sub-chunk 5/5 disabling configs for [memleak ubsan bug_or_warning kasan locking atomic_sleep], they are not needed testing commit 22c0b7236c43dfcea893a99fc11dc7d35a518341 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 failed building 22c0b7236c43dfcea893a99fc11dc7d35a518341: ld.lld: error: undefined symbol: wext_proc_init ld.lld: error: undefined symbol: wext_proc_exit ld.lld: error: undefined symbol: wext_handle_ioctl ld.lld: error: undefined symbol: compat_wext_handle_ioctl minimized to 53 configs; suspects: [HID_ZEROPLUS USB_MON USB_NET_CDC_MBIM USB_NET_CDC_SUBSET USB_NET_CDC_SUBSET_ENABLE USB_NET_DM9601 USB_NET_GL620A USB_NET_MCS7830 USB_NET_NET1080 USB_NET_PLUSB USB_NET_RNDIS_HOST USB_NET_SMSC75XX USB_NET_SMSC95XX USB_NET_SR9700 USB_NET_SR9800 USB_NET_ZAURUS USB_OHCI_HCD USB_OHCI_HCD_PCI USB_OHCI_HCD_PLATFORM USB_OTG USB_OTG_FSM USB_PRINTER USB_SERIAL_GENERIC USB_SERIAL_PL2303 USB_STORAGE_ALAUDA USB_STORAGE_CYPRESS_ATACB USB_STORAGE_DATAFAB USB_STORAGE_FREECOM USB_STORAGE_ISD200 USB_STORAGE_JUMPSHOT USB_STORAGE_KARMA USB_STORAGE_ONETOUCH USB_STORAGE_SDDR09 USB_STORAGE_SDDR55 USB_STORAGE_USBAT USB_TRANCEVIBRATOR USB_U_AUDIO USB_U_ETHER USB_U_SERIAL USB_WDM V4L2_ASYNC V4L2_FWNODE VIDEO_CAMERA_SENSOR WLAN WLAN_VENDOR_ATH WLAN_VENDOR_ATMEL WLAN_VENDOR_BROADCOM WLAN_VENDOR_INTERSIL WLAN_VENDOR_MARVELL WLAN_VENDOR_MEDIATEK WLAN_VENDOR_MICROCHIP WLAN_VENDOR_PURELIFI WLAN_VENDOR_RALINK WLAN_VENDOR_REALTEK WLAN_VENDOR_RSI WLAN_VENDOR_SILABS WLAN_VENDOR_ZYDAS ZEROPLUS_FF] disabling configs for [memleak ubsan bug_or_warning kasan locking atomic_sleep], they are not needed testing current HEAD 68876f5c70440453cd0fa3ef1041ec14e9154ce0 testing commit 68876f5c70440453cd0fa3ef1041ec14e9154ce0 gcc compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 kernel signature: 2e697cd650a5ac1a1a9f853f19eec40e0185cda9ca19677036304568488031c3 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: soft lockup in smp_call_function run #5: crashed: BUG: soft lockup in smp_call_function run #6: crashed: BUG: soft lockup in smp_call_function run #7: crashed: no output from test machine run #8: crashed: BUG: soft lockup in smp_call_function run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor444605105" "root@10.128.1.117:./syz-executor444605105"]: exit status 255 representative crash: BUG: workqueue lockup, types: [UNKNOWN] crash still not fixed/happens on the oldest tested release revisions tested: 7, total time: 2h29m8.404890066s (build: 31m23.487727632s, test: 1h17m36.165690428s) crash still not fixed or there were kernel test errors commit msg: BACKPORT: PM: sleep: Call pm_sleep_fs_sync() instead of ksys_sync_helper() crash: BUG: workqueue lockup RBP: 00007f492b611f91 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007f492b7e6038 R14: 00007f492b7e5fa0 R15: 00007ffee9197c48 BUG: workqueue lockup - pool cpus=0-1 flags=0x4 nice=0 stuck for 212s! Showing busy workqueues and worker pools: workqueue events: flags=0x0 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: vmstat_shepherd, psi_avgs_work, kfree_rcu_monitor workqueue events_unbound: flags=0x2 pwq 4: cpus=0-1 flags=0x4 nice=0 active=2/512 refcnt=4 pending: toggle_allocation_gate, flush_memcg_stats_dwork workqueue events_power_efficient: flags=0x80 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: neigh_periodic_work, neigh_managed_work, gc_worker workqueue mm_percpu_wq: flags=0x8 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: vmstat_update workqueue kblockd: flags=0x18 pwq 1: cpus=0 node=0 flags=0x0 nice=-20 active=1/256 refcnt=2 pending: blk_mq_timeout_work workqueue dm_bufio_cache: flags=0x8 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: work_fn workqueue wg-kex-wg0: flags=0x6 pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_encrypt_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_encrypt_worker pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_encrypt_worker workqueue wg-kex-wg2: flags=0x6 pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker