ci starts bisection 2025-03-27 07:52:23.175801122 +0000 UTC m=+16632.052318965
bisecting cause commit starting from 2df0c02dab829dd89360d98a8a1abaa026ef5798
building syzkaller on 89d30d7360d4a366f8fdf00d6ac56cced7a45b0b
ensuring issue is reproducible on original commit 2df0c02dab829dd89360d98a8a1abaa026ef5798
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 1bc6c47c57533ed6d52cb72913fa5c1f1bfdf6453b5360d15f36b762d6b51724
run #0: crashed: WARNING: ODEBUG bug in delete_node
run #1: crashed: WARNING: ODEBUG bug in delete_node
run #2: basic kernel testing failed: failed to copy binary to VM: timedout after 1m0s ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor2780200094" "root@10.128.1.185:./syz-executor2780200094"]
Executing: program /usr/bin/ssh host 10.128.1.185, user root, command sftp
OpenSSH_9.2p1 Debian-2+deb12u4, OpenSSL 3.0.15 3 Sep 2024
debug1: Reading configuration data /dev/null
debug1: Connecting to 10.128.1.185 [10.128.1.185] port 22.
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u4
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.1
debug1: compat_banner: match: OpenSSH_9.1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.128.1.185:22 as 'root'
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: sntrup761x25519-sha512@openssh.com
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
run #3: crashed: WARNING: ODEBUG bug in delete_node
run #4: crashed: WARNING: ODEBUG bug in delete_node
run #5: OK
run #6: OK
run #7: OK
run #8: OK
run #9: OK
run #10: OK
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING: ODEBUG bug in delete_node, types: [WARNING]
check whether we can drop unnecessary instrumentation
disabling configs for [LOCKDEP ATOMIC_SLEEP HANG LEAK UBSAN KASAN], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 7f15bc092286e87534dab836668f428d38f3c2cd71842561aa45020f8b137a8a
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: basic kernel testing failed: failed to copy binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "IdentitiesOnly=yes" "-o" "BatchMode=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-v" "/tmp/syz-executor1102340880" "root@10.128.1.183:./syz-executor1102340880"]: exit status 255
Executing: program /usr/bin/ssh host 10.128.1.183, user root, command sftp
OpenSSH_9.2p1 Debian-2+deb12u4, OpenSSL 3.0.15 3 Sep 2024
debug1: Reading configuration data /dev/null
debug1: Connecting to 10.128.1.183 [10.128.1.183] port 22.
debug1: connect to address 10.128.1.183 port 22: Connection timed out
ssh: connect to host 10.128.1.183 port 22: Connection timed out
scp: Connection closed
run #10: OK
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
the bug reproduces without the instrumentation
disabling configs for [UBSAN KASAN LOCKDEP ATOMIC_SLEEP HANG LEAK], they are not needed
kconfig minimization: base=4054 full=8267 leaves diff=2142
split chunks (needed=false): <2142>
split chunk #0 of len 2142 into 5 parts
testing without sub-chunk 1/5
disabling configs for [KASAN LOCKDEP ATOMIC_SLEEP HANG LEAK UBSAN], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: c84b0b0ada11968066aa8cb54057fe63e928877369aa6aa226f6f3fdc370a7c8
all runs: OK
false negative chance: 0.009
testing without sub-chunk 2/5
disabling configs for [KASAN LOCKDEP ATOMIC_SLEEP HANG LEAK UBSAN], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 7dbfa3beec62edc35ac5763ca387d6854daa170b7b5c3aad61dbd5ae83f00248
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: crashed: WARNING in call_rcu
run #10: OK
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
the chunk can be dropped
testing without sub-chunk 3/5
disabling configs for [LOCKDEP ATOMIC_SLEEP HANG LEAK UBSAN KASAN], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 37ad1afeeb490e320824af766272b78210068d008cdc02b7680f52ba01764436
all runs: OK
false negative chance: 0.009
testing without sub-chunk 4/5
disabling configs for [ATOMIC_SLEEP HANG LEAK UBSAN KASAN LOCKDEP], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: b145bdc207ba9e98bb989992dbb37023d4156639665b685de45eabfd4cdfe562
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: crashed: WARNING in call_rcu
run #10: crashed: WARNING in call_rcu
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
the chunk can be dropped
testing without sub-chunk 5/5
disabling configs for [LEAK UBSAN KASAN LOCKDEP ATOMIC_SLEEP HANG], they are not needed
testing commit 2df0c02dab829dd89360d98a8a1abaa026ef5798 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 9f9160087a6dbca17651990ab6b0b1fad390524554b6a8f2d3148dfeda2df3bb
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: OK
run #10: OK
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: crashed: WARNING in call_rcu
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
the chunk can be dropped
minimized to 858 configs; suspects: [6LOWPAN 6LOWPAN_GHC_EXT_HDR_DEST 6LOWPAN_GHC_EXT_HDR_FRAG 6LOWPAN_GHC_EXT_HDR_HOP 6LOWPAN_GHC_EXT_HDR_ROUTE 6LOWPAN_GHC_ICMPV6 6LOWPAN_GHC_UDP 6LOWPAN_NHC 6LOWPAN_NHC_DEST 6LOWPAN_NHC_FRAGMENT 6LOWPAN_NHC_HOP 6LOWPAN_NHC_IPV6 6LOWPAN_NHC_MOBILITY 6LOWPAN_NHC_ROUTING 6LOWPAN_NHC_UDP 6PACK 842_COMPRESS 842_DECOMPRESS 9P_FSCACHE 9P_FS_POSIX_ACL 9P_FS_SECURITY ACORN_PARTITION ACORN_PARTITION_ADFS ACORN_PARTITION_CUMANA ACORN_PARTITION_EESOX ACORN_PARTITION_ICS ACORN_PARTITION_POWERTEC ACORN_PARTITION_RISCIX ACPI_NFIT ACPI_NHLT ACPI_PLATFORM_PROFILE ADFS_FS AFFS_FS AFS_FS AFS_FSCACHE AF_KCM AF_RXRPC AF_RXRPC_IPV6 AIX_PARTITION AMD_SFH_HID AMIGA_PARTITION ANDROID_BINDER_IPC ANON_VMA_NAME APERTURE_HELPERS APPLE_MFI_FASTCHARGE AR5523 ARCH_ENABLE_MEMORY_HOTREMOVE ARCH_ENABLE_THP_MIGRATION ARCH_HAS_EXECMEM_ROX ARCH_HAS_USER_SHADOW_STACK ARCH_SUPPORTS_HUGE_PFNMAP ARCH_SUPPORTS_PMD_PFNMAP ARCH_SUPPORTS_PUD_PFNMAP ARCH_WANT_PMD_MKWRITE ASM_MODVERSIONS ASUS_TF103C_DOCK ASYNC_CORE ASYNC_MEMCPY ASYNC_PQ ASYNC_RAID6_RECOV ASYNC_TX_DMA ASYNC_XOR ATARI_PARTITION ATA_GENERIC ATA_OVER_ETH ATH10K ATH10K_CE ATH10K_LEDS ATH10K_PCI ATH10K_USB ATH11K ATH6KL ATH6KL_USB ATH9K ATH9K_AHB ATH9K_BTCOEX_SUPPORT ATH9K_CHANNEL_CONTEXT ATH9K_COMMON ATH9K_COMMON_DEBUG ATH9K_COMMON_SPECTRAL ATH9K_DEBUGFS ATH9K_DYNACK ATH9K_HTC ATH9K_HTC_DEBUGFS ATH9K_HW ATH9K_PCI ATH9K_PCOEM ATH9K_RFKILL ATH_COMMON ATM ATM_BR2684 ATM_CLIP ATM_DRIVERS ATM_LANE ATM_MPOA ATM_TCP AUXILIARY_BUS AX25 AX25_DAMA_SLAVE AX88796B_PHY BAREUDP BATMAN_ADV BATMAN_ADV_BATMAN_V BATMAN_ADV_BLA BATMAN_ADV_DAT BATMAN_ADV_MCAST BATMAN_ADV_NC BCACHE BCACHEFS_DEBUG BCACHEFS_ERASURE_CODING BCACHEFS_FS BCACHEFS_POSIX_ACL BCACHEFS_QUOTA BCACHEFS_SIX_OPTIMISTIC_SPIN BCMA BCMA_HOST_PCI_POSSIBLE BEFS_FS BFQ_CGROUP_DEBUG BFQ_GROUP_IOSCHED BFS_FS BIG_KEYS BLK_CGROUP_PUNT_BIO BLK_CGROUP_RWSTAT BLK_DEV_BSGLIB BLK_DEV_INTEGRITY BLK_DEV_NBD BLK_DEV_NULL_BLK BLK_DEV_NULL_BLK_FAULT_INJECTION BLK_DEV_NVME BLK_DEV_PMEM BLK_DEV_RAM BLK_DEV_RNBD BLK_DEV_RNBD_CLIENT BLK_DEV_THROTTLING BLK_DEV_ZONED BLK_ICQ BLK_INLINE_ENCRYPTION BLK_INLINE_ENCRYPTION_FALLBACK BLK_WBT BLK_WBT_MQ BONDING BOOT_VESA_SUPPORT BPF_EVENTS BPF_JIT BPF_JIT_ALWAYS_ON BPF_JIT_DEFAULT_ON BPF_LSM BPF_PRELOAD BPF_PRELOAD_UMD BPF_STREAM_PARSER BPF_SYSCALL BPQETHER BRIDGE BRIDGE_CFM BRIDGE_EBT_802_3 BRIDGE_EBT_AMONG BRIDGE_EBT_ARP BRIDGE_EBT_ARPREPLY BRIDGE_EBT_BROUTE BRIDGE_EBT_DNAT BRIDGE_EBT_IP BRIDGE_EBT_IP6 BRIDGE_EBT_LIMIT BRIDGE_EBT_LOG BRIDGE_EBT_MARK BRIDGE_EBT_MARK_T BRIDGE_EBT_NFLOG BRIDGE_EBT_PKTTYPE BRIDGE_EBT_REDIRECT BRIDGE_EBT_SNAT BRIDGE_EBT_STP BRIDGE_EBT_T_FILTER BRIDGE_EBT_T_NAT BRIDGE_EBT_VLAN BRIDGE_IGMP_SNOOPING BRIDGE_MRP BRIDGE_NETFILTER BRIDGE_NF_EBTABLES BRIDGE_NF_EBTABLES_LEGACY BRIDGE_VLAN_FILTERING BROADCAST_TLB_FLUSH BSD_DISKLABEL BSD_PROCESS_ACCT_V3 BT BTRFS_ASSERT BTRFS_FS BTRFS_FS_POSIX_ACL BTRFS_FS_REF_VERIFY BTT BT_6LOWPAN BT_ATH3K BT_BCM BT_BNEP BT_BNEP_MC_FILTER BT_BNEP_PROTO_FILTER BT_BREDR BT_HCIBCM203X BT_HCIBFUSB BT_HCIBPA10X BT_HCIBTUSB BT_HCIBTUSB_AUTOSUSPEND BT_HCIBTUSB_BCM BT_HCIBTUSB_MTK BT_HCIBTUSB_POLL_SYNC BT_HCIBTUSB_RTL BT_HCIUART BT_HCIUART_3WIRE BT_HCIUART_AG6XX BT_HCIUART_BCSP BT_HCIUART_H4 BT_HCIUART_LL BT_HCIUART_MRVL BT_HCIUART_QCA BT_HCIUART_SERDEV BT_HCIVHCI BT_INTEL BT_LE BT_LEDS BT_LE_L2CAP_ECRED BT_MRVL BT_MRVL_SDIO BT_MSFTEXT BT_MTK BT_MTKSDIO BT_MTKUART BT_QCA BT_RFCOMM BT_RFCOMM_TTY BT_RTL CACHEFILES CAIF CAIF_DEBUG CAIF_DRIVERS CAIF_NETDEV CAIF_TTY CAIF_USB CAIF_VIRTIO CAN CAN_8DEV_USB CAN_BCM CAN_CALC_BITTIMING CAN_DEV CAN_EMS_USB CAN_ESD_USB CAN_ETAS_ES58X CAN_F81604 CAN_GS_USB CAN_GW CAN_IFI_CANFD CAN_ISOTP CAN_J1939 CAN_KVASER_USB CAN_MCBA_USB CAN_NETLINK CAN_PEAK_USB CAN_RAW CAN_RX_OFFLOAD CAN_SLCAN CAN_UCAN CAN_VCAN CAN_VXCAN CARL9170 CARL9170_HWRNG CARL9170_LEDS CARL9170_WPC CEC_CORE CEPH_FS CEPH_FSCACHE CEPH_FS_POSIX_ACL CEPH_LIB CEPH_LIB_USE_DNS_RESOLVER CFG80211 CFG80211_CRDA_SUPPORT CFG80211_DEBUGFS CFG80211_DEFAULT_PS CFG80211_REQUIRE_SIGNED_REGDB CFG80211_USE_KERNEL_REGDB_KEYS CFG80211_WEXT CFS_BANDWIDTH CGROUP_BPF CHARGER_ISP1704 CHECKPOINT_RESTORE CHR_DEV_ST CIFS CIFS_ALLOW_INSECURE_LEGACY CIFS_DEBUG CIFS_DFS_UPCALL CIFS_FSCACHE CIFS_POSIX CIFS_SMB_DIRECT CIFS_SWN_UPCALL CIFS_UPCALL CIFS_XATTR CLOSURES CLS_U32_MARK CLS_U32_PERF CMA CMA_SIZE_SEL_MAX CMDLINE_PARTITION COMEDI COMEDI_DT9812 COMEDI_NI_USB6501 COMEDI_USBDUX COMEDI_USBDUXFAST COMEDI_USBDUXSIGMA COMEDI_USB_DRIVERS COMEDI_VMK80XX COMPAT_NETLINK_MESSAGES COUNTER CRAMFS CRAMFS_BLOCKDEV CRAMFS_MTD CRC4 CRC64 CRC64_ROCKSOFT CRC7 CRC8 CRC_ITU_T CRC_T10DIF CRC_T10DIF_ARCH CRYPTO_842 CRYPTO_ADIANTUM CRYPTO_AEGIS128 CRYPTO_AEGIS128_AESNI_SSE2 CRYPTO_AES_NI_INTEL CRYPTO_AES_TI CRYPTO_ANSI_CPRNG CRYPTO_ANUBIS CRYPTO_ARC4 CRYPTO_ARCH_HAVE_LIB_BLAKE2S CRYPTO_ARCH_HAVE_LIB_CHACHA CRYPTO_ARCH_HAVE_LIB_CURVE25519 CRYPTO_ARCH_HAVE_LIB_POLY1305 CRYPTO_ARIA CRYPTO_ARIA_AESNI_AVX_X86_64 CRYPTO_BLAKE2B CRYPTO_BLAKE2S_X86 CRYPTO_BLOWFISH CRYPTO_BLOWFISH_COMMON CRYPTO_BLOWFISH_X86_64 CRYPTO_CAMELLIA CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 CRYPTO_CAMELLIA_AESNI_AVX_X86_64 CRYPTO_CAMELLIA_X86_64 CRYPTO_CAST5 CRYPTO_CAST5_AVX_X86_64 CRYPTO_CAST6 CRYPTO_CAST6_AVX_X86_64 CRYPTO_CAST_COMMON CRYPTO_CHACHA20 CRYPTO_CHACHA20POLY1305 CRYPTO_CHACHA20_X86_64 CRYPTO_CRC32 CRYPTO_CRC32C CRYPTO_CRC64_ROCKSOFT CRYPTO_CRYPTD CRYPTO_CTS CRYPTO_CURVE25519 CRYPTO_CURVE25519_X86 CRYPTO_DEFLATE CRYPTO_DES CRYPTO_DES3_EDE_X86_64 CRYPTO_DEV_CCP CRYPTO_DEV_CCP_DD CRYPTO_DEV_PADLOCK CRYPTO_DEV_PADLOCK_AES CRYPTO_DEV_PADLOCK_SHA CRYPTO_DEV_QAT CRYPTO_DEV_QAT_C3XXX CRYPTO_DEV_QAT_C3XXXVF CRYPTO_DEV_QAT_C62X CRYPTO_DEV_QAT_C62XVF CRYPTO_DEV_QAT_DH895xCC CRYPTO_DEV_QAT_DH895xCCVF CRYPTO_DEV_VIRTIO CRYPTO_DH CRYPTO_DRBG_CTR CRYPTO_DRBG_HASH CRYPTO_ECDH CRYPTO_ECRDSA CRYPTO_ENGINE CRYPTO_ESSIV CRYPTO_FCRYPT CRYPTO_GHASH_CLMUL_NI_INTEL CRYPTO_HCTR2 CRYPTO_KDF800108_CTR CRYPTO_KHAZAD CRYPTO_KPP CRYPTO_LIB_ARC4 CRYPTO_LIB_CHACHA CRYPTO_LIB_CHACHA20POLY1305 CRYPTO_LIB_CHACHA_GENERIC CRYPTO_LIB_CURVE25519 CRYPTO_LIB_CURVE25519_GENERIC CRYPTO_LIB_DES CRYPTO_LIB_POLY1305 CRYPTO_LIB_POLY1305_GENERIC CRYPTO_LRW CRYPTO_LZ4 CRYPTO_LZ4HC CRYPTO_MICHAEL_MIC CRYPTO_NHPOLY1305 CRYPTO_NHPOLY1305_AVX2 CRYPTO_NHPOLY1305_SSE2 CRYPTO_PCBC CRYPTO_PCRYPT CRYPTO_POLY1305 CRYPTO_POLY1305_X86_64 CRYPTO_POLYVAL CRYPTO_POLYVAL_CLMUL_NI CRYPTO_RMD160 CRYPTO_SEED CRYPTO_SERPENT CRYPTO_SERPENT_AVX2_X86_64 CRYPTO_SERPENT_AVX_X86_64 CRYPTO_SERPENT_SSE2_X86_64 CRYPTO_SHA1_SSSE3 CRYPTO_SHA256_SSSE3 CRYPTO_SHA512_SSSE3 CRYPTO_SIMD CRYPTO_SM3 CRYPTO_SM3_AVX_X86_64 CRYPTO_SM4 CRYPTO_SM4_AESNI_AVX2_X86_64 CRYPTO_SM4_AESNI_AVX_X86_64 CRYPTO_SM4_GENERIC CRYPTO_STREEBOG CRYPTO_TEA CRYPTO_TWOFISH CRYPTO_TWOFISH_AVX_X86_64 CRYPTO_TWOFISH_COMMON CRYPTO_TWOFISH_X86_64 CRYPTO_TWOFISH_X86_64_3WAY CRYPTO_USER CRYPTO_USER_API CRYPTO_USER_API_AEAD CRYPTO_USER_API_ENABLE_OBSOLETE CRYPTO_USER_API_HASH CRYPTO_USER_API_RNG CRYPTO_USER_API_SKCIPHER CRYPTO_WP512 CRYPTO_XCBC CRYPTO_XCTR CRYPTO_XTS CRYPTO_XXHASH CRYPTO_ZSTD CUSE CYPRESS_FIRMWARE DAMON DAMON_PADDR DAMON_RECLAIM DAMON_VADDR DAX DCA DCB DEFAULT_CODEL DEVICE_MIGRATION DEVICE_PRIVATE DEV_COREDUMP DEV_DAX DLN2_ADC DMABUF_HEAPS DMABUF_HEAPS_CMA DMABUF_HEAPS_SYSTEM DMABUF_MOVE_NOTIFY DMA_CMA DMA_ENGINE_RAID DM_AUDIT DM_BIO_PRISON DM_BUFIO DM_CACHE DM_CACHE_SMQ DM_CLONE DM_CRYPT DM_FLAKEY DM_INTEGRITY DM_MULTIPATH DM_MULTIPATH_QL DM_MULTIPATH_ST DM_PERSISTENT_DATA DM_RAID DM_SNAPSHOT DM_THIN_PROVISIONING DM_UEVENT DM_VERITY DM_VERITY_FEC DM_WRITECACHE DM_ZONED DRAGONRISE_FF DRM DRM_AUX_BRIDGE DRM_BOCHS DRM_BRIDGE DRM_BUDDY DRM_CIRRUS_QEMU DRM_CLIENT DRM_CLIENT_DEFAULT_FBDEV DRM_CLIENT_LIB DRM_CLIENT_SELECTION DRM_CLIENT_SETUP DRM_DEBUG_MM DRM_DISPLAY_DP_AUX_BUS DRM_DISPLAY_DP_HELPER DRM_DISPLAY_DSC_HELPER DRM_DISPLAY_HDCP_HELPER DRM_DISPLAY_HDMI_HELPER DRM_DISPLAY_HELPER DRM_FBDEV_EMULATION DRM_GEM_SHMEM_HELPER DRM_GM12U320 DRM_GUD DRM_I915 DRM_I915_CAPTURE_ERROR DRM_I915_COMPRESS_ERROR DRM_I915_USERPTR DRM_KMS_HELPER DVB_CORE ENCRYPTED_KEYS FB_CORE FSCACHE FUSE_FS GPIOLIB HAMRADIO HID_DRAGONRISE HID_NINTENDO HSR IIO INFINIBAND INFINIBAND_ADDR_TRANS INFINIBAND_RTRS_CLIENT INFINIBAND_USER_ACCESS INPUT_JOYSTICK INPUT_MOUSE IOSCHED_BFQ IP6_NF_RAW IPV6_MULTIPLE_TABLES IP_NF_RAW IP_SET IP_VS IP_VS_NQ IP_VS_OVF IP_VS_PE_SIP IP_VS_PROTO_AH IP_VS_PROTO_AH_ESP IP_VS_PROTO_ESP IP_VS_PROTO_SCTP IP_VS_PROTO_UDP IP_VS_RR IP_VS_SED IP_VS_SH IP_VS_TWOS IP_VS_WLC IP_VS_WRR IRQ_BYPASS_MANAGER IRQ_POLL IR_IGORPLUGUSB IR_IGUANA IR_IMON IR_IMON_RAW IR_MCEUSB IR_REDRAT3 IR_STREAMZAP IR_TOY IR_TTUSBIR ISDN ISDN_CAPI JFFS2_CMODE_PRIORITY JFFS2_COMPRESSION_OPTIONS JFFS2_FS JFFS2_FS_POSIX_ACL JFFS2_FS_SECURITY JFFS2_FS_WRITEBUFFER JFFS2_FS_XATTR JFFS2_LZO JFFS2_RTIME JFFS2_RUBIN JFFS2_SUMMARY JFFS2_ZLIB JFS_DEBUG JFS_FS JFS_POSIX_ACL JFS_SECURITY JOYSTICK_IFORCE JOYSTICK_IFORCE_USB JOYSTICK_PXRC JOYSTICK_XPAD JOYSTICK_XPAD_FF JOYSTICK_XPAD_LEDS KARMA_PARTITION KCOV KCOV_ENABLE_COMPARISONS KCOV_INSTRUMENT_ALL KEYS_REQUEST_CACHE KEY_DH_OPERATIONS KEY_NOTIFICATIONS KSM KVFREE_RCU_BATCHED KVM KVM_AMD KVM_ASYNC_PF KVM_COMMON KVM_COMPAT KVM_ELIDE_TLB_FLUSH_IF_YOUNG KVM_GENERIC_DIRTYLOG_READ_PROTECT KVM_GENERIC_HARDWARE_ENABLING KVM_GENERIC_MEMORY_ATTRIBUTES KVM_GENERIC_MMU_NOTIFIER KVM_GENERIC_PRE_FAULT_MEMORY KVM_GENERIC_PRIVATE_MEM KVM_HYPERV KVM_MMIO KVM_PRIVATE_MEM KVM_PROVE_MMU KVM_SW_PROTECTED_VM KVM_VFIO KVM_X86 KVM_XEN KVM_XFER_TO_GUEST_WORK L2TP L2TP_ETH L2TP_IP L2TP_V3 LAPB LAPBETHER LDM_PARTITION LEGACY_PTYS LIBCRC32C LIBNVDIMM LINEAR_RANGES LLC LLC2 LOGIG940_FF LOGIRUMBLEPAD2_FF LOGO LOGO_LINUX_MONO LOGO_LINUX_VGA16 LPC_ICH LWTUNNEL LWTUNNEL_BPF LZ4HC_COMPRESS LZ4_COMPRESS MAC80211 MAC80211_DEBUGFS MAC80211_HAS_RC MAC80211_HWSIM MAC80211_LEDS MAC80211_MESH MAC80211_RC_DEFAULT_MINSTREL MAC80211_RC_MINSTREL MACSEC MACVLAN MACVTAP MAC_PARTITION MAPPING_DIRTY_HELPERS MDIO_MVUSB MD_RAID0 MD_RAID1 MD_RAID10 MD_RAID456 MEDIA_ANALOG_TV_SUPPORT MEDIA_ATTACH MEDIA_CONTROLLER MEDIA_CONTROLLER_DVB MEDIA_DIGITAL_TV_SUPPORT MEDIA_RADIO_SUPPORT MEDIA_SDR_SUPPORT MEDIA_SUPPORT MEDIA_SUPPORT_FILTER MEDIA_TUNER MEDIA_TUNER_MSI001 MEMORY_BALLOON MEMORY_HOTPLUG MEMORY_HOTREMOVE MEMORY_ISOLATION MEMREGION MEMSTICK MEMSTICK_REALTEK_USB MEM_SOFT_DIRTY MFD_CORE MFD_DLN2 MFD_SYSCON MHI_BUS MHI_WWAN_CTRL MHP_DEFAULT_ONLINE_TYPE_ONLINE_AUTO MHP_MEMMAP_ON_MEMORY MICROCHIP_PHY MINIX_FS MINIX_SUBPARTITION MIN_HEAP MISC_RTSX MISC_RTSX_USB MISDN MISDN_DSP MISDN_HFCUSB MISDN_L1OIP MKISS MLX4_CORE MLX4_INFINIBAND MMC MMC_REALTEK_USB MMC_USHC MMC_VUB300 MODULE_SRCVERSION_ALL MOST MOST_USB_HDM MOUSE_APPLETOUCH MOUSE_BCM5974 MOUSE_PS2 MOUSE_PS2_ALPS MOUSE_PS2_BYD MOUSE_PS2_CYPRESS MOUSE_PS2_FOCALTECH MOUSE_PS2_LIFEBOOK MOUSE_PS2_LOGIPS2PP MOUSE_PS2_SMBUS MOUSE_PS2_SYNAPTICS MOUSE_PS2_SYNAPTICS_SMBUS MOUSE_PS2_TRACKPOINT MOUSE_SYNAPTICS_USB MPLS MPLS_IPTUNNEL MPLS_ROUTING MPTCP MPTCP_IPV6 MRP MTD MTD_BLKDEVS MTD_BLOCK MTD_BLOCK2MTD MTD_CFI_I1 MTD_CFI_I2 MTD_MAP_BANK_WIDTH_1 MTD_MAP_BANK_WIDTH_2 MTD_MAP_BANK_WIDTH_4 MTD_MTDRAM MTD_PHRAM MTD_SLRAM MUSB_PIO_ONLY ND_BTT ND_CLAIM ND_PFN NETDEVSIM NETFILTER_ADVANCED NETFILTER_BPF_LINK NETFILTER_FAMILY_ARP NETFILTER_FAMILY_BRIDGE NETFILTER_NETLINK_ACCT NETFILTER_NETLINK_GLUE_CT NETFILTER_NETLINK_OSF NETFILTER_NETLINK_QUEUE NETFILTER_SYNPROXY NETFILTER_XTABLES_COMPAT NETFILTER_XT_CONNMARK NETFILTER_XT_MATCH_BPF NETFILTER_XT_MATCH_CGROUP NETFILTER_XT_MATCH_CLUSTER NETFILTER_XT_MATCH_COMMENT NETFILTER_XT_MATCH_CONNBYTES NETFILTER_XT_MATCH_CONNLABEL NETFILTER_XT_MATCH_CONNLIMIT NETFILTER_XT_MATCH_CONNMARK NETFILTER_XT_MATCH_CPU NETFILTER_XT_MATCH_DCCP NETFILTER_XT_MATCH_DEVGROUP NETFILTER_XT_MATCH_DSCP NETFILTER_XT_MATCH_ECN NETFILTER_XT_MATCH_ESP NETFILTER_XT_MATCH_HASHLIMIT NETFILTER_XT_MATCH_HELPER NETFILTER_XT_MATCH_HL NETFILTER_XT_MATCH_IPCOMP NETFILTER_XT_MATCH_IPRANGE NETFILTER_XT_MATCH_IPVS NETFILTER_XT_MATCH_L2TP NETFILTER_XT_MATCH_LENGTH NETFILTER_XT_MATCH_LIMIT NETFILTER_XT_MATCH_MAC NETFILTER_XT_MATCH_MARK NETFILTER_XT_MATCH_MULTIPORT NETFILTER_XT_MATCH_NFACCT NETFILTER_XT_MATCH_OSF NETFILTER_XT_MATCH_OWNER NETFILTER_XT_MATCH_PHYSDEV NETFILTER_XT_MATCH_PKTTYPE NETFILTER_XT_MATCH_QUOTA NETFILTER_XT_MATCH_RATEEST NETFILTER_XT_MATCH_REALM NETFILTER_XT_MATCH_RECENT NETFILTER_XT_MATCH_SCTP NETFILTER_XT_MATCH_SOCKET NETFILTER_XT_MATCH_STATISTIC NETFILTER_XT_MATCH_STRING NETFILTER_XT_MATCH_TCPMSS NETFILTER_XT_MATCH_TIME NETFILTER_XT_MATCH_U32 NETFILTER_XT_SET NETFILTER_XT_TARGET_AUDIT NETFILTER_XT_TARGET_CHECKSUM NETFILTER_XT_TARGET_CLASSIFY NETFILTER_XT_TARGET_CONNMARK NETFILTER_XT_TARGET_CT NETFILTER_XT_TARGET_DSCP NETFILTER_XT_TARGET_HL NETFILTER_XT_TARGET_HMARK NETFILTER_XT_TARGET_IDLETIMER NETFILTER_XT_TARGET_LED NETFILTER_XT_TARGET_MARK NETFILTER_XT_TARGET_NETMAP NETFILTER_XT_TARGET_NFQUEUE NETFILTER_XT_TARGET_NOTRACK NETFILTER_XT_TARGET_RATEEST NETFILTER_XT_TARGET_REDIRECT NETFILTER_XT_TARGET_TCPOPTSTRIP NETFILTER_XT_TARGET_TEE NETFILTER_XT_TARGET_TPROXY NETFILTER_XT_TARGET_TRACE NETLINK_DIAG NETROM NET_9P_RDMA NET_ACT_BPF NET_ACT_CONNMARK NET_ACT_CSUM NET_ACT_CT NET_ACT_CTINFO NET_ACT_GATE NET_ACT_IFE NET_ACT_MPLS NET_ACT_NAT NET_ACT_PEDIT NET_ACT_POLICE NET_ACT_SAMPLE NET_ACT_SIMP NET_ACT_SKBEDIT NET_ACT_SKBMOD NET_ACT_TUNNEL_KEY NET_ACT_VLAN NET_CLS_BASIC NET_CLS_BPF NET_CLS_FLOW NET_CLS_FLOWER NET_CLS_FW NET_CLS_MATCHALL NET_CLS_ROUTE4 NET_CLS_U32 NET_DEVLINK NET_DEVMEM NET_DROP_MONITOR NET_DSA NET_DSA_TAG_BRCM NET_DSA_TAG_BRCM_COMMON NET_DSA_TAG_BRCM_PREPEND NET_DSA_TAG_MTK NET_DSA_TAG_QCA NET_DSA_TAG_RTL4_A NET_EMATCH_CANID NET_EMATCH_CMP NET_EMATCH_IPSET NET_EMATCH_IPT NET_EMATCH_META NET_EMATCH_NBYTE NET_EMATCH_TEXT NET_EMATCH_U32 NET_FC NET_FOU NET_FOU_IP_TUNNELS NET_IFE NET_IFE_SKBMARK NET_IFE_SKBPRIO NET_IFE_SKBTCINDEX NET_IPGRE NET_IPGRE_BROADCAST NET_IPGRE_DEMUX NET_IPIP NET_IPVTI NET_KEY NET_KEY_MIGRATE NET_L3_MASTER_DEV NET_MPLS_GSO NET_NCSI NET_NSH NET_REDIRECT NET_SCH_CAKE NET_SCH_CBS NET_SCH_CHOKE NET_SCH_CODEL NET_SCH_DEFAULT NET_SCH_DRR NET_SCH_ETF NET_SCH_ETS NET_SCH_FQ NET_SCH_FQ_CODEL NET_SCH_FQ_PIE NET_SCH_GRED NET_SCH_HFSC NET_SCH_HHF NET_SCH_HTB NET_SCH_INGRESS NET_SCH_MQPRIO NET_SCH_MQPRIO_LIB NET_SCH_MULTIQ NET_SCH_NETEM NET_SCH_PIE NET_SCH_PLUG NET_SCH_PRIO NET_SCH_QFQ NET_SCH_RED NET_SCH_SFB NET_SCH_SFQ NET_SCH_SKBPRIO NET_SCH_TAPRIO NET_SCH_TBF NET_SCH_TEQL NET_SHAPER NET_SOCK_MSG NET_SWITCHDEV NET_TC_SKB_EXT NET_TEAM NET_TEAM_MODE_ACTIVEBACKUP NET_TEAM_MODE_BROADCAST NET_TEAM_MODE_LOADBALANCE NET_TEAM_MODE_RANDOM NET_TEAM_MODE_ROUNDROBIN NET_UDP_TUNNEL NET_VRF NFC NFC_DIGITAL NFC_FDP NFC_HCI NFC_MRVL NFC_MRVL_USB NFC_NCI NFC_NCI_UART NFC_PN533 NFC_PN533_USB NFC_PORT100 NFC_SHDLC NFC_SIM NFC_VIRTUAL_NCI NFSD NFSD_BLOCKLAYOUT NFSD_FLEXFILELAYOUT NFSD_PNFS NFSD_SCSILAYOUT NFSD_V3_ACL NFSD_V4 NFSD_V4_2_INTER_SSC NFSD_V4_SECURITY_LABEL NFS_FSCACHE NFS_V4_1 NFS_V4_2 NFS_V4_2_READ_PLUS NFS_V4_2_SSC_HELPER NFS_V4_SECURITY_LABEL NFT_BRIDGE_META NFT_BRIDGE_REJECT NFT_COMPAT NFT_CONNLIMIT NFT_CT NFT_DUP_IPV4 NFT_DUP_IPV6 NFT_DUP_NETDEV NFT_FIB NFT_FIB_INET NFT_FIB_IPV4 NFT_FIB_IPV6 NFT_FIB_NETDEV NFT_FLOW_OFFLOAD NFT_HASH NFT_LIMIT NFT_LOG NFT_MASQ NFT_NAT NFT_NUMGEN NFT_OSF NFT_QUEUE NFT_QUOTA NFT_REDIR NFT_REJECT NFT_REJECT_INET NFT_REJECT_IPV4 NFT_REJECT_IPV6 NFT_REJECT_NETDEV NFT_SOCKET NFT_SYNPROXY NFT_TPROXY NFT_TUNNEL NFT_XFRM NF_CONNTRACK_AMANDA NF_CONNTRACK_BRIDGE NF_CONNTRACK_BROADCAST NF_CONNTRACK_EVENTS NF_CONNTRACK_H323 NF_CONNTRACK_LABELS NF_CONNTRACK_MARK NF_CONNTRACK_NETBIOS_NS NF_CONNTRACK_OVS NF_CONNTRACK_PPTP NF_CONNTRACK_SANE NF_CONNTRACK_SNMP NF_CONNTRACK_TFTP NF_CONNTRACK_TIMEOUT NF_CONNTRACK_TIMESTAMP NF_CONNTRACK_ZONES NF_CT_NETLINK_HELPER NF_CT_NETLINK_TIMEOUT NF_CT_PROTO_DCCP NF_CT_PROTO_GRE NF_CT_PROTO_SCTP NF_CT_PROTO_UDPLITE NF_DUP_IPV4 NF_DUP_IPV6 NF_DUP_NETDEV NF_FLOW_TABLE NF_FLOW_TABLE_INET NF_NAT_AMANDA NF_NAT_H323 NF_NAT_OVS NF_NAT_PPTP NF_NAT_REDIRECT NF_NAT_SNMP_BASIC NF_NAT_TFTP NF_SOCKET_IPV4 NF_SOCKET_IPV6 NF_TABLES NF_TABLES_BRIDGE NF_TABLES_INET NF_TABLES_IPV4 NF_TABLES_IPV6 NF_TABLES_NETDEV NF_TPROXY_IPV4 NF_TPROXY_IPV6 NILFS2_FS NINTENDO_FF NLMON NLS_CODEPAGE_1250 NLS_CODEPAGE_1251 NLS_CODEPAGE_737 NLS_CODEPAGE_775 NLS_CODEPAGE_850 NLS_CODEPAGE_852 NLS_CODEPAGE_855 NLS_CODEPAGE_857 NLS_CODEPAGE_860 NLS_CODEPAGE_861 NLS_CODEPAGE_862 NLS_CODEPAGE_863 NLS_CODEPAGE_864 NLS_CODEPAGE_865 NLS_CODEPAGE_866 PAGE_POOL PARTITION_ADVANCED PSAMPLE RC_CORE RC_DEVICES RFKILL SERIAL_DEV_BUS SPI TLS TLS_DEVICE TRANSPARENT_HUGEPAGE TRUSTED_KEYS USB_GADGET USB_MUSB_HDRC USB_PHY VIDEO_DEV VLAN_8021Q WAN WANT_COMPAT_NETLINK_MESSAGES WATCH_QUEUE WEXT_CORE WIRELESS WLAN WLAN_VENDOR_ATH WWAN X25 X86_X32_ABI ZONE_DEVICE]
disabling configs for [HANG LEAK UBSAN KASAN LOCKDEP ATOMIC_SLEEP], they are not needed
picked [v6.14 v6.13 v6.12 v6.10 v6.8 v6.6 v6.4 v6.2 v5.19 v5.16 v5.13 v5.10 v5.7 v5.4 v5.1 v4.19] out of 37 release tags
testing release v6.14
testing commit 38fec10eb60d687e30c8c6b5420d86e8149f7557 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 34697e0fff84919e86b691a3082237a0b4a80b5cfcb0ac8ba07259dc5e1668a5
all runs: OK
false negative chance: 0.000
# git bisect start 2df0c02dab829dd89360d98a8a1abaa026ef5798 38fec10eb60d687e30c8c6b5420d86e8149f7557
Bisecting: 501 revisions left to test after this (roughly 9 steps)
[f81c2b81508c4f479f2cf1ac0dbb138927dc6188] Merge tag 'docs-6.15' of git://git.lwn.net/linux
testing commit f81c2b81508c4f479f2cf1ac0dbb138927dc6188 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: b2a68259b75cece62abe0dd2618085efb68b3e86353c0ce8d81cc3787e666bdb
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: crashed: WARNING in call_rcu
run #10: crashed: WARNING in call_rcu
run #11: crashed: WARNING in call_rcu
run #12: OK
run #13: crashed: WARNING in call_rcu
run #14: OK
run #15: OK
run #16: OK
run #17: crashed: WARNING in call_rcu
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
# git bisect bad f81c2b81508c4f479f2cf1ac0dbb138927dc6188
Bisecting: 303 revisions left to test after this (roughly 8 steps)
[56e7a8b051512da8e7eb5c5a00dcc8e2065a89bb] Merge tag 'vfs-6.15-rc1.rust' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
testing commit 56e7a8b051512da8e7eb5c5a00dcc8e2065a89bb gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: ee283b2efac160fe605d57419dfe5ad786ddb6a5202f1a234512220acd93ed6a
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in delete_node
run #8: crashed: WARNING in call_rcu
run #9: OK
run #10: OK
run #11: OK
run #12: OK
run #13: crashed: WARNING in call_rcu
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
# git bisect bad 56e7a8b051512da8e7eb5c5a00dcc8e2065a89bb
Bisecting: 129 revisions left to test after this (roughly 7 steps)
[0ec0d4ecdd8bda4d55c5ba7b11b1595df36e3179] Merge tag 'vfs-6.15-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
testing commit 0ec0d4ecdd8bda4d55c5ba7b11b1595df36e3179 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 37c75c554d1137b100ec86e02a5e27a3679e13e1be0cd5954d58c03987e7543a
all runs: OK
false negative chance: 0.000
# git bisect good 0ec0d4ecdd8bda4d55c5ba7b11b1595df36e3179
Bisecting: 63 revisions left to test after this (roughly 6 steps)
[e41170cc5ef235a6949ea18edf1444e7f77968c3] Merge tag 'vfs-6.15-rc1.pagesize' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
testing commit e41170cc5ef235a6949ea18edf1444e7f77968c3 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 526384eb8104903cecea0c4b767cae18cedc35dcbdbebdf06c44eb501041fedf
all runs: OK
false negative chance: 0.000
# git bisect good e41170cc5ef235a6949ea18edf1444e7f77968c3
Bisecting: 33 revisions left to test after this (roughly 5 steps)
[c1c98301ece230bd874a27c957879fcc16ca1dbb] Merge tag 'vfs-6.15-rc1.initramfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
testing commit c1c98301ece230bd874a27c957879fcc16ca1dbb gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 442e42ebdf48c6cc4af7206eca90669b1445fc111227a55de9b75992a134c057
all runs: OK
false negative chance: 0.000
# git bisect good c1c98301ece230bd874a27c957879fcc16ca1dbb
Bisecting: 21 revisions left to test after this (roughly 4 steps)
[9483c37e2d1c48e45d1416327122ff6010ec7f8d] Merge tag 'vfs-6.15-rc1.afs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
testing commit 9483c37e2d1c48e45d1416327122ff6010ec7f8d gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 30761ebee7e928b7242c23959d168735bb6e999f17995291c61736b0d400212f
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: crashed: WARNING in call_rcu
run #10: crashed: WARNING in call_rcu
run #11: crashed: WARNING in call_rcu
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
# git bisect bad 9483c37e2d1c48e45d1416327122ff6010ec7f8d
Bisecting: 5 revisions left to test after this (roughly 3 steps)
[469c82b558628482002e4819d35b7670f0a989fd] afs: Drop the net parameter from afs_unuse_cell()
testing commit 469c82b558628482002e4819d35b7670f0a989fd gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 7e0f6c7c101f075d596256995a2d1cedaa0fe5b248caf373225d1f9505da1630
all runs: OK
false negative chance: 0.000
# git bisect good 469c82b558628482002e4819d35b7670f0a989fd
Bisecting: 2 revisions left to test after this (roughly 2 steps)
[4882ba78574e2d8c579658f65f6784b0d139d173] afs: Fix afs_server ref accounting
testing commit 4882ba78574e2d8c579658f65f6784b0d139d173 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: af460dc5ace2591df9b63aba85da394e5ba9f11daf50b7e7ad174f3510f4ead8
all runs: OK
false negative chance: 0.000
# git bisect good 4882ba78574e2d8c579658f65f6784b0d139d173
Bisecting: 0 revisions left to test after this (roughly 1 step)
[58a5937d50d800e15a8fc3ab9103583fc7b49ebf] Merge tag 'afs-next-20250310' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs
testing commit 58a5937d50d800e15a8fc3ab9103583fc7b49ebf gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 9a93dcd13a76c00c4d4b29485bc27a3f86d2320be5bd01c17a42db0ebc59b8a5
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: crashed: WARNING in call_rcu
run #7: crashed: WARNING in call_rcu
run #8: crashed: WARNING in call_rcu
run #9: OK
run #10: OK
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
# git bisect bad 58a5937d50d800e15a8fc3ab9103583fc7b49ebf
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[e2c2cb8ef07affd9f69497ea128fa801240fdf32] afs: Simplify cell record handling
testing commit e2c2cb8ef07affd9f69497ea128fa801240fdf32 gcc
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
kernel signature: 5a77cbff9124738d8e57d1f472e8e2bbe0c1f3fe1ba8d446eb0aa5bc598e9eed
run #0: crashed: WARNING in call_rcu
run #1: crashed: WARNING in call_rcu
run #2: crashed: WARNING in call_rcu
run #3: crashed: WARNING in call_rcu
run #4: crashed: WARNING in call_rcu
run #5: crashed: WARNING in call_rcu
run #6: OK
run #7: OK
run #8: OK
run #9: OK
run #10: crashed: WARNING in call_rcu
run #11: OK
run #12: OK
run #13: OK
run #14: OK
run #15: OK
run #16: OK
run #17: OK
run #18: OK
run #19: OK
representative crash: WARNING in call_rcu, types: [WARNING]
# git bisect bad e2c2cb8ef07affd9f69497ea128fa801240fdf32
e2c2cb8ef07affd9f69497ea128fa801240fdf32 is the first bad commit
commit e2c2cb8ef07affd9f69497ea128fa801240fdf32
Author: David Howells <dhowells@redhat.com>
Date: Mon Feb 24 16:06:03 2025 +0000
afs: Simplify cell record handling
Simplify afs_cell record handling to avoid very occasional races that cause
module removal to hang (it waits for all cell records to be removed).
There are two things that particularly contribute to the difficulty:
firstly, the code tries to pass a ref on the cell to the cell's maintenance
work item (which gets awkward if the work item is already queued); and,
secondly, there's an overall cell manager that tries to use just one timer
for the entire cell collection (to avoid having loads of timers). However,
both of these are probably unnecessarily restrictive.
To simplify this, the following changes are made:
(1) The cell record collection manager is removed. Each cell record
manages itself individually.
(2) Each afs_cell is given a second work item (cell->destroyer) that is
queued when its refcount reaches zero. This is not done in the
context of the putting thread as it might be in an inconvenient place
to sleep.
(3) Each afs_cell is given its own timer. The timer is used to expire the
cell record after a period of unuse if not otherwise pinned and can
also be used for other maintenance tasks if necessary (of which there
are currently none as DNS refresh is triggered by filesystem
operations).
(4) The afs_cell manager work item (cell->manager) is no longer given a
ref on the cell when queued; rather, the manager must be deleted.
This does away with the need to deal with the consequences of losing a
race to queue cell->manager. Clean up of extra queuing is deferred to
the destroyer.
(5) The cell destroyer work item makes sure the cell timer is removed and
that the normal cell work is cancelled before farming the actual
destruction off to RCU.
(6) When a network namespace is destroyed or the kafs module is unloaded,
it's now a simple matter of marking the namespace as dead then just
waking up all the cell work items. They will then remove and destroy
themselves once all remaining activity counts and/or a ref counts are
dropped. This makes sure that all server records are dropped first.
(7) The cell record state set is reduced to just four states: SETTING_UP,
ACTIVE, REMOVING and DEAD. The record persists in the active state
even when it's not being used until the time comes to remove it rather
than downgrading it to an inactive state from whence it can be
restored.
This means that the cell still appears in /proc and /afs when not in
use until it switches to the REMOVING state - at which point it is
removed.
Note that the REMOVING state is included so that someone wanting to
resurrect the cell record is forced to wait whilst the cell is torn
down in that state. Once it's in the DEAD state, it has been removed
from net->cells tree and is no longer findable and can be replaced.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Marc Dionne <marc.dionne@auristor.com>
cc: linux-afs@lists.infradead.org
cc: linux-fsdevel@vger.kernel.org
Link: https://lore.kernel.org/r/20250224234154.2014840-16-dhowells@redhat.com/ # v1
Link: https://lore.kernel.org/r/20250310094206.801057-12-dhowells@redhat.com/ # v4
fs/afs/cell.c | 404 ++++++++++++++++++---------------------------
fs/afs/dynroot.c | 4 +-
fs/afs/internal.h | 16 +-
fs/afs/main.c | 3 -
fs/afs/server.c | 8 +-
fs/afs/vl_rotate.c | 2 +-
include/trace/events/afs.h | 23 ++-
7 files changed, 187 insertions(+), 273 deletions(-)
accumulated error probability: 0.00
culprit signature: 5a77cbff9124738d8e57d1f472e8e2bbe0c1f3fe1ba8d446eb0aa5bc598e9eed
parent signature: af460dc5ace2591df9b63aba85da394e5ba9f11daf50b7e7ad174f3510f4ead8
reproducer is flaky (0.41 repro chance estimate)
revisions tested: 18, total time: 10h0m54.687255237s (build: 5h53m58.80324561s, test: 3h41m9.669736882s)
first bad commit: e2c2cb8ef07affd9f69497ea128fa801240fdf32 afs: Simplify cell record handling
recipients (to): ["dhowells@redhat.com" "dhowells@redhat.com" "linux-afs@lists.infradead.org" "linux-trace-kernel@vger.kernel.org" "marc.dionne@auristor.com" "mhiramat@kernel.org" "rostedt@goodmis.org"]
recipients (cc): ["linux-kernel@vger.kernel.org" "mathieu.desnoyers@efficios.com"]
crash: WARNING in call_rcu
------------[ cut here ]------------
ODEBUG: activate active (active state 0) object: ffff88816a8cbb78 object type: rcu_head hint: 0x0
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:615 debug_print_object lib/debugobjects.c:612 [inline]
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:615 debug_object_activate+0x226/0x2f0 lib/debugobjects.c:842
Modules linked in:
CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 6.14.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/12/2025
RIP: 0010:debug_print_object lib/debugobjects.c:612 [inline]
RIP: 0010:debug_object_activate+0x226/0x2f0 lib/debugobjects.c:842
Code: e9 48 8b 14 cd e0 9f 6e 84 4c 8b 4d 00 48 c7 c7 62 86 14 85 48 c7 c6 1c 45 16 85 8b 4c 24 04 50 e8 5f 5c ec fe 48 83 c4 08 90 <0f> 0b 90 90 ff 05 4c 66 64 03 bd ea ff ff ff 41 83 fd 05 74 06 41
RSP: 0018:ffffc900000e8dd0 EFLAGS: 00010292
RAX: c5feb509c284c200 RBX: ffff88816a8cbb78 RCX: ffff8881012f8000
RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffffff8462a020 R08: ffffffff814b48c4 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff88816a8cbb78
R13: 0000000000000003 R14: ffffffff8462a020 R15: ffffffff8b4b33d0
FS: 0000000000000000(0000) GS:ffff888237d00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000000567c000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
debug_rcu_head_queue kernel/rcu/rcu.h:224 [inline]
__call_rcu_common kernel/rcu/tree.c:3050 [inline]
call_rcu+0x3f/0x390 kernel/rcu/tree.c:3172
radix_tree_node_free lib/radix-tree.c:310 [inline]
delete_node+0xb8/0x340 lib/radix-tree.c:573
radix_tree_delete_item+0x156/0x220 lib/radix-tree.c:1430
afs_cell_destroy+0x80/0x130 fs/afs/cell.c:522
rcu_do_batch kernel/rcu/tree.c:2546 [inline]
rcu_core+0x3e5/0x810 kernel/rcu/tree.c:2802
handle_softirqs+0x10d/0x390 kernel/softirq.c:561
__do_softirq kernel/softirq.c:595 [inline]
invoke_softirq kernel/softirq.c:435 [inline]
__irq_exit_rcu+0x48/0x110 kernel/softirq.c:662
instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1049 [inline]
sysvec_apic_timer_interrupt+0x92/0xb0 arch/x86/kernel/apic/apic.c:1049
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:native_irq_disable arch/x86/include/asm/irqflags.h:37 [inline]
RIP: 0010:arch_local_irq_disable arch/x86/include/asm/irqflags.h:92 [inline]
RIP: 0010:acpi_safe_halt+0x21/0x30 drivers/acpi/processor_idle.c:112
Code: 90 90 90 90 90 90 90 90 90 65 48 8b 04 25 80 06 03 00 48 f7 00 08 00 00 00 75 10 66 90 0f 00 2d 9f d9 22 00 f3 0f 1e fa fb f4 <fa> c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90
RSP: 0018:ffffc900000b3e50 EFLAGS: 00000246
RAX: ffff8881012f8000 RBX: ffff888102297864 RCX: ffff888237d00000
RDX: 0000000000000001 RSI: ffff888102297800 RDI: ffff888102297864
RBP: ffff8881022b5800 R08: 0000001d942cb717 R09: 00000000000000ff
R10: 0000000000000000 R11: ffffffff844f1d50 R12: ffff8881022b5800
R13: ffffffff8595cbe0 R14: 0000000000000001 R15: ffffffff8595cb60
acpi_idle_enter+0xc1/0xe0 drivers/acpi/processor_idle.c:699
cpuidle_enter_state+0xc5/0x260 drivers/cpuidle/cpuidle.c:268
cpuidle_enter+0x2d/0x40 drivers/cpuidle/cpuidle.c:389
call_cpuidle kernel/sched/idle.c:155 [inline]
cpuidle_idle_call kernel/sched/idle.c:230 [inline]
do_idle+0x1a8/0x260 kernel/sched/idle.c:325
cpu_startup_entry+0x25/0x30 kernel/sched/idle.c:423
start_secondary+0x96/0xa0 arch/x86/kernel/smpboot.c:315
common_startup_64+0x12c/0x137
</TASK>
----------------
Code disassembly (best guess):
0: 90 nop
1: 90 nop
2: 90 nop
3: 90 nop
4: 90 nop
5: 90 nop
6: 90 nop
7: 90 nop
8: 90 nop
9: 65 48 8b 04 25 80 06 mov %gs:0x30680,%rax
10: 03 00
12: 48 f7 00 08 00 00 00 testq $0x8,(%rax)
19: 75 10 jne 0x2b
1b: 66 90 xchg %ax,%ax
1d: 0f 00 2d 9f d9 22 00 verw 0x22d99f(%rip) # 0x22d9c3
24: f3 0f 1e fa endbr64
28: fb sti
29: f4 hlt
* 2a: fa cli <-- trapping instruction
2b: c3 ret
2c: cc int3
2d: cc int3
2e: cc int3
2f: cc int3
30: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
37: 00 00
39: 90 nop
3a: 90 nop
3b: 90 nop
3c: 90 nop
3d: 90 nop
3e: 90 nop
3f: 90 nop