ci2 starts bisection 2023-09-16 18:26:07.268542979 +0000 UTC m=+295699.297717201 bisecting fixing commit since fa74641fb6b93a19ccb50579886ecc98320230f9 building syzkaller on f325deb023e4e2fb9197004be1b3da738680429c ensuring issue is reproducible on original commit fa74641fb6b93a19ccb50579886ecc98320230f9 testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 0982bc7783b255d5c46dab8d1c6c79d19b1a2aa6207989988c988c7d630b3552 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: crashed: INFO: rcu detected stall in corrupted run #8: crashed: INFO: rcu detected stall in corrupted run #9: crashed: INFO: rcu detected stall in corrupted run #10: crashed: INFO: rcu detected stall in corrupted run #11: crashed: INFO: rcu detected stall in corrupted run #12: crashed: INFO: rcu detected stall in corrupted run #13: crashed: invalid opcode in bpf_dispatcher_xdp run #14: crashed: general protection fault in bpf_dispatcher_xdp run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] check whether we can drop unnecessary instrumentation disabling configs for [BUG KASAN LOCKDEP ATOMIC_SLEEP LEAK UBSAN], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: c582c14586aa1d327430426e244874d7ebfd3a0c8019750dd189c4882f82629f run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: crashed: INFO: rcu detected stall in corrupted run #8: crashed: INFO: rcu detected stall in corrupted run #9: crashed: kernel panic: Fatal exception representative crash: INFO: rcu detected stall in corrupted, types: [HANG] the bug reproduces without the instrumentation disabling configs for [BUG KASAN LOCKDEP ATOMIC_SLEEP LEAK UBSAN], they are not needed kconfig minimization: base=3820 full=7521 leaves diff=1987 split chunks (needed=false): <1987> split chunk #0 of len 1987 into 5 parts testing without sub-chunk 1/5 disabling configs for [LEAK UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 2bb4750cb382442ceb814394c3fbcea6a2084454ecc8335d76b8e8a95a65fca2 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] testing without sub-chunk 2/5 disabling configs for [LEAK UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: d1ba62f209c1236a1c28131de75e5a5db2ea29237c456900435f0467be82fc76 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: invalid opcode in bpf_dispatcher_xdp run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: invalid opcode in bpf_dispatcher_xdp run #7: crashed: INFO: rcu detected stall in corrupted run #8: crashed: INFO: rcu detected stall in corrupted run #9: crashed: INFO: rcu detected stall in corrupted run #10: crashed: invalid opcode in bpf_dispatcher_xdp run #11: crashed: invalid opcode in xhci_gen_setup run #12: crashed: invalid opcode in bpf_dispatcher_xdp run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG UNKNOWN] the chunk can be dropped testing without sub-chunk 3/5 disabling configs for [LEAK UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 52ba968f88c2cd09027c45d4708df439be42cbf26cc9dcda5592b14b5e8b2cdd all runs: OK false negative chance: 0.000 testing without sub-chunk 4/5 disabling configs for [LEAK UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: db7285dd898e0bf285b8c1119e71b03ddf7f4987c39a86e97f4b286c4584ba37 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: OK run #8: crashed: INFO: rcu detected stall in corrupted run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] the chunk can be dropped testing without sub-chunk 5/5 disabling configs for [LEAK UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP], they are not needed testing commit fa74641fb6b93a19ccb50579886ecc98320230f9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 76dd2352e58d81c6a678aa4174a14cf8d8908ac926aa1bc985ca7dd3401e7164 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] minimized to 1191 configs; suspects: [6LOWPAN 6LOWPAN_GHC_EXT_HDR_DEST 6LOWPAN_GHC_EXT_HDR_FRAG 6LOWPAN_GHC_EXT_HDR_HOP 6LOWPAN_GHC_EXT_HDR_ROUTE 6LOWPAN_GHC_ICMPV6 6LOWPAN_GHC_UDP 6LOWPAN_NHC 6LOWPAN_NHC_DEST 6LOWPAN_NHC_FRAGMENT 6LOWPAN_NHC_HOP 6LOWPAN_NHC_IPV6 6LOWPAN_NHC_MOBILITY 6LOWPAN_NHC_ROUTING 6LOWPAN_NHC_UDP 6PACK 842_COMPRESS 842_DECOMPRESS 9P_FSCACHE 9P_FS_POSIX_ACL 9P_FS_SECURITY ACORN_PARTITION ACORN_PARTITION_ADFS ACORN_PARTITION_CUMANA ACORN_PARTITION_EESOX ACORN_PARTITION_ICS ACORN_PARTITION_POWERTEC ACORN_PARTITION_RISCIX ACPI_NFIT ACPI_PLATFORM_PROFILE ADFS_FS AFFS_FS AFS_FS AFS_FSCACHE AF_KCM AF_RXRPC AF_RXRPC_IPV6 AIX_PARTITION AMIGA_PARTITION ANDROID_BINDERFS ANDROID_BINDER_IPC ANON_VMA_NAME APERTURE_HELPERS AR5523 ARCH_ENABLE_MEMORY_HOTREMOVE ARCH_ENABLE_THP_MIGRATION ASM_MODVERSIONS ASYNC_CORE ASYNC_MEMCPY ASYNC_PQ ASYNC_RAID6_RECOV ASYNC_TX_DMA ASYNC_XOR ATARI_PARTITION ATA_GENERIC ATA_OVER_ETH ATH10K ATH10K_CE ATH10K_PCI ATH10K_USB ATH11K ATH6KL ATH6KL_USB ATH9K ATH9K_AHB ATH9K_BTCOEX_SUPPORT ATH9K_CHANNEL_CONTEXT ATH9K_COMMON ATH9K_COMMON_DEBUG ATH9K_DEBUGFS ATH9K_DYNACK ATH9K_HTC ATH9K_HTC_DEBUGFS ATH9K_HW ATH9K_PCI ATH9K_PCOEM ATH9K_RFKILL ATH_COMMON ATM ATM_BR2684 ATM_CLIP ATM_DRIVERS ATM_LANE ATM_MPOA ATM_TCP AUXILIARY_BUS AX25 AX25_DAMA_SLAVE AX88796B_PHY BAREUDP BATMAN_ADV BATMAN_ADV_BATMAN_V BATMAN_ADV_BLA BATMAN_ADV_DAT BATMAN_ADV_MCAST BATMAN_ADV_NC BCACHE BCMA BCMA_HOST_PCI_POSSIBLE BEFS_FS BFQ_CGROUP_DEBUG BFQ_GROUP_IOSCHED BFS_FS BIG_KEYS BLK_CGROUP_RWSTAT BLK_DEBUG_FS_ZONED BLK_DEV_BSGLIB BLK_DEV_FD BLK_DEV_INTEGRITY BLK_DEV_INTEGRITY_T10 BLK_DEV_NBD BLK_DEV_NULL_BLK BLK_DEV_NULL_BLK_FAULT_INJECTION BLK_DEV_NVME BLK_DEV_PMEM BLK_DEV_RAM BLK_DEV_RNBD BLK_DEV_RNBD_CLIENT BLK_DEV_THROTTLING BLK_DEV_ZONED BLK_ICQ BLK_INLINE_ENCRYPTION BLK_INLINE_ENCRYPTION_FALLBACK BLK_MQ_RDMA BLK_WBT BLK_WBT_MQ BONDING BOOT_VESA_SUPPORT BPF_EVENTS BPF_JIT BPF_JIT_ALWAYS_ON BPF_JIT_DEFAULT_ON BPF_LSM BPF_PRELOAD BPF_PRELOAD_UMD BPF_STREAM_PARSER BPF_SYSCALL BPQETHER BRIDGE BRIDGE_CFM BRIDGE_EBT_802_3 BRIDGE_EBT_AMONG BRIDGE_EBT_ARP BRIDGE_EBT_ARPREPLY BRIDGE_EBT_BROUTE BRIDGE_EBT_DNAT BRIDGE_EBT_IP BRIDGE_EBT_IP6 BRIDGE_EBT_LIMIT BRIDGE_EBT_LOG BRIDGE_EBT_MARK BRIDGE_EBT_MARK_T BRIDGE_EBT_NFLOG BRIDGE_EBT_PKTTYPE BRIDGE_EBT_REDIRECT BRIDGE_EBT_SNAT BRIDGE_EBT_STP BRIDGE_EBT_T_FILTER BRIDGE_EBT_T_NAT BRIDGE_EBT_VLAN BRIDGE_IGMP_SNOOPING BRIDGE_MRP BRIDGE_NETFILTER BRIDGE_NF_EBTABLES BRIDGE_VLAN_FILTERING BSD_DISKLABEL BSD_PROCESS_ACCT_V3 BT BTRFS_ASSERT BTRFS_FS BTRFS_FS_POSIX_ACL BTRFS_FS_REF_VERIFY BTT BT_6LOWPAN BT_ATH3K BT_BCM BT_BNEP BT_BNEP_MC_FILTER BT_BNEP_PROTO_FILTER BT_BREDR BT_CMTP BT_HCIBCM203X BT_HCIBFUSB BT_HCIBPA10X BT_HCIBTUSB BT_HCIBTUSB_BCM BT_HCIBTUSB_MTK BT_HCIBTUSB_RTL BT_HCIUART BT_HCIUART_3WIRE BT_HCIUART_AG6XX BT_HCIUART_BCSP BT_HCIUART_H4 BT_HCIUART_LL BT_HCIUART_MRVL BT_HCIUART_QCA BT_HCIUART_SERDEV BT_HCIVHCI BT_HIDP BT_HS BT_INTEL BT_LE BT_LEDS BT_MSFTEXT BT_MTK BT_QCA BT_RFCOMM BT_RFCOMM_TTY BT_RTL CACHEFILES CAIF CAIF_DEBUG CAIF_DRIVERS CAIF_NETDEV CAIF_TTY CAIF_USB CAIF_VIRTIO CAN CAN_8DEV_USB CAN_BCM CAN_CALC_BITTIMING CAN_DEV CAN_EMS_USB CAN_GS_USB CAN_GW CAN_IFI_CANFD CAN_ISOTP CAN_J1939 CAN_KVASER_USB CAN_MCBA_USB CAN_NETLINK CAN_PEAK_USB CAN_RAW CAN_SLCAN CAN_VCAN CAN_VXCAN CAPI_TRACE CARDBUS CARL9170 CARL9170_HWRNG CARL9170_LEDS CARL9170_WPC CC_HAS_ZERO_CALL_USED_REGS CEC_CORE CEPH_FS CEPH_FSCACHE CEPH_FS_POSIX_ACL CEPH_LIB CEPH_LIB_USE_DNS_RESOLVER CFG80211 CFG80211_CRDA_SUPPORT CFG80211_DEBUGFS CFG80211_DEFAULT_PS CFG80211_REQUIRE_SIGNED_REGDB CFG80211_USE_KERNEL_REGDB_KEYS CFG80211_WEXT CFS_BANDWIDTH CGROUP_BPF CHARGER_BQ24190 CHARGER_ISP1704 CHECKPOINT_RESTORE CHR_DEV_ST CIFS CIFS_ALLOW_INSECURE_LEGACY CIFS_DEBUG CIFS_DFS_UPCALL CIFS_FSCACHE CIFS_POSIX CIFS_SMB_DIRECT CIFS_SWN_UPCALL CIFS_UPCALL CIFS_XATTR CLS_U32_MARK CLS_U32_PERF CMA CMA_SIZE_SEL_MBYTES CMDLINE_PARTITION COMEDI COMEDI_DT9812 COMEDI_NI_USB6501 COMEDI_USBDUX COMEDI_USBDUXFAST COMEDI_USBDUXSIGMA COMEDI_USB_DRIVERS COMEDI_VMK80XX COMMON_CLK COMPAT_NETLINK_MESSAGES CONTIG_ALLOC COUNTER CRAMFS CRAMFS_BLOCKDEV CRAMFS_MTD CRC4 CRC64 CRC64_ROCKSOFT CRC7 CRC8 CRC_ITU_T CRC_T10DIF CRYPTO_842 CRYPTO_ADIANTUM CRYPTO_AEGIS128 CRYPTO_AEGIS128_AESNI_SSE2 CRYPTO_AES_NI_INTEL CRYPTO_AES_TI CRYPTO_ANSI_CPRNG CRYPTO_ANUBIS CRYPTO_ARC4 CRYPTO_ARCH_HAVE_LIB_BLAKE2S CRYPTO_ARCH_HAVE_LIB_CHACHA CRYPTO_ARCH_HAVE_LIB_CURVE25519 CRYPTO_ARCH_HAVE_LIB_POLY1305 CRYPTO_ARIA CRYPTO_ARIA_AESNI_AVX_X86_64 CRYPTO_BLAKE2B CRYPTO_BLAKE2S_X86 CRYPTO_BLOWFISH CRYPTO_BLOWFISH_COMMON CRYPTO_BLOWFISH_X86_64 CRYPTO_CAMELLIA CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 CRYPTO_CAMELLIA_AESNI_AVX_X86_64 CRYPTO_CAMELLIA_X86_64 CRYPTO_CAST5 CRYPTO_CAST5_AVX_X86_64 CRYPTO_CAST6 CRYPTO_CAST6_AVX_X86_64 CRYPTO_CAST_COMMON CRYPTO_CFB CRYPTO_CHACHA20 CRYPTO_CHACHA20POLY1305 CRYPTO_CHACHA20_X86_64 CRYPTO_CRC32 CRYPTO_CRC32C_INTEL CRYPTO_CRC32_PCLMUL CRYPTO_CRC64_ROCKSOFT CRYPTO_CRCT10DIF CRYPTO_CRCT10DIF_PCLMUL CRYPTO_CRYPTD CRYPTO_CTS CRYPTO_CURVE25519 CRYPTO_CURVE25519_X86 CRYPTO_DEFLATE CRYPTO_DES CRYPTO_DES3_EDE_X86_64 CRYPTO_DEV_CCP CRYPTO_DEV_CCP_DD CRYPTO_DEV_PADLOCK CRYPTO_DEV_PADLOCK_AES CRYPTO_DEV_PADLOCK_SHA CRYPTO_DEV_QAT CRYPTO_DEV_QAT_C3XXX CRYPTO_DEV_QAT_C3XXXVF CRYPTO_DEV_QAT_C62X CRYPTO_DEV_QAT_C62XVF CRYPTO_DEV_QAT_DH895xCC CRYPTO_DEV_QAT_DH895xCCVF CRYPTO_DEV_VIRTIO CRYPTO_DH CRYPTO_DRBG_CTR CRYPTO_DRBG_HASH CRYPTO_ECB CRYPTO_ECC CRYPTO_ECDH CRYPTO_ECRDSA CRYPTO_ENGINE CRYPTO_ESSIV CRYPTO_FCRYPT CRYPTO_GHASH_CLMUL_NI_INTEL CRYPTO_HCTR2 CRYPTO_KDF800108_CTR CRYPTO_KEYWRAP CRYPTO_KHAZAD CRYPTO_KPP CRYPTO_LIB_ARC4 CRYPTO_LIB_CHACHA CRYPTO_LIB_CHACHA20POLY1305 CRYPTO_LIB_CHACHA_GENERIC CRYPTO_LIB_CURVE25519 CRYPTO_LIB_CURVE25519_GENERIC CRYPTO_LIB_DES CRYPTO_LIB_POLY1305 CRYPTO_LIB_POLY1305_GENERIC CRYPTO_LRW CRYPTO_LZ4 CRYPTO_LZ4HC CRYPTO_LZO CRYPTO_MICHAEL_MIC CRYPTO_NHPOLY1305 CRYPTO_NHPOLY1305_AVX2 CRYPTO_NHPOLY1305_SSE2 CRYPTO_OFB CRYPTO_PCBC CRYPTO_PCRYPT CRYPTO_POLY1305 CRYPTO_POLY1305_X86_64 CRYPTO_POLYVAL CRYPTO_POLYVAL_CLMUL_NI CRYPTO_RMD160 CRYPTO_SEED CRYPTO_SERPENT CRYPTO_SERPENT_AVX2_X86_64 CRYPTO_SERPENT_AVX_X86_64 CRYPTO_SERPENT_SSE2_X86_64 CRYPTO_SHA1_SSSE3 CRYPTO_SHA256_SSSE3 CRYPTO_SHA3 CRYPTO_SHA512_SSSE3 CRYPTO_SIMD CRYPTO_SM2 CRYPTO_SM3 CRYPTO_SM3_AVX_X86_64 CRYPTO_SM4 CRYPTO_SM4_AESNI_AVX2_X86_64 CRYPTO_SM4_AESNI_AVX_X86_64 CRYPTO_SM4_GENERIC CRYPTO_STREEBOG CRYPTO_TEA CRYPTO_TWOFISH CRYPTO_TWOFISH_AVX_X86_64 CRYPTO_TWOFISH_COMMON CRYPTO_TWOFISH_X86_64 CRYPTO_TWOFISH_X86_64_3WAY CRYPTO_USER CRYPTO_USER_API CRYPTO_USER_API_AEAD CRYPTO_USER_API_ENABLE_OBSOLETE CRYPTO_USER_API_HASH CRYPTO_USER_API_RNG CRYPTO_USER_API_SKCIPHER CRYPTO_VMAC CRYPTO_WP512 CRYPTO_XCBC CRYPTO_XCTR CRYPTO_XTS CRYPTO_XXHASH CRYPTO_ZSTD CUSE CYPRESS_FIRMWARE DAMON DAMON_DBGFS DAMON_PADDR DAMON_RECLAIM DAMON_VADDR DAX DCB DEFAULT_PFIFO_FAST DEVICE_MIGRATION DEVICE_PRIVATE DEV_COREDUMP DEV_DAX DIMLIB DLN2_ADC DMABUF_HEAPS DMABUF_HEAPS_CMA DMABUF_HEAPS_SYSTEM DMABUF_MOVE_NOTIFY DMA_CMA DM_AUDIT DM_BIO_PRISON DM_BUFIO DM_CACHE DM_CACHE_SMQ DM_CLONE DM_CRYPT DM_FLAKEY DM_INTEGRITY DM_MULTIPATH DM_MULTIPATH_QL DM_MULTIPATH_ST DM_PERSISTENT_DATA DM_RAID DM_SNAPSHOT DM_THIN_PROVISIONING DM_UEVENT DM_VERITY DM_VERITY_FEC DM_WRITECACHE DM_ZONED DRAGONRISE_FF DRM DRM_BOCHS DRM_BUDDY DRM_CIRRUS_QEMU DRM_DEBUG_MM DRM_DISPLAY_DP_HELPER DRM_DISPLAY_HELPER DVB_CORE ENCRYPTED_KEYS EXTCON FB FSCACHE FUSE_FS GPIOLIB HAMRADIO HID_DRAGONRISE HID_ZEROPLUS HSR I2C_MUX IIO INFINIBAND INFINIBAND_ADDR_TRANS INFINIBAND_RTRS_CLIENT INFINIBAND_USER_ACCESS INPUT_JOYSTICK INPUT_MOUSE IOSCHED_BFQ IP6_NF_RAW IPV6_MULTIPLE_TABLES IP_NF_RAW IP_SET IP_VS IP_VS_SED IP_VS_SH IP_VS_TWOS IP_VS_WLC IP_VS_WRR IRQ_BYPASS_MANAGER IRQ_POLL IR_IGORPLUGUSB IR_IGUANA IR_IMON IR_MCEUSB IR_REDRAT3 IR_STREAMZAP IR_TTUSBIR ISDN ISDN_CAPI ISDN_CAPI_MIDDLEWARE JFFS2_CMODE_PRIORITY JFFS2_COMPRESSION_OPTIONS JFFS2_FS JFFS2_FS_POSIX_ACL JFFS2_FS_SECURITY JFFS2_FS_WRITEBUFFER JFFS2_FS_XATTR JFFS2_LZO JFFS2_RTIME JFFS2_RUBIN JFFS2_SUMMARY JFFS2_ZLIB JFS_DEBUG JFS_FS JFS_POSIX_ACL JFS_SECURITY JOYSTICK_IFORCE JOYSTICK_IFORCE_USB JOYSTICK_XPAD JOYSTICK_XPAD_FF JOYSTICK_XPAD_LEDS KARMA_PARTITION KCOV KCOV_ENABLE_COMPARISONS KCOV_INSTRUMENT_ALL KEYS_REQUEST_CACHE KEY_DH_OPERATIONS KEY_NOTIFICATIONS KSM KVM KVM_AMD KVM_ASYNC_PF KVM_COMPAT KVM_GENERIC_DIRTYLOG_READ_PROTECT KVM_INTEL KVM_MMIO KVM_VFIO KVM_XEN KVM_XFER_TO_GUEST_WORK L2TP L2TP_ETH L2TP_IP L2TP_V3 LAPB LAPBETHER LDM_PARTITION LEDS_TRIGGER_AUDIO LEGACY_PTYS LIBCRC32C LIBNVDIMM LINEAR_RANGES LLC LLC2 LOGIG940_FF LOGIRUMBLEPAD2_FF LOGO LOGO_LINUX_MONO LOGO_LINUX_VGA16 LPC_ICH LRU_GEN LRU_GEN_ENABLED LWTUNNEL LWTUNNEL_BPF LZ4HC_COMPRESS LZ4_COMPRESS MAC80211 MAC80211_DEBUGFS MAC80211_HAS_RC MAC80211_HWSIM MAC80211_LEDS MAC80211_MESH MAC80211_RC_DEFAULT_MINSTREL MAC80211_RC_MINSTREL MACSEC MACVLAN MACVTAP MAC_PARTITION MAPPING_DIRTY_HELPERS MD_LINEAR MD_MULTIPATH MD_RAID0 MD_RAID1 MD_RAID10 MD_RAID456 MEDIA_ANALOG_TV_SUPPORT MEDIA_ATTACH MEDIA_CAMERA_SUPPORT MEDIA_CEC_SUPPORT MEDIA_CONTROLLER MEDIA_CONTROLLER_DVB MEDIA_CONTROLLER_REQUEST_API MEDIA_DIGITAL_TV_SUPPORT MEDIA_RADIO_SUPPORT MEDIA_SDR_SUPPORT MEDIA_SUPPORT MEDIA_SUPPORT_FILTER MEDIA_TEST_SUPPORT MEDIA_TUNER MEDIA_TUNER_MSI001 MEDIA_USB_SUPPORT MEMORY_BALLOON MEMORY_HOTPLUG MEMORY_HOTPLUG_DEFAULT_ONLINE MEMORY_HOTREMOVE MEMORY_ISOLATION MEMREGION MEMSTICK MEMSTICK_REALTEK_USB MEM_SOFT_DIRTY MFD_CORE MFD_DLN2 MFD_SYSCON MFD_VIPERBOARD MHI_BUS MHI_WWAN_CTRL MHP_MEMMAP_ON_MEMORY MICROCHIP_PHY MINIX_FS MINIX_SUBPARTITION MISC_RTSX MISC_RTSX_USB MISDN MISDN_DSP MISDN_HFCUSB MISDN_L1OIP MKISS MLX4_CORE MLX4_INFINIBAND MMC MMC_REALTEK_USB MMC_USHC MMC_VUB300 MMU_NOTIFIER MODULE_SRCVERSION_ALL MODVERSIONS MOST MOUSE_APPLETOUCH MOUSE_BCM5974 MOUSE_PS2 MOUSE_PS2_ALPS MOUSE_PS2_BYD MOUSE_PS2_CYPRESS MOUSE_PS2_FOCALTECH MOUSE_PS2_LIFEBOOK MOUSE_PS2_LOGIPS2PP MOUSE_PS2_SMBUS MOUSE_PS2_SYNAPTICS MOUSE_PS2_SYNAPTICS_SMBUS MOUSE_PS2_TRACKPOINT MOUSE_SYNAPTICS_USB MPLS MPLS_IPTUNNEL MPLS_ROUTING MPTCP MPTCP_IPV6 MRP MTD MTD_BLKDEVS MTD_BLOCK MTD_BLOCK2MTD MTD_CFI_I1 MTD_CFI_I2 MTD_MAP_BANK_WIDTH_1 MTD_MAP_BANK_WIDTH_2 MTD_MAP_BANK_WIDTH_4 MTD_MTDRAM MTD_PHRAM MTD_SLRAM MUSB_PIO_ONLY ND_BTT ND_CLAIM ND_PFN NETDEVSIM NETFILTER_ADVANCED NETFILTER_FAMILY_ARP NETFILTER_FAMILY_BRIDGE NETFILTER_NETLINK_ACCT NETFILTER_NETLINK_GLUE_CT NETFILTER_NETLINK_OSF NETFILTER_NETLINK_QUEUE NETFILTER_SYNPROXY NETFILTER_XT_CONNMARK NETFILTER_XT_MATCH_BPF NETFILTER_XT_MATCH_CGROUP NETFILTER_XT_MATCH_CLUSTER NETFILTER_XT_MATCH_COMMENT NETFILTER_XT_MATCH_CONNBYTES NETFILTER_XT_MATCH_CONNLABEL NETFILTER_XT_MATCH_CONNLIMIT NETFILTER_XT_MATCH_CONNMARK NETFILTER_XT_MATCH_CPU NETFILTER_XT_MATCH_DCCP NETFILTER_XT_MATCH_DEVGROUP NETFILTER_XT_MATCH_DSCP NETFILTER_XT_MATCH_ECN NETFILTER_XT_MATCH_ESP NETFILTER_XT_MATCH_HASHLIMIT NETFILTER_XT_MATCH_HELPER NETFILTER_XT_MATCH_HL NETFILTER_XT_MATCH_IPCOMP NETFILTER_XT_MATCH_IPRANGE NETFILTER_XT_MATCH_IPVS NETFILTER_XT_MATCH_L2TP NETFILTER_XT_MATCH_LENGTH NETFILTER_XT_MATCH_LIMIT NETFILTER_XT_MATCH_MAC NETFILTER_XT_MATCH_MARK NETFILTER_XT_MATCH_MULTIPORT NETFILTER_XT_MATCH_NFACCT NETFILTER_XT_MATCH_OSF NETFILTER_XT_MATCH_OWNER NETFILTER_XT_MATCH_PHYSDEV NETFILTER_XT_MATCH_PKTTYPE NETFILTER_XT_MATCH_QUOTA NETFILTER_XT_MATCH_RATEEST NETFILTER_XT_MATCH_REALM NETFILTER_XT_MATCH_RECENT NETFILTER_XT_MATCH_SCTP NETFILTER_XT_MATCH_SOCKET NETFILTER_XT_MATCH_STATISTIC NETFILTER_XT_MATCH_STRING NETFILTER_XT_MATCH_TCPMSS NETFILTER_XT_MATCH_TIME NETFILTER_XT_MATCH_U32 NETFILTER_XT_SET NETFILTER_XT_TARGET_AUDIT NETFILTER_XT_TARGET_CHECKSUM NETFILTER_XT_TARGET_CLASSIFY NETFILTER_XT_TARGET_CONNMARK NETFILTER_XT_TARGET_CT NETFILTER_XT_TARGET_DSCP NETFILTER_XT_TARGET_HL NETFILTER_XT_TARGET_HMARK NETFILTER_XT_TARGET_IDLETIMER NETFILTER_XT_TARGET_LED NETFILTER_XT_TARGET_MARK NETFILTER_XT_TARGET_NETMAP NETFILTER_XT_TARGET_NFQUEUE NETFILTER_XT_TARGET_NOTRACK NETFILTER_XT_TARGET_RATEEST NETFILTER_XT_TARGET_REDIRECT NETFILTER_XT_TARGET_TCPOPTSTRIP NETFILTER_XT_TARGET_TEE NETFILTER_XT_TARGET_TPROXY NETFILTER_XT_TARGET_TRACE NETLABEL NETLINK_DIAG NETROM NET_9P_RDMA NET_ACT_BPF NET_ACT_CONNMARK NET_ACT_CSUM NET_ACT_CT NET_ACT_CTINFO NET_ACT_GATE NET_ACT_IFE NET_ACT_IPT NET_ACT_MPLS NET_ACT_NAT NET_ACT_PEDIT NET_ACT_POLICE NET_ACT_SAMPLE NET_ACT_SIMP NET_ACT_SKBEDIT NET_ACT_SKBMOD NET_ACT_TUNNEL_KEY NET_ACT_VLAN NET_CLS_BASIC NET_CLS_BPF NET_CLS_FLOW NET_CLS_FLOWER NET_CLS_FW NET_CLS_MATCHALL NET_CLS_ROUTE4 NET_CLS_RSVP NET_CLS_RSVP6 NET_CLS_U32 NET_DEVLINK NET_DROP_MONITOR NET_DSA NET_DSA_TAG_BRCM NET_DSA_TAG_BRCM_COMMON NET_DSA_TAG_BRCM_PREPEND NET_DSA_TAG_MTK NET_DSA_TAG_QCA NET_DSA_TAG_RTL4_A NET_EMATCH_CANID NET_EMATCH_CMP NET_EMATCH_IPSET NET_EMATCH_IPT NET_EMATCH_META NET_EMATCH_NBYTE NET_EMATCH_TEXT NET_EMATCH_U32 NET_FC NET_FOU NET_FOU_IP_TUNNELS NET_IFE NET_IFE_SKBMARK NET_IFE_SKBPRIO NET_IFE_SKBTCINDEX NET_IPGRE NET_IPGRE_BROADCAST NET_IPGRE_DEMUX NET_IPIP NET_IPVTI NET_KEY NET_KEY_MIGRATE NET_L3_MASTER_DEV NET_MPLS_GSO NET_NCSI NET_NSH NET_REDIRECT NET_SCH_ATM NET_SCH_CAKE NET_SCH_CBQ NET_SCH_CBS NET_SCH_CHOKE NET_SCH_CODEL NET_SCH_DEFAULT NET_SCH_DRR NET_SCH_DSMARK NET_SCH_ETF NET_SCH_ETS NET_SCH_FQ NET_SCH_FQ_CODEL NET_SCH_FQ_PIE NET_SCH_GRED NET_SCH_HFSC NET_SCH_HHF NET_SCH_HTB NET_SCH_INGRESS NET_SCH_MQPRIO NET_SCH_MULTIQ NET_SCH_NETEM NET_SCH_PIE NET_SCH_PLUG NET_SCH_PRIO NET_SCH_QFQ NET_SCH_RED NET_SCH_SFB NET_SCH_SFQ NET_SCH_SKBPRIO NET_SCH_TAPRIO NET_SCH_TBF NET_SCH_TEQL NET_SOCK_MSG NET_SWITCHDEV NET_TC_SKB_EXT NET_TEAM NET_TEAM_MODE_ACTIVEBACKUP NET_TEAM_MODE_BROADCAST NET_TEAM_MODE_LOADBALANCE NET_TEAM_MODE_RANDOM NET_TEAM_MODE_ROUNDROBIN NET_UDP_TUNNEL NET_VRF NFC NFC_DIGITAL NFC_FDP NFC_HCI NFC_MRVL NFC_MRVL_USB NFC_NCI NFC_NCI_UART NFC_PN533 NFC_PN533_USB NFC_PORT100 NFC_SHDLC NFC_SIM NFC_VIRTUAL_NCI NFSD NFSD_BLOCKLAYOUT NFSD_FLEXFILELAYOUT NFSD_PNFS NFSD_SCSILAYOUT NFSD_V2_ACL NFSD_V3_ACL NFSD_V4 NFSD_V4_2_INTER_SSC NFSD_V4_SECURITY_LABEL NFS_FSCACHE NFS_V4_1 NFS_V4_2 NFS_V4_2_READ_PLUS NFS_V4_2_SSC_HELPER NFS_V4_SECURITY_LABEL NFT_BRIDGE_META NFT_BRIDGE_REJECT NFT_COMPAT NFT_CONNLIMIT NFT_CT NFT_DUP_IPV4 NFT_DUP_IPV6 NFT_DUP_NETDEV NFT_FIB NFT_FIB_INET NFT_FIB_IPV4 NFT_FIB_IPV6 NFT_FIB_NETDEV NFT_FLOW_OFFLOAD NFT_HASH NFT_LIMIT NFT_LOG NFT_MASQ NFT_NAT NFT_NUMGEN NFT_OSF NFT_QUEUE NFT_QUOTA NFT_REDIR NFT_REJECT NFT_REJECT_INET NFT_REJECT_IPV4 NFT_REJECT_IPV6 NFT_REJECT_NETDEV NFT_SOCKET NFT_SYNPROXY NFT_TPROXY NFT_TUNNEL NFT_XFRM NF_CONNTRACK_AMANDA NF_CONNTRACK_BRIDGE NF_CONNTRACK_BROADCAST NF_CONNTRACK_EVENTS NF_CONNTRACK_H323 NF_CONNTRACK_LABELS NF_CONNTRACK_MARK NF_CONNTRACK_NETBIOS_NS NF_CONNTRACK_PPTP NF_CONNTRACK_SANE NF_CONNTRACK_SNMP NF_CONNTRACK_TFTP NF_CONNTRACK_TIMEOUT NF_CONNTRACK_TIMESTAMP NF_CONNTRACK_ZONES NF_CT_NETLINK_HELPER NF_CT_NETLINK_TIMEOUT NF_CT_PROTO_DCCP NF_CT_PROTO_GRE NF_CT_PROTO_SCTP NF_CT_PROTO_UDPLITE NF_DUP_IPV4 NF_DUP_IPV6 NF_DUP_NETDEV NF_FLOW_TABLE NF_FLOW_TABLE_INET NF_NAT_AMANDA NF_NAT_H323 NF_NAT_PPTP NF_NAT_REDIRECT NF_NAT_SNMP_BASIC NF_NAT_TFTP NF_SOCKET_IPV4 NF_SOCKET_IPV6 NF_TABLES NF_TABLES_ARP NF_TABLES_BRIDGE NF_TABLES_INET NF_TABLES_IPV4 NF_TABLES_IPV6 NF_TABLES_NETDEV NF_TPROXY_IPV4 NF_TPROXY_IPV6 PARPORT PARTITION_ADVANCED PCCARD PCMCIA PHONET PSAMPLE RADIO_ADAPTERS RADIO_SI470X RADIO_SI4713 RC_CORE RC_DEVICES RFKILL SERIAL_DEV_BUS SND SOUND SPI SSB TAP TLS TLS_DEVICE TRANSPARENT_HUGEPAGE TRUSTED_KEYS TUN USB_AMD5536UDC USB_ATM USB_CONFIGFS USB_CONFIGFS_F_UAC1 USB_CONFIGFS_F_UAC1_LEGACY USB_CONFIGFS_F_UAC2 USB_CONFIGFS_F_UVC USB_CONFIGFS_MASS_STORAGE USB_CONFIGFS_NCM USB_CONFIGFS_OBEX USB_CONFIGFS_PHONET USB_CONFIGFS_RNDIS USB_CONFIGFS_SERIAL USB_CXACRU USB_CYPRESS_CY7C63 USB_CYTHERM USB_DSBR USB_DUMMY_HCD USB_DWC2 USB_DWC2_HOST USB_DWC2_PCI USB_DWC3 USB_DWC3_GADGET USB_DWC3_OF_SIMPLE USB_DWC3_PCI USB_DWC3_ULPI USB_DYNAMIC_MINORS USB_EG20T USB_EHCI_HCD_PLATFORM USB_EHCI_ROOT_HUB_TT USB_EHSET_TEST_FIXTURE USB_EMI26 USB_EMI62 USB_EPSON2888 USB_EZUSB_FX2 USB_FEW_INIT_RETRIES USB_FOTG210_HCD USB_FOTG210_UDC USB_FTDI_ELAN USB_F_ACM USB_F_ECM USB_F_EEM USB_F_FS USB_F_HID USB_F_MASS_STORAGE USB_F_MIDI USB_F_NCM USB_F_OBEX USB_F_PHONET USB_F_PRINTER USB_F_RNDIS USB_F_SERIAL USB_F_SS_LB USB_F_SUBSET USB_F_TCM USB_F_UAC1 USB_F_UAC1_LEGACY USB_F_UAC2 USB_F_UVC USB_GADGET USB_GADGETFS USB_GADGET_DEBUG_FILES USB_GADGET_DEBUG_FS USB_GL860 USB_GOKU USB_GPIO_VBUS USB_GR_UDC USB_GSPCA USB_GSPCA_BENQ USB_GSPCA_CONEX USB_GSPCA_CPIA1 USB_GSPCA_DTCS033 USB_GSPCA_ETOMS USB_GSPCA_FINEPIX USB_GSPCA_JEILINJ USB_GSPCA_JL2005BCD USB_GSPCA_KINECT USB_GSPCA_KONICA USB_GSPCA_MARS USB_GSPCA_MR97310A USB_GSPCA_NW80X USB_GSPCA_OV519 USB_GSPCA_OV534 USB_GSPCA_OV534_9 USB_GSPCA_PAC207 USB_GSPCA_PAC7302 USB_GSPCA_PAC7311 USB_GSPCA_SE401 USB_GSPCA_SN9C2028 USB_GSPCA_SN9C20X USB_GSPCA_SONIXB USB_GSPCA_SONIXJ USB_GSPCA_SPCA1528 USB_GSPCA_SPCA500 USB_GSPCA_SPCA501 USB_GSPCA_SPCA505 USB_GSPCA_SPCA506 USB_GSPCA_SPCA508 USB_GSPCA_SPCA561 USB_GSPCA_SQ905 USB_GSPCA_SQ905C USB_GSPCA_SQ930X USB_GSPCA_STK014 USB_GSPCA_STK1135 USB_GSPCA_STV0680 USB_GSPCA_SUNPLUS USB_GSPCA_T613 USB_GSPCA_TOPRO USB_GSPCA_TOUPTEK USB_GSPCA_TV8532 USB_GSPCA_VC032X USB_GSPCA_VICAM USB_GSPCA_XIRLINK_CIT USB_GSPCA_ZC3XX USB_HACKRF USB_HCD_BCMA USB_HCD_SSB USB_HSIC_USB3503 USB_HSIC_USB4604 USB_HSO USB_HUB_USB251XB USB_IDMOUSE USB_IOWARRIOR USB_IPHETH USB_ISIGHTFW USB_ISP116X_HCD USB_ISP1301 USB_ISP1760 USB_ISP1760_DUAL_ROLE USB_ISP1760_HCD USB_ISP1761_UDC USB_KAWETH USB_KC2190 USB_KEENE USB_LAN78XX USB_LCD USB_LD USB_LEDS_TRIGGER_USBPORT USB_LED_TRIG USB_LEGOTOWER USB_LIBCOMPOSITE USB_LINK_LAYER_TEST USB_M5602 USB_MA901 USB_MAX3421_HCD USB_MDC800 USB_MICROTEK USB_MR800 USB_MSI2500 USB_MUSB_DUAL_ROLE USB_MUSB_HDRC USB_MV_U3D USB_MV_UDC USB_NET2272 USB_NET2272_DMA USB_NET2280 USB_NET_AX88179_178A USB_NET_AX8817X USB_NET_CDCETHER USB_NET_CDC_EEM USB_NET_CDC_MBIM USB_NET_CDC_NCM USB_NET_CDC_SUBSET USB_NET_CDC_SUBSET_ENABLE USB_NET_CH9200 USB_NET_CX82310_ETH USB_NET_DM9601 USB_NET_GL620A USB_NET_HUAWEI_CDC_NCM USB_NET_INT51X1 USB_NET_KALMIA USB_NET_MCS7830 USB_NET_NET1080 USB_NET_PLUSB USB_NET_QMI_WWAN USB_NET_RNDIS_HOST USB_NET_RNDIS_WLAN USB_NET_SMSC75XX USB_NET_SMSC95XX USB_NET_SR9700 USB_NET_SR9800 USB_NET_ZAURUS USB_OHCI_HCD_PLATFORM USB_OTG USB_OTG_FSM USB_OXU210HP_HCD USB_PEGASUS USB_PHY USB_PULSE8_CEC USB_PWC USB_PWC_INPUT_EVDEV USB_PXA27X USB_R8A66597 USB_R8A66597_HCD USB_RAINSHADOW_CEC USB_RAREMONO USB_RAW_GADGET USB_RTL8150 USB_RTL8152 USB_RTL8153_ECM USB_S2255 USB_SERIAL USB_SERIAL_AIRCABLE USB_SERIAL_ARK3116 USB_SERIAL_BELKIN USB_SERIAL_CH341 USB_SERIAL_CONSOLE USB_SERIAL_CP210X USB_SERIAL_CYBERJACK USB_SERIAL_CYPRESS_M8 USB_SERIAL_DEBUG USB_SERIAL_DIGI_ACCELEPORT USB_SERIAL_EDGEPORT USB_SERIAL_EDGEPORT_TI USB_SERIAL_EMPEG USB_SERIAL_F81232 USB_SERIAL_F8153X USB_SERIAL_FTDI_SIO USB_SERIAL_GARMIN USB_SERIAL_GENERIC USB_SERIAL_IPAQ USB_SERIAL_IPW USB_SERIAL_IR USB_SERIAL_IUU USB_SERIAL_KEYSPAN USB_SERIAL_KEYSPAN_PDA USB_SERIAL_KLSI USB_SERIAL_KOBIL_SCT USB_SERIAL_MCT_U232 USB_SERIAL_METRO USB_SERIAL_MOS7715_PARPORT USB_SERIAL_MOS7720 USB_SERIAL_MOS7840 USB_SERIAL_MXUPORT USB_SERIAL_NAVMAN USB_SERIAL_OMNINET USB_SERIAL_OPTICON USB_SERIAL_OPTION USB_SERIAL_OTI6858 USB_SERIAL_PL2303 USB_SERIAL_QCAUX USB_SERIAL_QT2 USB_SERIAL_QUALCOMM USB_SERIAL_SAFE USB_SERIAL_SIERRAWIRELESS USB_SERIAL_SIMPLE USB_SERIAL_SPCP8X5 USB_SERIAL_SSU100 USB_SERIAL_SYMBOL USB_SERIAL_TI USB_SERIAL_UPD78F0730 USB_SERIAL_VISOR USB_SERIAL_WHITEHEAT USB_SERIAL_WISHBONE USB_SERIAL_WWAN USB_SERIAL_XR USB_SERIAL_XSENS_MT USB_SEVSEG USB_SI470X USB_SI4713 USB_SIERRA_NET USB_SISUSBVGA USB_SL811_CS USB_SL811_HCD USB_SL811_HCD_ISO USB_SNP_CORE USB_SPEEDTOUCH USB_STORAGE_ALAUDA USB_STORAGE_CYPRESS_ATACB USB_STORAGE_DATAFAB USB_STORAGE_ENE_UB6250 USB_STORAGE_FREECOM USB_STORAGE_ISD200 USB_STORAGE_JUMPSHOT USB_STORAGE_KARMA USB_STORAGE_ONETOUCH USB_STORAGE_SDDR09 USB_STORAGE_SDDR55 USB_STORAGE_USBAT USB_STV06XX USB_TEST USB_TMC USB_TRANCEVIBRATOR USB_U132_HCD USB_UAS USB_UEAGLEATM USB_ULPI_BUS USB_USBNET USB_USS720 USB_U_AUDIO USB_U_ETHER USB_U_SERIAL USB_VIDEO_CLASS USB_VIDEO_CLASS_INPUT_EVDEV USB_VL600 USB_WDM USB_XHCI_DBGCAP USB_XHCI_PLATFORM USB_XUSBATM USB_YUREX USERFAULTFD USERIO USERMODE_DRIVER USER_RETURN_NOTIFIER U_SERIAL_CONSOLE V4L2_MEM2MEM_DEV V4L_TEST_DRIVERS VALIDATE_FS_PARSER VDPA VDPA_SIM VDPA_SIM_BLOCK VDPA_SIM_NET VDPA_USER VETH VFIO VFIO_IOMMU_TYPE1 VFIO_PCI VFIO_PCI_CORE VFIO_PCI_INTX VFIO_PCI_MMAP VFIO_VIRQFD VGASTATE VHOST VHOST_CROSS_ENDIAN_LEGACY VHOST_IOTLB VHOST_NET VHOST_RING VHOST_VDPA VHOST_VSOCK VIDEOBUF2_CORE VIDEOBUF2_DMA_CONTIG VIDEOBUF2_DMA_SG VIDEOBUF2_MEMOPS VIDEOBUF2_V4L2 VIDEOBUF2_VMALLOC VIDEOMODE_HELPERS VIDEO_AU0828 VIDEO_AU0828_RC VIDEO_AU0828_V4L2 VIDEO_CS53L32A VIDEO_CX231XX VIDEO_CX231XX_ALSA VIDEO_CX231XX_DVB VIDEO_CX231XX_RC VIDEO_CX2341X VIDEO_CX25840 VIDEO_DEV VIDEO_EM28XX VIDEO_EM28XX_ALSA VIDEO_EM28XX_DVB VIDEO_EM28XX_RC VIDEO_EM28XX_V4L2 VIDEO_GO7007 VIDEO_GO7007_LOADER VIDEO_GO7007_USB VIDEO_GO7007_USB_S2250_BOARD VIDEO_HDPVR VIDEO_MSP3400 VIDEO_PVRUSB2 VIDEO_PVRUSB2_DVB VIDEO_PVRUSB2_SYSFS VIDEO_SAA711X VIDEO_STK1160 VIDEO_STK1160_COMMON VIDEO_TUNER VIDEO_TVEEPROM VIDEO_USBTV VIDEO_V4L2_I2C VIDEO_V4L2_SUBDEV_API VIDEO_V4L2_TPG VIDEO_VICODEC VIDEO_VIM2M VIDEO_VIMC VIDEO_VIVID VIDEO_VIVID_CEC VIDEO_WM8775 VIPERBOARD_ADC VIRTIO_BALLOON VIRTIO_DMA_SHARED_BUFFER VIRTIO_MEM VIRTIO_MMIO VIRTIO_MMIO_CMDLINE_DEVICES VIRTIO_PMEM VIRTIO_VDPA VIRTIO_VSOCKETS VIRTIO_VSOCKETS_COMMON VIRT_WIFI VLAN_8021Q VLAN_8021Q_GVRP VLAN_8021Q_MVRP VMAP_PFN VMWARE_VMCI VMXNET3 VP_VDPA VSOCKETS VSOCKETS_DIAG VSOCKETS_LOOPBACK VSOCKMON VT_HW_CONSOLE_BINDING VXFS_FS WAN WANT_COMPAT_NETLINK_MESSAGES WANT_DEV_COREDUMP WATCH_QUEUE WEXT_CORE WEXT_PRIV WEXT_PROC WIREGUARD WIRELESS WIRELESS_EXT WLAN WLAN_VENDOR_ADMTEK WLAN_VENDOR_ATH WLAN_VENDOR_PURELIFI WLAN_VENDOR_SILABS WWAN X25 X86_SGX X86_SGX_KVM X86_X2APIC X86_X32_ABI XARRAY_MULTI XDP_SOCKETS XDP_SOCKETS_DIAG XFRM_ESPINTCP XFRM_INTERFACE XFRM_IPCOMP XFRM_MIGRATE XFRM_OFFLOAD XFRM_STATISTICS XFRM_SUB_POLICY XFRM_USER_COMPAT XFS_FS XFS_POSIX_ACL XFS_QUOTA XFS_RT XFS_SUPPORT_V4 XOR_BLOCKS YENTA YENTA_ENE_TUNE YENTA_O2 YENTA_RICOH YENTA_TI YENTA_TOSHIBA ZBUD ZEROPLUS_FF ZLIB_DEFLATE ZONEFS_FS ZONE_DEVICE ZPOOL ZRAM ZRAM_DEF_COMP_LZORLE ZSMALLOC ZSTD_COMPRESS ZSWAP ZSWAP_COMPRESSOR_DEFAULT_LZO ZSWAP_ZPOOL_DEFAULT_ZBUD] disabling configs for [UBSAN BUG KASAN LOCKDEP ATOMIC_SLEEP LEAK], they are not needed determining the merge base between fa74641fb6b93a19ccb50579886ecc98320230f9 and 57d88e8a5974644039fbc47806bac7bb12025636 830b3c68c1fb1e9176028d02ef86f3cf76aa2476/Linux 6.1 is a merge base, check if it has the bug testing commit 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: e8a29c59dadfe43402989576b82d1b650291d7088d1b7081ac5701cb1c5798ae run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] testing current HEAD 57d88e8a5974644039fbc47806bac7bb12025636 testing commit 57d88e8a5974644039fbc47806bac7bb12025636 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 14b3ba9fffa07d461c1d6915a7164eb11dcbc6b23c6be96c1cef858fb060b57a all runs: OK false negative chance: 0.000 # git bisect start 57d88e8a5974644039fbc47806bac7bb12025636 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 Bisecting: 38401 revisions left to test after this (roughly 15 steps) [b68ee1c6131c540a62ecd443be89c406401df091] Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit b68ee1c6131c540a62ecd443be89c406401df091 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 97a8b193d1f2028696c35576b700a528d131c47f1ae6487242f2102c4f2accd9 all runs: OK false negative chance: 0.000 # git bisect bad b68ee1c6131c540a62ecd443be89c406401df091 Bisecting: 19083 revisions left to test after this (roughly 14 steps) [8ca8d89b43caf9a02a18414d6eeff966d2b14512] Merge tag 'cgroup-for-6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 8ca8d89b43caf9a02a18414d6eeff966d2b14512 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 9ac09ebbe28c6a0a0312a32bda51849bf1d7e1526d1950d9c17db3c266595374 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good 8ca8d89b43caf9a02a18414d6eeff966d2b14512 Bisecting: 9554 revisions left to test after this (roughly 13 steps) [60e2bf7d10e9cd5641f4a5183a19058d9a2c8782] Merge tag 'input-for-v6.3-rc0' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input determine whether the revision contains the guilty commit revision 8ca8d89b43caf9a02a18414d6eeff966d2b14512 crashed and is reachable testing commit 60e2bf7d10e9cd5641f4a5183a19058d9a2c8782 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: f7b9914b95e4be4060ad70d6169f02f7a01f1ba7a2dc82818b066209232b64d9 all runs: OK false negative chance: 0.000 # git bisect bad 60e2bf7d10e9cd5641f4a5183a19058d9a2c8782 Bisecting: 4979 revisions left to test after this (roughly 12 steps) [307e14c039063f0c9bd7a18a7add8f940580dcc9] Merge tag '6.3-rc-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6 determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 307e14c039063f0c9bd7a18a7add8f940580dcc9 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: a53b3386cc3e120663ac20f1c5ea8a42bb6bf8eb47a0e0725b29800b57db9aa1 all runs: OK false negative chance: 0.000 # git bisect bad 307e14c039063f0c9bd7a18a7add8f940580dcc9 Bisecting: 2481 revisions left to test after this (roughly 11 steps) [d1fabc68f8e0541d41657096dc713cb01775652d] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit d1fabc68f8e0541d41657096dc713cb01775652d gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: a514fa933f577cd6061bae2a7d36333f9b48a44c7aa162e9825914cdc3244dd0 all runs: OK false negative chance: 0.000 # git bisect bad d1fabc68f8e0541d41657096dc713cb01775652d Bisecting: 1033 revisions left to test after this (roughly 10 steps) [8b6f322e47ba4ac95b63bff1fcdd0d3f85fbc61b] Merge branch 'net-sched-transition-act_pedit-to-rcu-and-percpu-stats' determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 8b6f322e47ba4ac95b63bff1fcdd0d3f85fbc61b gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 0d3e3d85012255b4dc0d2e7c465b1c02f9c15990008c5d759391443947597ef7 all runs: OK false negative chance: 0.000 # git bisect bad 8b6f322e47ba4ac95b63bff1fcdd0d3f85fbc61b Bisecting: 530 revisions left to test after this (roughly 9 steps) [306f208259ff7a89c022f1e5f5d2144b0ccb236c] dt-bindings: net: asix,ax88796c: allow SPI peripheral properties determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 306f208259ff7a89c022f1e5f5d2144b0ccb236c gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: ba4d3c2c87e285c94e83afae65a5e2122b3c5d9456324ff27094c9450bb90ef8 all runs: OK false negative chance: 0.000 # git bisect bad 306f208259ff7a89c022f1e5f5d2144b0ccb236c Bisecting: 250 revisions left to test after this (roughly 8 steps) [fab89bafa95b6f333b0e2dca0ae07a0b3600e954] virtio-net: support multi-buffer xdp determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit fab89bafa95b6f333b0e2dca0ae07a0b3600e954 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: f69b1e2be9899abca8a604f31648f63e07b468bccc600624cf5fcd908d10a3f3 all runs: OK false negative chance: 0.000 # git bisect bad fab89bafa95b6f333b0e2dca0ae07a0b3600e954 Bisecting: 125 revisions left to test after this (roughly 7 steps) [4a753ca5013d465694d6639fbe1378831a399bda] selftest: mptcp: exit from copyfd_io_poll() when receive SIGUSR1 determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 4a753ca5013d465694d6639fbe1378831a399bda gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: eb0141fae340df460c91ba208737e42a3637d622301f72496779e9ada649ae43 all runs: OK false negative chance: 0.000 # git bisect bad 4a753ca5013d465694d6639fbe1378831a399bda Bisecting: 62 revisions left to test after this (roughly 6 steps) [4aea86b4033f92f01547e6d4388d4451ae9b0980] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 4aea86b4033f92f01547e6d4388d4451ae9b0980 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 6fc593fb9c1a179c0e7f3f3338efb51a4f699832d04a8b3597f97c121881c200 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good 4aea86b4033f92f01547e6d4388d4451ae9b0980 Bisecting: 31 revisions left to test after this (roughly 5 steps) [870c7ad4a52be2acff92d0355ca118654c7efece] devlink: protect devlink->dev by the instance lock determine whether the revision contains the guilty commit revision 4aea86b4033f92f01547e6d4388d4451ae9b0980 crashed and is reachable testing commit 870c7ad4a52be2acff92d0355ca118654c7efece gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 8814a0f5b9e1ac7ea7dc66454ad7551ceac9b1812ce36a56361a1e2827080d1e all runs: OK false negative chance: 0.001 # git bisect bad 870c7ad4a52be2acff92d0355ca118654c7efece Bisecting: 15 revisions left to test after this (roughly 4 steps) [623cd13b165486afaa2df706d49209392f3764ca] devlink: split out netlink code determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 623cd13b165486afaa2df706d49209392f3764ca gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 2a9bc7d31744381ebe48a593906bd892756d88642df901cd28d065408ed964ca run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good 623cd13b165486afaa2df706d49209392f3764ca Bisecting: 7 revisions left to test after this (roughly 3 steps) [c9666bac537e0f98ce15ea57c5fb25fc0a9a29b1] devlink: restart dump based on devlink instance ids (function) determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit c9666bac537e0f98ce15ea57c5fb25fc0a9a29b1 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: f4a27d654ace3a3bf2eb968a2597171eded39a952d6fff4951c68ffa947b092a run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: crashed: INFO: rcu detected stall in corrupted run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good c9666bac537e0f98ce15ea57c5fb25fc0a9a29b1 Bisecting: 3 revisions left to test after this (roughly 2 steps) [3d759e9e24c38758abc19a4f5e1872a6460d5745] Merge branch 'devlink-code-split-and-structured-instance-walk' determine whether the revision contains the guilty commit revision 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 crashed and is reachable testing commit 3d759e9e24c38758abc19a4f5e1872a6460d5745 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: e4c1383b991c5b889069f8ec35f87a3770f13126bcf2e2e2758b307265071345 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good 3d759e9e24c38758abc19a4f5e1872a6460d5745 Bisecting: 1 revision left to test after this (roughly 1 step) [d772781964415c63759572b917e21c4f7ec08d9f] devlink: bump the instance index directly when iterating determine whether the revision contains the guilty commit revision 623cd13b165486afaa2df706d49209392f3764ca crashed and is reachable testing commit d772781964415c63759572b917e21c4f7ec08d9f gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: 1d1caf4798a721f3bb7fd31916adb03dc59c8de727ca68246b2bc298514caa6c all runs: OK false negative chance: 0.001 # git bisect bad d772781964415c63759572b917e21c4f7ec08d9f Bisecting: 0 revisions left to test after this (roughly 0 steps) [6b754d7bd007c5f68fbb2d9abd5c00d253b033d0] sysctl: expose all net/core sysctls inside netns determine whether the revision contains the guilty commit revision 4aea86b4033f92f01547e6d4388d4451ae9b0980 crashed and is reachable testing commit 6b754d7bd007c5f68fbb2d9abd5c00d253b033d0 gcc compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 kernel signature: ac2e1eff5a672e4a2dbab2ef731f2d76182bfb2bb3fb7210d41edb80203f10a4 run #0: crashed: INFO: rcu detected stall in corrupted run #1: crashed: INFO: rcu detected stall in corrupted run #2: crashed: INFO: rcu detected stall in corrupted run #3: crashed: INFO: rcu detected stall in corrupted run #4: crashed: INFO: rcu detected stall in corrupted run #5: crashed: INFO: rcu detected stall in corrupted run #6: crashed: INFO: rcu detected stall in corrupted run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK representative crash: INFO: rcu detected stall in corrupted, types: [HANG] # git bisect good 6b754d7bd007c5f68fbb2d9abd5c00d253b033d0 d772781964415c63759572b917e21c4f7ec08d9f is the first bad commit commit d772781964415c63759572b917e21c4f7ec08d9f Author: Jakub Kicinski Date: Thu Jan 5 22:33:54 2023 -0800 devlink: bump the instance index directly when iterating xa_find_after() is designed to handle multi-index entries correctly. If a xarray has two entries one which spans indexes 0-3 and one at index 4 xa_find_after(0) will return the entry at index 4. Having to juggle the two callbacks, however, is unnecessary in case of the devlink xarray, as there is 1:1 relationship with indexes. Always use xa_find() and increment the index manually. Signed-off-by: Jakub Kicinski Reviewed-by: Jiri Pirko Signed-off-by: David S. Miller net/devlink/core.c | 31 +++++++++---------------------- net/devlink/devl_internal.h | 17 ++++------------- 2 files changed, 13 insertions(+), 35 deletions(-) accumulated error probability: 0.00 culprit signature: 1d1caf4798a721f3bb7fd31916adb03dc59c8de727ca68246b2bc298514caa6c parent signature: ac2e1eff5a672e4a2dbab2ef731f2d76182bfb2bb3fb7210d41edb80203f10a4 reproducer is flaky (0.30 repro chance estimate) revisions tested: 25, total time: 7h21m13.381552903s (build: 2h33m37.951539863s, test: 4h37m9.316774552s) first good commit: d772781964415c63759572b917e21c4f7ec08d9f devlink: bump the instance index directly when iterating recipients (to): ["davem@davemloft.net" "jiri@nvidia.com" "kuba@kernel.org"] recipients (cc): []