syzbot


panic: Empty stailq ADDR->stqh_last is ADDR, not head's first field address (2)

Status: fixed on 2025/03/29 11:48
Reported-by: syzbot+50ed414818d99ddd2eb9@syzkaller.appspotmail.com
Fix commit: e9a846468acf ktrace: Use STAILQ_EMPTY_ATOMIC when checking for records in userret()
First crash: 73d, last: 73d
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
freebsd panic: Empty stailq ADDR->stqh_last is ADDR, not head's first field address 1 119d 119d 2/2 fixed on 2025/02/21 18:47

Sample crash report:
panic: Empty stailq 0xfffffe005492f500->stqh_last is 0xfffffe00083cfc58, not head's first field address
cpuid = 1
time = 1742685658
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0xc6/frame 0xfffffe00576b6a90
kdb_backtrace() at kdb_backtrace+0xd0/frame 0xfffffe00576b6bf0
vpanic() at vpanic+0x257/frame 0xfffffe00576b6db0
panic() at panic+0xb5/frame 0xfffffe00576b6e70
ast_ktrace() at ast_ktrace+0x95/frame 0xfffffe00576b6ea0
ast_handler() at ast_handler+0x2b0/frame 0xfffffe00576b6f10
ast() at ast+0x25/frame 0xfffffe00576b6f30
doreti_ast() at doreti_ast+0x1c/frame 0x825dbdf80
KDB: enter: panic
[ thread pid 885 tid 100247 ]
Stopped at      kdb_enter+0x6e: movq    $0,0x23e7737(%rip)
db> 
db> set $lines = 0
db> set $maxwidth = 0
db> show registers
cs                        0x20
ds                        0x3b
es                        0x3b
fs                        0x13
gs                        0x1b
ss                        0x28
rax                       0x12
rcx         0xfffffe0076800000
rdx                    0x7ffff
rbx         0xffffffff82750300  .str.27
rsp         0xfffffe00576b6bd0
rbp         0xfffffe00576b6bf0
rsi                    0x80001
rdi         0xffffffff815dbf59  printf+0x149
r8                           0
r9                  0xffffffff
r10                          0
r11                        0x2
r12         0xfffffe0054945000
r13         0xfffffffffffffffd
r14         0xffffffff82750300  .str.27
r15                          0
rip         0xffffffff815c68de  kdb_enter+0x6e
rflags                    0x46
kdb_enter+0x6e: movq    $0,0x23e7737(%rip)
db> show proc
Process 885 (syz-executor) at 0xfffffe005492f020:
 state: NORMAL
 uid: 0  gids: 0, 0, 5
 parent: pid 766 at 0xfffffe00548f25a0
 ABI: FreeBSD ELF64
 flag: 0x10000080  flag2: 0
 arguments: ./syz-executor exec
 reaper: 0xfffffe0008007040 reapsubtree: 1
 sigparent: 20
 vmspace: 0xfffffe00548d5920
   (map 0xfffffe00548d5920)
   (map.pmap 0xfffffe00548d59c0)
   (pmap 0xfffffe00548d5a30)
 threads: 3
100236                   RunQ                                syz-executor
100246                   S       kqread  0xfffffe0058d50600  syz-executor
100247                   Run     CPU 1                       syz-executor
db> ps
  pid  ppid  pgrp   uid  state   wmesg   wchan               cmd
  886   764   764     0  R       (threaded)                  syz-executor
100125                   RunQ                                syz-executor
100248                   S       uwait   0xfffffe0071f0a080  syz-executor
100250                   S       uwait   0xfffffe0071793080  syz-executor
  885   766   766     0  R       (threaded)                  syz-executor
100236                   RunQ                                syz-executor
100246                   S       kqread  0xfffffe0058d50600  syz-executor
100247                   Run     CPU 1                       syz-executor
  884   765   765     0  R       (threaded)                  syz-executor
100088                   RunQ                                syz-executor
100244                   S       connec  0xfffffe0059fcfcda  syz-executor
100249                   S       uwait   0xfffffe0071f0a280  syz-executor
  883   767   767     0  R       (threaded)                  syz-executor
100229                   RunQ                                syz-executor
100245                   S       uwait   0xfffffe0071791680  syz-executor
  881     1   766     0  S       uwait   0xfffffe0071791a80  syz-executor
  878     1   764     0  S       uwait   0xfffffe0059e4df00  syz-executor
  873     1   764     0  S       uwait   0xfffffe0059fd9380  syz-executor
  868     1   765     0  S       uwait   0xfffffe0059fdb980  syz-executor
  861     1   767     0  S       uwait   0xfffffe0071f0ac80  syz-executor
  856     1   765     0  S       uwait   0xfffffe0071f0ab80  syz-executor
  846     1   765     0  S       uwait   0xfffffe0071f0aa80  syz-executor
  836     1   766     0  S       uwait   0xfffffe0059e4d500  syz-executor
  830     1   765     0  S       uwait   0xfffffe0059fd9180  syz-executor
  826     1   767     0  S       uwait   0xfffffe0071792d80  syz-executor
  825     0     0     0  DL      aiordy  0xfffffe0054930040  [aiod4]
  824     0     0     0  DL      aiordy  0xfffffe0054804b00  [aiod3]
  823     0     0     0  DL      aiordy  0xfffffe00080085c0  [aiod2]
  822     0     0     0  DL      aiordy  0xfffffe00548f1580  [aiod1]
  817   805   817     0  Ss      select  0xfffffe00587ff640  dhclient
  805   789   424    65  S       select  0xfffffe0058cec9c0  dhclient
  789   424   424     0  S       wait    0xfffffe005491f060  sh
  767   763   767     0  R                                   syz-executor
  766   763   766     0  R                                   syz-executor
  765   763   765     0  R                                   syz-executor
  764   763   764     0  R                                   syz-executor
  763   761   761     0  S       select  0xfffffe00587ff340  syz-executor
  761   759   761     0  Ss      pause   0xfffffe00548f10d0  csh
  759     1   759     0  Ss      select  0xfffffe00587ff3c0  sshd
  750     1   750     0  Ss+     ttyin   0xfffffe0058a7fcb0  getty
  749     1   749     0  Ss+     ttyin   0xfffffe0059bd14b0  getty
  748     1   748     0  Ss+     ttyin   0xfffffe0059bd18b0  getty
  747     1   747     0  Ss+     ttyin   0xfffffe0059bd1cb0  getty
  746     1   746     0  Ss+     ttyin   0xfffffe0058a7ecb0  getty
  745     1   745     0  Ss+     ttyin   0xfffffe0059bd30b0  getty
  744     1   744     0  Ss+     ttyin   0xfffffe0059bd34b0  getty
  743     1   743     0  Ss+     ttyin   0xfffffe0059bd38b0  getty
  742     1   742     0  Ss+     ttyin   0xfffffe0059bd3cb0  getty
  740     1    18     0  S+      piperd  0xfffffe0059f36700  logger
  739   738    18     0  S+      nanslp  0xffffffff8398f3c0  sleep
  738     1    18     0  S+      wait    0xfffffe0008008060  sh
  495     1   495     0  Ss      select  0xfffffe00587ff5c0  syslogd
  424     1   424     0  Ss      wait    0xfffffe00548f1ae0  devd
   17     0     0     0  DL      syncer  0xffffffff83aacc20  [syncer]
   16     0     0     0  DL      vlruwt  0xfffffe0008026040  [vnlru]
   15     0     0     0  DL      (threaded)                  [bufdaemon]
100080                   D       psleep  0xffffffff83aab1e0  [bufdaemon]
100083                   D       -       0xffffffff82e02140  [bufspacedaemon-0]
100096                   D       sdflush 0xfffffe0058a7d8e8  [/ worker]
    9     0     0     0  DL      psleep  0xffffffff83af60e0  [vmdaemon]
    8     0     0     0  DL      (threaded)                  [pagedaemon]
100078                   D       psleep  0xffffffff83adc078  [dom0]
100084                   D       launds  0xffffffff83adc084  [laundry: dom0]
100085                   D       umarcl  0xffffffff81d87d60  [uma]
    7     0     0     0  DL      -       0xffffffff8370cbd0  [rand_harvestq]
    6     0     0     0  DL      pftm    0xffffffff84200600  [pf purge]
    5     0     0     0  DL      waiting 0xffffffff8449e580  [sctp_iterator]
    4     0     0     0  DL      (threaded)                  [cam]
100046                   D       -       0xffffffff836d7340  [doneq0]
100047                   D       -       0xffffffff836d72c0  [async]
100076                   D       -       0xffffffff836d7140  [scanner]
    3     0     0     0  DL      (threaded)                  [crypto]
100043                   D       crypto_ 0xffffffff83ad78a0  [crypto]
100044                   D       crypto_ 0xfffffe0058613e30  [crypto returns 0]
100045                   D       crypto_ 0xfffffe0058613e80  [crypto returns 1]
   14     0     0     0  DL      seqstat 0xfffffe0007f59c88  [sequencer 00]
   13     0     0     0  DL      (threaded)                  [geom]
100037                   D       -       0xffffffff83937da0  [g_event]
100038                   D       -       0xffffffff83937dc0  [g_up]
100039                   D       -       0xffffffff83937de0  [g_down]
    2     0     0     0  RL      (threaded)                  [clock]
100031                   I                                   [clock (0)]
100032                   Run     CPU 0                       [clock (1)]
   12     0     0     0  WL      (threaded)                  [intr]
100013                   I                                   [swi6: task queue]
100014                   I                                   [swi6: Giant taskq]
100016                   I                                   [swi5: fast taskq]
100033                   I                                   [swi1: netisr 0]
100034                   I                                   [swi1: hpts]
100035                   I                                   [swi1: hpts]
100048                   I                                   [irq24: virtio_pci0]
100049                   I                                   [irq25: virtio_pci0]
100050                   I                                   [irq26: virtio_pci0]
100051                   I                                   [irq27: virtio_pci0]
100052                   I                                   [irq28: virtio_pci1]
100053                   I                                   [irq29: virtio_pci1]
100054                   I                                   [irq30: virtio_pci1]
100055                   I                                   [irq31: virtio_pci1]
100056                   I                                   [irq32: virtio_pci1]
100061                   I                                   [irq10: virtio_pci2]
100063                   I                                   [irq1: atkbd0]
100064                   I                                   [irq12: psm0]
100065                   I                                   [swi0: uart uart++]
100069                   I                                   [swi1: pf send]
   11     0     0     0  RL      (threaded)                  [idle]
100003                   CanRun                              [idle: cpu0]
100004                   CanRun                              [idle: cpu1]
    1     0     1     0  TLs                                 [init]
   10     0     0     0  DL      audit_w 0xffffffff83ad8340  [audit]
    0     0     0     0  DLs     (threaded)                  [kernel]
100000                   D       parked  0xffffffff84a12ff0  [swapper]
100005                   D       -       0xfffffe0058016000  [softirq_0]
100006                   D       -       0xfffffe0058015e00  [softirq_1]
100007                   D       -       0xfffffe0058015d00  [if_io_tqg_0]
100008                   D       -       0xfffffe0058015c00  [if_io_tqg_1]
100009                   D       -       0xfffffe0058015b00  [if_config_tqg_0]
100010                   D       -       0xfffffe0007fc7500  [kqueue_ctx taskq]
100011                   D       -       0xfffffe0007fc7300  [jail_remove taskq]
100012                   D       -       0xfffffe0007fc7100  [bus taskq]
100015                   D       -       0xfffffe0007fc6a00  [thread taskq]
100017                   D       -       0xfffffe0007fc6600  [aiod_kick taskq]
100018                   D       -       0xfffffe0007fc6400  [deferred_unmount ta]
100019                   D       -       0xfffffe0007fc6200  [inm_free taskq]
100020                   D       -       0xfffffe0007fc6000  [in6m_free taskq]
100021                   D       -       0xfffffe0007fc5d00  [linuxkpi_irq_wq]
100022                   D       -       0xfffffe0007fc5b00  [linuxkpi_short_wq_0]
100023                   D       -       0xfffffe0007fc5b00  [linuxkpi_short_wq_1]
100024                   D       -       0xfffffe0007fc5b00  [linuxkpi_short_wq_2]
100025                   D       -       0xfffffe0007fc5b00  [linuxkpi_short_wq_3]
100026                   D       -       0xfffffe0007fc5600  [linuxkpi_long_wq_0]
100027                   D       -       0xfffffe0007fc5600  [linuxkpi_long_wq_1]
100028                   D       -       0xfffffe0007fc5600  [linuxkpi_long_wq_2]
100029                   D       -       0xfffffe0007fc5600  [linuxkpi_long_wq_3]
100036                   D       -       0xfffffe0008bffa00  [firmware taskq]
100041                   D       -       0xfffffe0008bff400  [crypto_0]
100042                   D       -       0xfffffe0008bff400  [crypto_1]
100057                   D       -       0xfffffe00587abb00  [vtnet0 rxq 0]
100058                   D       -       0xfffffe00587aba00  [vtnet0 txq 0]
100059                   D       -       0xfffffe00587ab900  [vtnet0 rxq 1]
100060                   D       -       0xfffffe00587ab800  [vtnet0 txq 1]
100062                   D       vtbslp  0xfffffe00587ff880  [virtio_balloon]
100066                   D       -       0xffffffff827555a1  [deadlkres]
100070                   D       -       0xfffffe0058cd9100  [acpi_task_0]
100071                   D       -       0xfffffe0058cd9100  [acpi_task_1]
100072                   D       -       0xfffffe0058cd9100  [acpi_task_2]
100074                   D       -       0xfffffe0007fc8100  [mca taskq]
100075                   D       -       0xfffffe00587ac300  [CAM taskq]
100077                   D       -       0xfffffe00587aa800  [ipsec_offload]
db> show all locks
Process 885 (syz-executor) thread 0xfffffe0054949000 (100236)
exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0071993908) locked @ /syzkaller/managers/main/kernel/sys/kern/kern_ktrace.c:1382
exclusive sx ktrace_sx (ktrace_sx) r = 0 (0xffffffff8393bd60) locked @ /syzkaller/managers/main/kernel/sys/kern/kern_ktrace.c:423
db> show malloc
              Type        InUse        MemUse     Requests
           pf_hash            6        12804K            6
            linker          375         5019K          485
          tcp_hpts            7         4801K            7
            devbuf         4188         4324K         4213
         sysctloid        34737         2046K        34812
             vtbuf           24         1968K           46
              kobj          330         1320K          494
            newblk           14         1028K         1221
          vfscache            3         1025K            3
               pcb           31          678K          202
          inodedep           12          517K          248
         ufs_quota            1          512K            1
          vfs_hash            1          512K            1
           callout            2          512K            2
              intr            4          472K            4
          filedesc           38          297K          199
           subproc          127          266K          955
         vnet_data            2          224K            2
          acpitask            1          224K            1
            KTRACE          104          201K         5598
            acpica         1674          184K        54408
              vmem            5          144K            6
           tidhash            3          141K            3
           pagedep           10          131K          110
        tfo_ccache            1          128K            1
          IP reass            1          128K            1
            DEVFS1          109          109K          126
               sem            4          106K            4
        gtaskqueue           18           98K           18
               bus          994           81K         5040
          mtx_pool            3           74K            3
          syncache            1           68K            1
     NFSD srvcache            3           68K            3
            module          519           65K          519
       ddb_capture            1           64K            1
           kdtrace          200           39K         1138
              umtx          304           38K          304
              temp           35           37K         1861
            DEVFS3          129           33K          139
         hostcache            1           32K            1
               shm            1           32K            1
               msg            4           30K            4
            kbdmux            6           28K            6
        DEVFS_RULE           56           20K           56
            ifaddr           67           19K           69
               LRO           18           19K           18
               BPF           14           19K           22
         ufs_mount            4           17K            5
              proc            3           17K            3
               tty           16           16K           16
          routetbl          130           16K          410
           ithread           90           15K           90
            bus-sc           34           15K         1637
           lltable           45           14K           47
      eventhandler          163           14K          163
             ifnet            7           13K            7
       ether_multi          152           13K          167
              kenv           95           12K           95
              GEOM           61           11K          477
         CAM queue            5           11K         1528
              rman           82           10K          437
             shmfd            4           10K            5
               rpc            8            9K            8
         in6_multi           65            9K           65
         bmsafemap            2            9K          189
            plimit           22            9K          333
           devstat            4            9K            4
              UART           12            9K           12
              ksem            1            8K            1
       pfs_vncache            1            8K            1
     audit_evclass          239            8K          301
         taskqueue           69            8K           69
            sglist            6            7K            6
              cred           24            6K          206
           CAM DEV            3            6K          510
            kqueue           66            6K          998
         pfs_nodes           22            6K           22
       ufs_dirhash           24            5K           24
               UMA          265            5K          265
          pf_ifnet           10            5K           19
                vt           11            5K           11
           memdesc            1            4K            1
               MCA           32            4K           32
             evdev            4            4K            4
           pwddesc           60            4K          890
           acpisem           28            4K           28
            DEVFSP           54            4K           87
          terminal           11            3K           11
           acpidev           20            3K           20
         proc-args           80            3K         1930
             hhook            8            3K           10
             clone            9            3K            9
             selfd           36            3K         9500
           uidinfo            3            3K            6
          kcovinfo           36            3K           36
          filecaps            5            3K           73
         sctp_timw            8            2K            8
        local_apic            1            2K            1
           io_apic            1            2K            1
         ipsec-saq            2            2K            2
           session           16            2K           36
         sctp_atcl            5            2K           89
            ip6ndp           12            2K           13
            Unitno           28            2K           50
          sctp_ifa           13            2K           14
             lockf           15            2K           51
           CAM XPT           22            2K          543
            crypto            7            2K           19
          in_multi            6            2K            9
               tun            4            2K            4
         toponodes            6            2K            6
       ipsecpolicy            2            2K            2
               msi            9            2K            9
           netlink            2            2K           74
         sctp_stro            1            1K           11
           softdep            1            1K            1
          freefrag            8            1K          154
            sahead            1            1K            1
          secasvar            1            1K            1
             nhops            6            1K            8
       vnodemarker            2            1K           22
      NFSD session            1            1K            1
         cryptodev           14            1K           87
            diradd            7            1K          188
            CC Mem            7            1K           53
        CAM periph            4            1K          271
             ipsec            3            1K            3
          sctp_ifn            6            1K           14
               mld            6            1K            6
              igmp            6            1K            6
              pfil            6            1K            6
            select            6            1K           30
            isadev            6            1K            6
             mount           16            1K          293
          pci_link           10            1K           10
         newdirblk            5            1K           92
             mkdir            5            1K          184
 encap_export_host           12            1K           12
       inpcbpolicy           17            1K          221
              cdev            2            1K            2
               osd           12            1K           66
       lkpikmalloc            8            1K            9
          freefile            3            1K          126
    chacha20random            1            1K            1
            biobuf            1            1K            1
          freework            2            1K          271
             DEVFS           10            1K           11
            dirrem            1            1K          169
          freeblks            1            1K          114
            vnodes            1            1K            1
      NFSD lckfile            1            1K            1
     NFSD V4client            1            1K            1
           CAM SIM            2            1K            2
         sctp_atky            6            1K          103
            feeder            7            1K            7
           tcpfunc            3            1K            3
        loginclass            3            1K            5
            prison            6            1K            6
          nexusdev            8            1K            8
            apmdev            1            1K            1
          atkbddev            2            1K            2
               aio            4            1K            4
            soname            5            1K         3413
          pmchooks            1            1K            1
          CAM path            4            1K         1034
     CAM dev queue            2            1K            2
 CAM I/O Scheduler            1            1K            1
         sctp_athm            5            1K           92
          sctp_vrf            1            1K            1
              vnet            1            1K            1
               pmc            1            1K            1
           entropy            2            1K           33
          acpiintr            1            1K            1
            DEVFS4            2            1K            2
          sctp_map            2            1K           22
              cpus            2            1K            2
    vnet_data_free            1            1K            1
           Per-cpu            1            1K            1
          p1003.1b            1            1K            1
            ipcomp            0            0K            0
               esp            0            0K            0
                ah            0            0K            0
      tcp_pcm_rack            0            0K            2
       tcp_do_rack            0            0K            0
      tcp_fsb_rack            0            0K            4
        sctp_mcore            0            0K            0
        sctp_socko            0            0K           10
         sctp_iter            0            0K           10
         sctp_mvrf            0            0K            0
         sctp_cpal            0            0K            0
         sctp_cmsg            0            0K            0
         sctp_stre            0            0K            0
         sctp_athi            0            0K            0
         sctp_a_it            0            0K           10
         sctp_aadr            0            0K            0
         sctp_stri            0            0K            6
          pf_table            0            0K            0
           pf_rule            0            0K            0
           pf_altq            0            0K            0
           pf_osfp            0            0K            0
     pf_krule_item            0            0K            0
           pf_temp            0            0K            0
            mqdata            0            0K            0
           filemon            0            0K            1
        madt_table            0            0K            2
          smartpqi            0            0K            0
               ixl            0            0K            0
        ice-resmgr            0            0K            0
         ice-osdep            0            0K            0
               ice            0            0K            0
              iavf            0            0K            0
             axgbe            0            0K            0
       fpukern_ctx            0            0K            0
          xen_intr            0            0K            0
           xen_hvm            0            0K            0
         legacydrv            0            0K            0
      NMI handlers            0            0K            0
            bounce            0            0K            0
            busdma            0            0K            0
            qpidrv            0            0K            0
      dmar_idpgtbl            0            0K            0
          dmar_dom            0            0K            0
          dmar_ctx            0            0K            0
      amdiommu_dom            0            0K            0
      amdiommu_ctx            0            0K            0
              isci            0            0K            0
      iommu_dmamap            0            0K            0
     hyperv_socket            0            0K            0
           bxe_ilt            0            0K            0
        aesni_data            0            0K            0
            xenbus            0            0K            0
     vm_fictitious            0            0K            0
           UMAHash            0            0K            0
         vm_pgdata            0            0K            0
           jblocks            0            0K            0
          savedino            0            0K          168
          sentinel            0            0K            0
            jfsync            0            0K            0
            jtrunc            0            0K            0
             sbdep            0            0K            9
           jsegdep            0            0K            0
              jseg            0            0K            0
         jfreefrag            0            0K            0
          jfreeblk            0            0K            0
           jnewblk            0            0K            0
            jmvref            0            0K            0
           jremref            0            0K            0
           jaddref            0            0K            0
           freedep            0            0K            0
        allocindir            0            0K            0
          indirdep            0            0K          125
       allocdirect            0            0K            0
          ufs_trim            0            0K            0
           mactemp            0            0K            0
     audit_trigger            0            0K            0
 audit_pipe_presel            0            0K            0
     audit_pipeent            0            0K            0
        audit_pipe            0            0K            0
      audit_evname            0            0K            0
         audit_bsm            0            0K            0
      audit_gidset            0            0K            0
        audit_text            0            0K            0
        audit_path            0            0K            0
        audit_data            0            0K            0
        audit_cred            0            0K            0
          ktls_ocf            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
      MLX5E_TLS_RX            0            0K            0
        MLX5EEPROM            0            0K            0
         MLX5E_TLS            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
            MLX5EN            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
          MLX5DUMP            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
       simple_attr            0            0K            0
          seq_file            0            0K            0
           lkpiskb            0            0K            0
             radix            0            0K            0
               idr            0            0K            0
          lkpindev            0            0K            0
           lkpimhi            0            0K            0
            lkpifw            0            0K            0
         lkpi80211            0            0K            0
               NLM            0            0K            0
    ipsec-spdcache            0            0K            0
         ipsec-reg            0            0K            0
        ipsec-misc            0            0K            0
      ipsecrequest            0            0K            0
            ip6opt            0            0K            5
       ip6_msource            0            0K            0
      ip6_moptions            0            0K            1
       in6_mfilter            0            0K            1
             frag6            0            0K            0
            tcplog            0            0K            0
        tcp_hwpace            0            0K            0
        ip_msource            0            0K            1
       ip_moptions            0            0K            1
        in_mfilter            0            0K            2
              ipid            0            0K            0
         80211scan            0            0K            0
      80211ratectl            0            0K            0
        80211power            0            0K            0
       80211nodeie            0            0K            0
         80211node            0            0K            0
      80211mesh_gt            0            0K            0
      80211mesh_rt            0            0K            0
         80211perr            0            0K            0
         80211prep            0            0K            0
         80211preq            0            0K            0
          80211dfs            0            0K            0
       80211crypto            0            0K            0
          80211vap            0            0K            0
             iflib            0            0K            0
              vlan            0            0K            0
               gif            0            0K            0
           ifdescr            0            0K            0
              zlib            0            0K           19
           fadvise            0            0K            0
           VN POLL            0            0K            0
            statfs            0            0K          183
     namei_tracker            0            0K            0
       export_host            0            0K            0
        cl_savebuf            0            0K           55
               lio            0            0K            2
               acl            0            0K            0
          mbuf_tag            0            0K            0
              ktls            0            0K            0
              accf            0            0K            0
               pts            0            0K            0
           timerfd            0            0K            0
          procdesc            0            0K            8
               iov            0            0K        14067
          ioctlops            0            0K          108
           eventfd            0            0K            0
           Witness            0            0K            0
             stack            0            0K            0
              sbuf            0            0K          312
          firmware            0            0K            0
        compressor            0            0K            0
              SWAP            0            0K            0
         sysctltmp            0            0K          649
            sysctl            0            0K            3
              ekcd            0            0K            0
            dumper            0            0K            0
          sendfile            0            0K            0
              rctl            0            0K            0
             cache            0            0K            0
      prison_racct            0            0K            0
       Fail Points            0            0K            0
             sigio            0            0K            1
filedesc_to_leader            0            0K            0
               pwd            0            0K            0
       tty console            0            0K            0
         boottrace            0            0K            0
        isofs_node            0            0K            0
       isofs_mount            0            0K            0
     tr_raid5_data            0            0K            0
    tr_raid1e_data            0            0K            0
     tr_raid1_data            0            0K            0
     tr_raid0_data            0            0K            0
    tr_concat_data            0            0K            0
       md_sii_data            0            0K            0
   md_promise_data            0            0K            0
    md_nvidia_data            0            0K            0
   md_jmicron_data            0            0K            0
     md_intel_data            0            0K            0
       md_ddf_data            0            0K            0
         raid_data            0            0K           72
     geom_flashmap            0            0K            0
         tmpfs dir            0            0K            0
        tmpfs name            0            0K            0
       tmpfs mount            0            0K            0
     tmpfs extattr            0            0K            0
           NFS FHA            0            0K            0
         newnfsmnt            0            0K            0
  newnfsclient_req            0            0K            0
   NFSCL layrecall            0            0K            0
     NFSCL session            0            0K            0
     NFSCL sockreq            0            0K            0
     NFSCL devinfo            0            0K            0
     NFSCL flayout            0            0K            0
      NFSCL layout            0            0K            0
     NFSD rollback            0            0K            0
      NFSCL diroff            0            0K            0
        NEWNFSnode            0            0K            0
         NFSCL lck            0            0K            0
      NFSCL lckown            0            0K            0
      NFSCL client            0            0K            0
       NFSCL deleg            0            0K            0
        NFSCL open            0            0K            0
       NFSCL owner            0            0K            0
            NFS fh            0            0K            0
           NFS req            0            0K            0
     NFSD usrgroup            0            0K            0
       NFSD string            0            0K            0
       NFSD V4lock            0            0K            0
      NFSD V4state            0            0K            0
       msdosfs_fat            0            0K            0
     msdosfs_mount            0            0K            0
      msdosfs_node            0            0K            0
            DEVFS2            0            0K            0
            gntdev            0            0K            0
       privcmd_dev            0            0K            0
        evtchn_dev            0            0K            0
          xenstore            0            0K            0
               xnb            0            0K            0
          xen_acpi            0            0K            0
              xbbd            0            0K            0
               xbd            0            0K            0
           Balloon            0            0K            0
          sysmouse            0            0K            0
            vtfont            0            0K            0
            pvscsi            0            0K            0
            USBdev            0            0K            0
               USB            0            0K            0
            twsbuf            0            0K            0
       tcp_log_dev            0            0K            0
      midi buffers            0            0K            0
             mixer            0            0K            0
              ac97            0            0K            0
             hdacc            0            0K            0
              hdac            0            0K            0
              hdaa            0            0K            0
       SIIS driver            0            0K            0
               PUC            0            0K            0
          ppbusdev            0            0K            0
            sr_iov            0            0K            0
               OCS            0            0K            0
               OCS            0            0K            0
              nvme            0            0K            0
               nvd            0            0K            0
            netmap            0            0K            0
            mwldev            0            0K            0
        MVS driver            0            0K            0
          mrsasbuf            0            0K            0
          mpt_user            0            0K            0
          mps_user            0            0K            0
            MPSSAS            0            0K            0
               mps            0            0K            0
          mpr_user            0            0K            0
            MPRSAS            0            0K            0
               mpr            0            0K            0
            mfibuf            0            0K            0
        md_sectors            0            0K            0
           md_disk            0            0K            0
           malodev            0            0K            0
               LED            0            0K            0
          ix_sriov            0            0K            0
                ix            0            0K            0
            ipsbuf            0            0K            0
         ciss_data            0            0K            0
         BACKLIGHT            0            0K            0
           ath_hal            0            0K            0
            athdev            0            0K            0
           ata_pci            0            0K            0
           ata_dma            0            0K            0
       ata_generic            0            0K            0
       AHCI driver            0            0K            0
               agp            0            0K            0
           acpipwr            0            0K            0
         acpi_perf            0            0K            0
         acpicmbat            0            0K            0
        aacraidcam            0            0K            0
       aacraid_buf            0            0K            0
            aaccam            0            0K            0
            aacbuf            0            0K            0
              zstd            0            0K            0
            XZ_DEC            0            0K            0
            nvlist            0            0K         1400
          SCSI ENC            0            0K            0
           SCSI sa            0            0K            0
         scsi_pass            0            0K            0
           scsi_da            0            0K           69
            ata_da            0            0K            0
           scsi_ch            0            0K            0
           scsi_cd            0            0K            0
           nvme_da            0            0K            0
           CAM CCB            0            0K          523
     CAM ccb queue            0            0K            0
db> show uma
              Zone   Size    Used    Free    Requests  Sleeps  Bucket  Total Mem    XFree
   mbuf_jumbo_page   4096    8320    1078       12800       0     254   38494208        0
              mbuf    256    8633    1029       21501       0     254    2473472        0
          BUF TRIE    152     298   11506        1126       0      62    1794208        0
       malloc-4096   4096     390       4        1381       0       2    1613824        0
        malloc-384    384    4149      51        4162       0      30    1612800        0
        malloc-128    128   12305     188       12519       0     126    1599104        0
       UMA Slabs 0    112   10911      27       10911       0     126    1225056        0
        RADIX NODE    152    7756     247       30891       0      62    1216456        0
         sctp_asoc   2256       1     509          11       0     254    1150560        0
      malloc-65536  65536      14       2          79       0       1    1048576        0
      mbuf_cluster   2048     508       0         508       0     254    1040384        0
         vmem btag     56   16218      93       16218       0     254     913416        0
         FFS inode   1168     530      30         656       0       8     654080        0
              pbuf   2624       0     249           0       0       2     653376        0
           sctp_ep   1152       4     507          75       0     254     588672        0
            socket   1024      32     476        1501       0     254     520192        0
          lkpicurr    168       2    3094           2       0      62     520128        0
        sctp_raddr    736       1     516          26       0     254     380512        0
      malloc-16384  16384      18       3          97       0       1     344064        0
        256 Bucket   2048     141      11        1029       0       8     311296        0
         VM OBJECT    264    1053      87       14140       0      30     300960        0
             VNODE    440     570      96         698       0      30     293040        0
            THREAD   1824     139      13         250       0       8     277248        0
      malloc-32768  32768       2       6         104       0       1     262144        0
       malloc-2048   2048     111      17         976       0       8     262144        0
         malloc-64     64    3942     153       13562       0     254     262080        0
         malloc-16     16   14333     417       14578       0     254     236000        0
            DEVCTL   1024      21     199         148       0       0     225280        0
           tcp_log    416       0     513          48       0     254     213408        0
      malloc-32768  32768       4       2         124       0       1     196608        0
         UMA Zones    768     237       2         237       0      16     183552        0
         malloc-32     32    5326     344        5495       0     254     181440        0
        malloc-128    128    1212     183       25826       0     126     178560        0
            lkpimm     56       1    3095           1       0     254     173376        0
             unpcb    320      10     506        1179       0     254     165120        0
     FPU_save_area    832     141      39         310       0      16     149760        0
       FFS2 dinode    256     530      40         656       0      62     145920        0
        malloc-256    256      49     521        2023       0      62     145920        0
         MAP ENTRY     96    1220     292       45037       0     126     145152        0
       S VFS Cache    104    1000     287        1184       0     126     133848        0
    ertt_txseginfo     40       0    3333        4807       0     254     133320        0
      malloc-65536  65536       2       0           2       0       1     131072        0
      malloc-65536  65536       0       2         110       0       1     131072        0
      malloc-65536  65536       1       1           9       0       1     131072        0
       malloc-1024   1024     122       6         314       0      16     131072        0
       mbuf_packet    256       5     503         718       0     254     130048        0
          ksiginfo    112      55     989          99       0     126     116928        0
       malloc-8192   8192      12       2          16       0       1     114688        0
      malloc-16384  16384       6       0           7       0       1      98304        0
        malloc-256    256     190     185        1090       0      62      96000        0
              PROC   1376      59       7         886       0       8      90816        0
          UMA Kegs    384     223       0         223       0      30      85632        0
         malloc-64     64     935     388        3170       0     254      84672        0
        malloc-128    128     440     211        4927       0     126      83328        0
         filedesc0   1072      60      17         890       0       8      82544        0
       malloc-2048   2048      12      28         524       0       8      81920        0
        sctp_chunk    152       0     520          15       0     254      79040        0
       malloc-4096   4096      15       3         502       0       2      73728        0
             g_bio    408       0     180        7532       0      30      73440        0
         64 Bucket    512      83      53        1877       0      30      69632        0
        128 Bucket   1024      53      14         251       0      16      68608        0
         malloc-64     64     575     496        3194       0     254      68544        0
      malloc-32768  32768       0       2          13       0       1      65536        0
       malloc-8192   8192       6       2          88       0       1      65536        0
        malloc-256    256     155     100         517       0      62      65280        0
        malloc-256    256     164      91         318       0      62      65280        0
sctp_stream_msg_out    112       1     539           4       0     254      60480        0
       malloc-4096   4096      12       2         206       0       2      57344        0
       tcp_bbr_map    128       0     403         354       0     126      51584        0
        malloc-256    256      29     166         384       0      62      49920        0
         32 Bucket    256      76     119        1663       0      62      49920        0
           DIRHASH   1024      35      13          35       0      16      49152        0
             NAMEI   1024       0      48       13295       0      16      49152        0
      malloc-16384  16384       1       2         181       0       1      49152        0
       malloc-2048   2048      11      13          97       0       8      49152        0
       malloc-1024   1024       6      42         514       0      16      49152        0
        malloc-384    384      16     104         337       0      30      46080        0
        malloc-384    384      88      32          90       0      30      46080        0
          syncache    168       0     264           3       0     254      44352        0
         tcp_inpcb   1304       7      26          53       0       8      43032        0
       malloc-4096   4096       6       4          10       0       2      40960        0
            pcpu-8      8    4694     426        4864       0     254      40960        0
           VMSPACE    584      39      31         867       0      16      40880        0
        sctp_readq    152       0     260           3       0     254      39520        0
            da_ccb    544       0      70        2008       0      16      38080        0
         udp_inpcb    408       9      81         160       0      30      36720        0
         hostcache     64       3     564           3       0     254      36288        0
         malloc-64     64      50     517       13788       0     254      36288        0
         malloc-64     64      40     527          59       0     254      36288        0
         malloc-64     64      62     505        1554       0     254      36288        0
      tcp_rack_map    128       0     279         513       0     126      35712        0
        malloc-128    128      19     260         103       0     126      35712        0
        malloc-128    128      17     262          40       0     126      35712        0
        malloc-128    128       5     274         184       0     126      35712        0
        malloc-128    128     110     169         577       0     126      35712        0
        malloc-128    128      13     266         282       0     126      35712        0
     routing nhops    256      27     108          34       0      62      34560        0
           ttyoutq    256      72      63         160       0      62      34560        0
        malloc-384    384      59      31         437       0      30      34560        0
        malloc-256    256      45      90         558       0      62      34560        0
        malloc-256    256      35     100          36       0      62      34560        0
        malloc-256    256      11     124         524       0      62      34560        0
         TURNSTILE    136     153      99         153       0      62      34272        0
        SLEEPQUEUE     88     153     231         153       0     126      33792        0
      malloc-32768  32768       1       0           1       0       1      32768        0
       malloc-4096   4096       6       2           9       0       2      32768        0
       malloc-2048   2048       1      15          13       0       8      32768        0
       malloc-2048   2048       9       7           9       0       8      32768        0
       malloc-2048   2048       2      14           4       0       8      32768        0
       malloc-1024   1024       4      28          53       0      16      32768        0
       malloc-1024   1024      15      17        1015       0      16      32768        0
       malloc-1024   1024      18      14          39       0      16      32768        0
        malloc-512    512       1      63         117       0      30      32768        0
        malloc-512    512       8      56          79       0      30      32768        0
        malloc-512    512       0      64           4       0      30      32768        0
        malloc-512    512       5      59          73       0      30      32768        0
        malloc-512    512       8      56           8       0      30      32768        0
        malloc-512    512       2      62           5       0      30      32768        0
           pcpu-64     64     495      17         495       0     254      32768        0
       tcp_bbr_pcb    896       2      34          23       0      16      32256        0
            ttyinq    160     135      65         300       0      62      32000        0
              PGRP    120      20     244          40       0     126      31680        0
            clpbuf   2624       0      12          56       0       4      31488        0
        sctp_laddr     48       0     588          16       0     254      28224        0
         malloc-32     32     300     582         949       0     254      28224        0
         16 Bucket    144      58     138         372       0      62      28224        0
          4 Bucket     48       7     581          11       0     254      28224        0
               AIO    208       0     133           6       0      62      27664        0
            cpuset    200       8     120          15       0      62      25600        0
       malloc-8192   8192       2       1          28       0       1      24576        0
       malloc-4096   4096       1       5         154       0       2      24576        0
              pipe    736      19      14         289       0      16      24288        0
               PWD     40      26     580         156       0     254      24240        0
      tcp_rack_pcb   1152       0      21           2       0       8      24192        0
           rtentry    168      30     114          34       0      62      24192        0
             Files     80     179     121        7221       0     126      24000        0
          8 Bucket     80      46     254         308       0     126      24000        0
        malloc-384    384      11      49          11       0      30      23040        0
        malloc-384    384      21      39          29       0      30      23040        0
        malloc-384    384      23      37         335       0      30      23040        0
         domainset     40       0     567          10       0     254      22680        0
             ripcb    376       1      59           8       0      30      22560        0
       Mountpoints   2816       2       6           6       0       4      22528        0
              ertt     72       7     273          53       0     126      20160        0
         malloc-64     64       6     309          10       0     254      20160        0
         malloc-32     32      75     555          99       0     254      20160        0
         malloc-32     32     276     354        2203       0     254      20160        0
         malloc-32     32      44     586        5058       0     254      20160        0
         malloc-32     32      14     616          20       0     254      20160        0
          2 Bucket     32      54     576         374       0     254      20160        0
             KNOTE    160       3     122          44       0      62      20000        0
             AIOCB    552       0      35           4       0      16      19320        0
       L VFS Cache    320       0      60           4       0      30      19200        0
              vmem   1856       2       7           2       0       8      16704        0
 epoch_record pcpu    256       4      60           4       0      62      16384        0
      malloc-16384  16384       1       0           1       0       1      16384        0
       malloc-8192   8192       2       0           2       0       1      16384        0
       malloc-8192   8192       1       1          20       0       1      16384        0
       malloc-4096   4096       0       4          13       0       2      16384        0
       malloc-4096   4096       0       4          41       0       2      16384        0
       malloc-2048   2048       1       7           1       0       8      16384        0
       malloc-2048   2048       2       6           2       0       8      16384        0
       malloc-1024   1024       4      12           4       0      16      16384        0
       malloc-1024   1024       8       8           8       0      16      16384        0
           SMR CPU     32       8     503           8       0     254      16352        0
         malloc-16     16     532     468        4939       0     254      16000        0
              kenv    258      17      43        1066       0      30      15480        0
            mqnode    416       3      33           3       0      30      14976        0
        SMR SHARED     24       8     503           8       0     254      12264        0
         malloc-32     32      21     357         176       0     254      12096        0
         malloc-16     16      16     734         217       0     254      12000        0
         malloc-16     16      50     700       27394       0     254      12000        0
         malloc-16     16      13     737          13       0     254      12000        0
         malloc-16     16      31     719          37       0     254      12000        0
         malloc-16     16      13     737          14       0     254      12000        0
           cryptop    280       0      42           1       0      30      11760        0
            itimer    352       0      33           1       0      30      11616        0
        malloc-384    384       1      29           1       0      30      11520        0
            AIOLIO    272       0      42           2       0      30      11424        0
       malloc-8192   8192       1       0           1       0       1       8192        0
       malloc-8192   8192       1       0           1       0       1       8192        0
           pcpu-16     16       8     504           8       0     254       8192        0
      vtnet_tx_hdr     24       0     334        5522       0     254       8016        0
         malloc-16     16      10     490          10       0     254       8000        0
       UMA Slabs 1    176       9      13           9       0      62       3872        0
        KMAP ENTRY     96      12      27          14       0       0       3744        0
       FFS1 dinode    128       0       0           0       0     126          0        0
           ada_ccb    272       0       0           0       0      30          0        0
             swblk    136       0       0           0       0      62          0        0
          swpctrie    152       0       0           0       0      62          0        0
   cdg_qdiffsample     16       0       0           0       0     254          0        0
   pf state scrubs     40       0       0           0       0     254          0        0
   pf frag entries     40       0       0           0       0     254          0        0
          pf frags    256       0       0           0       0      62          0        0
  pf table entries    160       0       0           0       0     254          0        0
pf table entry counters     64       0       0           0       0     254          0        0
   pf UDP mappings    104       0       0           0       0     126          0        0
   pf source nodes    152       0       0           0       0     254          0        0
     pf state keys     88       0       0           0       0     126          0        0
         pf states    376       0       0           0       0     254          0        0
           pf tags    104       0       0           0       0     126          0        0
          pf mtags    184       0       0           0       0      62          0        0
tfo_ccache_entries     80       0       0           0       0     126          0        0
               tfo      4       0       0           0       0     254          0        0
          sackhole     32       0       0           0       0     254          0        0
               ipq     56       0       0           0       0     254          0        0
   sctp_asconf_ack     48       0       0           0       0     254          0        0
       sctp_asconf     40       0       0           0       0     254          0        0
   tcp_log_id_node    120       0       0           0       0     126          0        0
 tcp_log_id_bucket    176       0       0           0       0      62          0        0
          tcpreass     48       0       0           0       0     254          0        0
     udplite_inpcb    408       0       0           0       0      30          0        0
    IPsec SA lft_c     16       0       0           0       0     254          0        0
        mqnotifier    216       0       0           0       0      62          0        0
            mvdata     64       0       0           0       0     254          0        0
            mqueue    248       0       0           0       0      62          0        0
        TMPFS node    240       0       0           0       0      62          0        0
           NCLNODE    608       0       0           0       0      16          0        0
     LTS VFS Cache    360       0       0           0       0      30          0        0
     STS VFS Cache    144       0       0           0       0      62          0        0
  linux_dma_object     32       0       0           0       0     254          0        0
  linux_dma_pctrie    152       0       0           0       0      62          0        0
   IOMMU_MAP_ENTRY    112       0       0           0       0     126          0        0
    mbuf_jumbo_16k  16384       0       0           0       0     254          0        0
     mbuf_jumbo_9k   9216       0       0           0       0     254          0        0
      audit_record   1280       0       0           0       0       8          0        0
        MAC labels     40       0       0           0       0     254          0        0
            vnpbuf   2624       0       0           0       0      16          0        0
           nfspbuf   2624       0       0           0       0       4          0        0
            swwbuf   2624       0       0           0       0       2          0        0
            swrbuf   2624       0       0           0       0       4          0        0
          umtx_shm     88       0       0           0       0     126          0        0
           umtx pi     96       0       0           0       0     126          0        0
rangeset pctrie nodes    152       0       0           0       0      62          0        0
          rl_entry     48       0       0           0       0     254          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
       malloc-8192   8192       0       0           0       0       1          0        0
       malloc-1024   1024       0       0           0       0      16          0        0
        malloc-512    512       0       0           0       0      30          0        0
        malloc-512    512       0       0           0       0      30          0        0
         malloc-64     64       0       0           0       0     254          0        0
         malloc-32     32       0       0           0       0     254          0        0
           pcpu-32     32       0       0           0       0     254          0        0
            pcpu-4      4       0       0           0       0     254          0        0
            fakepg    104       0       0           0       0     126          0        0
          UMA Hash    256       0       0           0       0      62          0        0

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/03/22 23:21 freebsd-src 23db5fccb60a c6512ef7 console log report ci-freebsd-main panic: Empty stailq ADDR->stqh_last is ADDR, not head's first field address
* Struck through repros no longer work on HEAD.