syzbot


Fatal trap NUM: page fault in tcp_input_with_port

Status: fixed on 2023/07/27 11:28
Reported-by: syzbot+e7d2e451f89fb444319b@syzkaller.appspotmail.com
Fix commit: a43e7a96b64e inpcb: use internal flag to mark pcbs that are inserted into lbgroup
First crash: 541d, last: 501d

Sample crash report:
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0xb8
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff81961d39
stack pointer	        = 0x28:0xfffffe0053fbb760
executing program
frame pointer	        = 0x28:0xfffffe0053fbba20
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi1: netisr 0)
rdi: 00000000000000b8 rsi: 0000000000000000 rdx: ffffffff82771e80
rcx: fffffe00033eee30  r8: 0000000000000000  r9: ffffffff82767c20
rax: fffffe00033eee30 rbx: fffffe006cd8a088 rbp: fffffe0053fbba20
r10: 00000000000005c0 r11: 0000000000000002 r12: fffffe006d128300
r13: fffffe006d128347 r14: 0000000000000000 r15: fffffe006cd8a000
trap number		= 12
panic: page fault
cpuid = 0
time = 1689028549
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0xc6/frame 0xfffffe0053fbaf50
kdb_backtrace() at kdb_backtrace+0xd0/frame 0xfffffe0053fbb0b0
vpanic() at vpanic+0x24b/frame 0xfffffe0053fbb190
panic() at panic+0xb5/frame 0xfffffe0053fbb250
trap_fatal() at trap_fatal+0x7ee/frame 0xfffffe0053fbb370
trap_pfault() at trap_pfault+0x17b/frame 0xfffffe0053fbb4b0
trap() at trap+0x5f3/frame 0xfffffe0053fbb690
calltrap() at calltrap+0x8/frame 0xfffffe0053fbb690
--- trap 0xc, rip = 0xffffffff81961d39, rsp = 0xfffffe0053fbb760, rbp = 0xfffffe0053fbba20 ---
tcp_input_with_port() at tcp_input_with_port+0x1109/frame 0xfffffe0053fbba20
tcp6_input_with_port() at tcp6_input_with_port+0xeb/frame 0xfffffe0053fbba60
tcp6_input() at tcp6_input+0x26/frame 0xfffffe0053fbba90
ip6_input() at ip6_input+0x2297/frame 0xfffffe0053fbbcf0
swi_net() at swi_net+0x2f3/frame 0xfffffe0053fbbd90
ithread_loop() at ithread_loop+0x4eb/frame 0xfffffe0053fbbef0
fork_exit() at fork_exit+0xc9/frame 0xfffffe0053fbbf30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0053fbbf30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
[ thread pid 12 tid 100031 ]
Stopped at      kdb_enter+0x6e: movq    $0,0x2133517(%rip)
db> 
db> set $lines = 0
db> set $maxwidth = 0
db> show registers
cs                        0x20
ds                        0x3b
es                        0x3b
fs                        0x13
gs                        0x1b
ss                        0x28
rax                       0x12
rcx         0xfffffe00033eee30
rdx         0xdffff7c000000000
rbx         0xffffffff826bbb80  .str.28
rsp         0xfffffe0053fbb090
rbp         0xfffffe0053fbb0b0
rsi                        0x1
rdi                          0
r8                           0
r9                  0xffffffff
r10                          0
r11                        0x1
r12                          0
r13         0xfffffe00542c2560
r14         0xffffffff826bbb80  .str.28
r15                          0
rip         0xffffffff815beb7e  kdb_enter+0x6e
rflags                    0x46
kdb_enter+0x6e: movq    $0,0x2133517(%rip)
db> show proc
Process 12 (intr) at 0xfffffe00541fa580:
 state: NORMAL
 uid: 0  gids: 0
 parent: pid 0 at 0xffffffff836aa060
 ABI: null
 flag: 0x10000284  flag2: 0
 reaper: 0xffffffff836aa060 reapsubtree: 12
 sigparent: 20
 vmspace: 0xffffffff836ab000
   (map 0xffffffff836ab000)
   (map.pmap 0xffffffff836ab0c0)
   (pmap 0xffffffff836ab130)
 threads: 22
100016                   I                                   [swi5: fast taskq]
100019                   I                                   [swi6: task queue]
100020                   I                                   [swi6: Giant taskq]
100031                   Run     CPU 0                       [swi1: netisr 0]
100032                   I                                   [swi1: hpts]
100033                   I                                   [swi1: hpts]
100046                   I                                   [irq24: virtio_pci0]
100047                   I                                   [irq25: virtio_pci0]
100048                   I                                   [irq26: virtio_pci0]
100049                   I                                   [irq27: virtio_pci0]
100050                   I                                   [irq28: virtio_pci1]
100051                   I                                   [irq29: virtio_pci1]
100052                   I                                   [irq30: virtio_pci1]
100053                   I                                   [irq31: virtio_pci1]
100054                   I                                   [irq32: virtio_pci1]
100059                   I                                   [irq33: virtio_pci2]
100060                   I                                   [irq34: virtio_pci2]
100061                   I                                   [irq35: virtio_pci2]
100063                   I                                   [irq1: atkbd0]
100064                   I                                   [irq12: psm0]
100065                   I                                   [swi0: uart uart++]
100069                   I                                   [swi1: pf send]
db> ps
  pid  ppid  pgrp   uid  state   wmesg   wchan               cmd
 4939   780   774     0  R       (threaded)                  syz-executor4164038
100426                   RunQ                                syz-executor4164038
105096                   S       connec  0xfffffe0058cde85a  syz-executor4164038
 4938   777   774     0  R       (threaded)                  syz-executor4164038
100763                   RunQ                                syz-executor4164038
105094                   S       connec  0xfffffe0058cd685a  syz-executor4164038
 4937   778   774     0  R       (threaded)                  syz-executor4164038
101338                   RunQ                                syz-executor4164038
105095                   S       connec  0xfffffe0058cd5c1a  syz-executor4164038
 4935   781   774     0  R       (threaded)                  syz-executor4164038
100103                   RunQ                                syz-executor4164038
105092                   RunQ                                syz-executor4164038
  781   776   774     0  R                                   syz-executor4164038
  780   776   774     0  R                                   syz-executor4164038
  778   776   774     0  R                                   syz-executor4164038
  777   776   774     0  R                                   syz-executor4164038
  776   774   774     0  S       nanslp  0xffffffff836d3841  syz-executor4164038
  774   772   774     0  Ss      pause   0xfffffe0058ecdbb0  csh
  772   682   772     0  Ss      select  0xfffffe006c9ca5c0  sshd
  748     1   748     0  Ss+     ttyin   0xfffffe00576728b0  getty
  747     1   747     0  Ss+     ttyin   0xfffffe0057687cb0  getty
  746     1   746     0  Ss+     ttyin   0xfffffe00589e08b0  getty
  745     1   745     0  Ss+     ttyin   0xfffffe00589e10b0  getty
  744     1   744     0  Ss+     ttyin   0xfffffe00589e18b0  getty
  743     1   743     0  Ss+     ttyin   0xfffffe005433e0b0  getty
  742     1   742     0  Ss+     ttyin   0xfffffe005433e8b0  getty
  741     1   741     0  Ss+     ttyin   0xfffffe005433f0b0  getty
  740     1   740     0  Ss+     ttyin   0xfffffe005433f8b0  getty
  686     1   686     0  Ss      nanslp  0xffffffff836d3840  cron
  682     1   682     0  Ss      select  0xfffffe006c9cb040  sshd
  495     1   495     0  Ds      getbuf  0xfffffe0007e36ea4  syslogd
  424     1   424     0  Ss      select  0xfffffe006c9cb2c0  devd
  423     1   423    65  Ss      select  0xfffffe006cc25dc0  dhclient
  338     1   338     0  Ss      select  0xfffffe006c9cb1c0  dhclient
  335     1   335     0  Ss      select  0xfffffe006c9cb840  dhclient
   17     0     0     0  DL      vlruwt  0xfffffe00541fc5c0  [vnlru]
   16     0     0     0  DL      syncer  0xffffffff837d5d20  [syncer]
   15     0     0     0  DL      (threaded)                  [bufdaemon]
100079                   D       psleep  0xffffffff837d4340  [bufdaemon]
100082                   D       -       0xffffffff82c0a140  [bufspacedaemon-0]
100094                   D       sdflush 0xfffffe00085f98e8  [/ worker]
    9     0     0     0  DL      psleep  0xffffffff8380bc00  [vmdaemon]
    8     0     0     0  DL      (threaded)                  [pagedaemon]
100077                   D       psleep  0xffffffff837ffab8  [dom0]
100080                   D       launds  0xffffffff837ffac4  [laundry: dom0]
100081                   D       umarcl  0xffffffff81d3e340  [uma]
    7     0     0     0  DL      -       0xffffffff83497e48  [rand_harvestq]
    6     0     0     0  DL      pftm    0xffffffff844713d0  [pf purge]
    5     0     0     0  DL      waiting 0xffffffff8425c1c0  [sctp_iterator]
    4     0     0     0  RL      (threaded)                  [cam]
100044                   Run     CPU 1                       [doneq0]
100045                   D       -       0xffffffff8347a2c0  [async]
100076                   D       -       0xffffffff8347a140  [scanner]
   14     0     0     0  DL      seqstat 0xfffffe00085fb888  [sequencer 00]
    3     0     0     0  DL      (threaded)                  [crypto]
100040                   D       crypto_ 0xffffffff837fb360  [crypto]
100041                   D       crypto_ 0xfffffe005436f030  [crypto returns 0]
100042                   D       crypto_ 0xfffffe005436f080  [crypto returns 1]
   13     0     0     0  DL      (threaded)                  [geom]
100035                   D       -       0xffffffff836a9640  [g_event]
100036                   D       -       0xffffffff836a9660  [g_up]
100037                   D       -       0xffffffff836a9680  [g_down]
    2     0     0     0  WL      (threaded)                  [clock]
100029                   I                                   [clock (0)]
100030                   I                                   [clock (1)]
   12     0     0     0  RL      (threaded)                  [intr]
100016                   I                                   [swi5: fast taskq]
100019                   I                                   [swi6: task queue]
100020                   I                                   [swi6: Giant taskq]
100031                   Run     CPU 0                       [swi1: netisr 0]
100032                   I                                   [swi1: hpts]
100033                   I                                   [swi1: hpts]
100046                   I                                   [irq24: virtio_pci0]
100047                   I                                   [irq25: virtio_pci0]
100048                   I                                   [irq26: virtio_pci0]
100049                   I                                   [irq27: virtio_pci0]
100050                   I                                   [irq28: virtio_pci1]
100051                   I                                   [irq29: virtio_pci1]
100052                   I                                   [irq30: virtio_pci1]
100053                   I                                   [irq31: virtio_pci1]
100054                   I                                   [irq32: virtio_pci1]
100059                   I                                   [irq33: virtio_pci2]
100060                   I                                   [irq34: virtio_pci2]
100061                   I                                   [irq35: virtio_pci2]
100063                   I                                   [irq1: atkbd0]
100064                   I                                   [irq12: psm0]
100065                   I                                   [swi0: uart uart++]
100069                   I                                   [swi1: pf send]
   11     0     0     0  RL      (threaded)                  [idle]
100003                   CanRun                              [idle: cpu0]
100004                   CanRun                              [idle: cpu1]
    1     0     1     0  SLs     wait    0xfffffe00541fb040  [init]
   10     0     0     0  DL      audit_w 0xffffffff837fbda0  [audit]
    0     0     0     0  DLs     (threaded)                  [kernel]
100000                   D       swapin  0xffffffff836aa060  [swapper]
100005                   D       -       0xfffffe00542a8100  [if_io_tqg_0]
100006                   D       -       0xfffffe00542a8000  [if_io_tqg_1]
100007                   D       -       0xfffffe00542a7e00  [if_config_tqg_0]
100008                   D       -       0xfffffe00542a7d00  [softirq_0]
100009                   D       -       0xfffffe00542a7c00  [softirq_1]
100010                   D       -       0xfffffe000799f000  [linuxkpi_irq_wq]
100011                   D       -       0xfffffe00085ffe00  [thread taskq]
100012                   D       -       0xfffffe00085ffd00  [inm_free taskq]
100013                   D       -       0xfffffe00085ffc00  [aiod_kick taskq]
100014                   D       -       0xfffffe00085ffb00  [deferred_unmount ta]
100015                   D       -       0xfffffe00085ffa00  [in6m_free taskq]
100017                   D       -       0xfffffe00085ff800  [kqueue_ctx taskq]
100018                   D       -       0xfffffe00085ff700  [pci_hp taskq]
100021                   D       -       0xfffffe00085ff400  [linuxkpi_short_wq_0]
100022                   D       -       0xfffffe00085ff400  [linuxkpi_short_wq_1]
100023                   D       -       0xfffffe00085ff400  [linuxkpi_short_wq_2]
100024                   D       -       0xfffffe00085ff400  [linuxkpi_short_wq_3]
100025                   D       -       0xfffffe00085ff300  [linuxkpi_long_wq_0]
100026                   D       -       0xfffffe00085ff300  [linuxkpi_long_wq_1]
100027                   D       -       0xfffffe00085ff300  [linuxkpi_long_wq_2]
100028                   D       -       0xfffffe00085ff300  [linuxkpi_long_wq_3]
100034                   D       -       0xfffffe00085ff200  [firmware taskq]
100038                   D       -       0xfffffe00085fe700  [crypto_0]
100039                   D       -       0xfffffe00085fe700  [crypto_1]
100055                   D       -       0xfffffe00085fde00  [vtnet0 rxq 0]
100056                   D       -       0xfffffe00085fdd00  [vtnet0 txq 0]
100057                   D       -       0xfffffe00085fdc00  [vtnet0 rxq 1]
100058                   D       -       0xfffffe00085fdb00  [vtnet0 txq 1]
100062                   D       vtbslp  0xfffffe0057176080  [virtio_balloon]
100066                   D       -       0xffffffff826c0b20  [deadlkres]
100070                   D       -       0xfffffe00085fe400  [acpi_task_0]
100071                   D       -       0xfffffe00085fe400  [acpi_task_1]
100072                   D       -       0xfffffe00085fe400  [acpi_task_2]
100073                   D       -       0xfffffe000799f100  [mca taskq]
100075                   D       -       0xfffffe00085fe000  [CAM taskq]
db> show all locks
Process 4935 (syz-executor4164038) thread 0xfffffe006d185560 (105092)
exclusive rw tcpinp (tcpinp) r = 0 (0xfffffe006d17e020) locked @ /syzkaller/managers/main/kernel/sys/netinet/tcp_usrreq.c:560
Process 495 (syslogd) thread 0xfffffe0058ead720 (100098)
exclusive lockmgr ufs (ufs) r = 0 (0xfffffe006cd0c070) locked @ /syzkaller/managers/main/kernel/sys/kern/vfs_syscalls.c:3551
Process 12 (intr) thread 0xfffffe00542c2560 (100031)
shared rw tcpinp (tcpinp) r = 0 (0xfffffe006cd8a020) locked @ /syzkaller/managers/main/kernel/sys/netinet/in_pcb.c:1472
db> show malloc
              Type        InUse        MemUse     Requests
           pf_hash            5        11524K            5
          tcp_hpts            7         4801K            7
            devbuf         4218         4324K         4246
         sysctloid        34809         2051K        34880
             vtbuf           24         1968K           46
          inodedep         3718         1906K         4236
              kobj          326         1304K          488
            newblk          145         1060K         4983
          vfscache            3         1025K            3
            dirrem         3707          927K         4183
               pcb           29          670K         4199
         ufs_quota            1          512K            1
          vfs_hash            1          512K            1
           callout            2          512K            2
              intr            4          472K            4
          freefile         3707          464K         4181
           subproc          112          206K         5007
            acpica         1674          184K        57212
           tidhash            3          141K            3
              vmem            3          134K            4
           pagedep           10          131K         4182
        tfo_ccache            1          128K            1
          IP reass            1          128K            1
            linker          324          127K          353
         vnet_data            1          112K            1
               sem            4          106K            4
            DEVFS1          105          105K          114
               bus          985           81K         5069
          mtx_pool            2           72K            2
     NFSD srvcache            3           68K            3
          syncache            1           68K            1
          acpitask            1           64K            1
       ddb_capture            1           64K            1
            module          508           64K          508
              temp           19           53K         1592
              umtx          308           39K          308
           kdtrace          182           38K        10037
         hostcache            1           32K            1
               shm            1           32K            1
            DEVFS3          124           31K          134
               msg            4           30K            4
            kbdmux            6           28K            6
        gtaskqueue           18           26K           18
        DEVFS_RULE           56           20K           56
               BPF           10           18K           10
         ufs_mount            4           17K            5
              proc            3           17K            3
               tty           16           16K           16
           ithread           97           16K           97
            bus-sc           34           15K         1648
      eventhandler          157           13K          157
            KTRACE          100           13K          100
              kenv           95           12K           95
            ifaddr           30           12K           32
              rman           88           11K          431
              GEOM           61           11K          481
          routetbl           50           11K          176
         CAM queue            5           11K         1528
         bmsafemap            2            9K         4204
               rpc            4            9K            4
              UART           12            9K           12
           devstat            4            9K            4
              ksem            1            8K            1
       pfs_vncache            1            8K            1
             shmfd            1            8K            1
     audit_evclass          237            8K          297
         taskqueue           63            7K           63
            sglist            5            7K            5
           CAM DEV            3            6K          510
              cred           22            6K          244
         pfs_nodes           20            5K           20
            plimit           20            5K          362
       ufs_dirhash           24            5K           24
               UMA          267            5K          267
                vt           11            5K           11
             ifnet            3            5K            3
           memdesc            1            4K            1
               MCA           32            4K           32
          filedesc            1            4K            1
             evdev            4            4K            4
           acpisem           28            4K           28
             hhook           15            4K           17
       ether_multi           40            4K           50
           lltable           11            4K           11
          pf_ifnet            5            3K            6
         in6_multi           25            3K           25
            kqueue           45            3K         4942
           pwddesc           45            3K         4940
            CC Mem           11            3K         8325
          terminal           11            3K           11
           session           19            3K           32
             clone            9            3K            9
           uidinfo            3            3K            9
         proc-args           64            3K         5873
        local_apic            1            2K            1
           io_apic            1            2K            1
       fpukern_ctx            2            2K            2
         ipsec-saq            2            2K            2
             lockf           16            2K           26
            Unitno           27            2K           41
           CAM XPT           22            2K          543
               msi           12            2K           12
             mkdir           12            2K         8342
             selfd           22            2K        64925
       ipsecpolicy            2            2K            2
           acpidev           20            2K           20
            diradd            9            2K         4199
          freework            5            2K         4181
      NFSD session            1            1K            1
           softdep            1            1K            1
         newdirblk            8            1K         4171
          freeblks            4            1K         4180
            sahead            1            1K            1
          secasvar            1            1K            1
       vnodemarker            2            1K           20
        CAM periph            4            1K          271
            select            7            1K           29
             ipsec            3            1K            3
             nhops            6            1K            6
              pfil            6            1K            6
         toponodes            6            1K            6
            isadev            6            1K            6
             mount           16            1K           89
          pci_link           10            1K           10
          sctp_ifa            5            1K            6
            crypto            4            1K            4
            ip6ndp            4            1K            5
 encap_export_host           12            1K           12
       inpcbpolicy           18            1K         8457
               osd           16            1K         8338
          in_multi            2            1K            4
              cdev            2            1K            2
    chacha20random            1            1K            1
            biobuf            1            1K            1
          sctp_ifn            2            1K            6
      NFSD lckfile            1            1K            1
     NFSD V4client            1            1K            1
            DEVFSP            4            1K            9
             DEVFS            9            1K           10
          indirdep            1            1K            3
               mld            2            1K            2
              igmp            2            1K            2
            vnodes            1            1K            1
           CAM SIM            2            1K            2
            soname            8            1K        11713
            feeder            7            1K            7
           tcpfunc            3            1K            3
        loginclass            3            1K            7
            prison            6            1K            6
       lkpikmalloc            5            1K            6
        aesni_data            2            1K            2
         cryptodev            2            1K           49
          nexusdev            8            1K            8
            apmdev            1            1K            1
          atkbddev            2            1K            2
          freefrag            1            1K            2
           netlink            1            1K            1
     CAM dev queue            2            1K            2
 CAM I/O Scheduler            1            1K            1
          CAM path            4            1K         1034
          procdesc            1            1K            6
          pmchooks            1            1K            1
               tun            3            1K            3
          sctp_vrf            1            1K            1
              vnet            1            1K            1
           entropy            2            1K           39
               pmc            1            1K            1
          acpiintr            1            1K            1
              cpus            2            1K            2
    vnet_data_free            1            1K            1
           Per-cpu            1            1K            1
          p1003.1b            1            1K            1
          filecaps            1            1K           66
          pf_table            0            0K            0
           pf_rule            0            0K            0
           pf_altq            0            0K            0
           pf_osfp            0            0K            0
     pf_krule_item            0            0K            0
           pf_temp            0            0K            0
            mqdata            0            0K            0
           filemon            0            0K            0
        sctp_mcore            0            0K            0
        sctp_socko            0            0K            0
         sctp_iter            0            0K            4
         sctp_mvrf            0            0K            0
         sctp_timw            0            0K            0
         sctp_cpal            0            0K            0
         sctp_cmsg            0            0K            0
         sctp_stre            0            0K            0
         sctp_athi            0            0K            0
         sctp_athm            0            0K            0
         sctp_atky            0            0K            0
         sctp_atcl            0            0K            0
         sctp_a_it            0            0K            4
         sctp_aadr            0            0K            0
         sctp_stro            0            0K            0
         sctp_stri            0            0K            0
          sctp_map            0            0K            0
            tcp_do            0            0K            0
           tcp_fsb            0            0K            0
            ipcomp            0            0K            0
               esp            0            0K            0
                ah            0            0K            0
     md_intel_data            0            0K            0
       md_ddf_data            0            0K            0
        madt_table            0            0K            2
          smartpqi            0            0K            0
         raid_data            0            0K           72
     geom_flashmap            0            0K            0
               ixl            0            0K            0
         tmpfs dir            0            0K            0
        tmpfs name            0            0K            0
       tmpfs mount            0            0K            0
     tmpfs extattr            0            0K            0
           NFS FHA            0            0K            0
        ice-resmgr            0            0K            0
         ice-osdep            0            0K            0
               ice            0            0K            0
              iavf            0            0K            0
             axgbe            0            0K            0
         newnfsmnt            0            0K            0
  newnfsclient_req            0            0K            0
   NFSCL layrecall            0            0K            0
     NFSCL session            0            0K            0
     NFSCL sockreq            0            0K            0
     NFSCL devinfo            0            0K            0
     NFSCL flayout            0            0K            0
      NFSCL layout            0            0K            0
     NFSD rollback            0            0K            0
          xen_intr            0            0K            0
      NFSCL diroff            0            0K            0
       NEWdirectio            0            0K            0
           xen_hvm            0            0K            0
         legacydrv            0            0K            0
            bounce            0            0K            0
            busdma            0            0K            0
            qpidrv            0            0K            0
        NEWNFSnode            0            0K            0
         NFSCL lck            0            0K            0
      dmar_idpgtbl            0            0K            0
          dmar_dom            0            0K            0
          dmar_ctx            0            0K            0
      NFSCL lckown            0            0K            0
      NFSCL client            0            0K            0
       NFSCL deleg            0            0K            0
              isci            0            0K            0
      iommu_dmamap            0            0K            0
        NFSCL open            0            0K            0
     hyperv_socket            0            0K            0
           bxe_ilt            0            0K            0
       NFSCL owner            0            0K            0
            xenbus            0            0K            0
            NFS fh            0            0K            0
           NFS req            0            0K            0
     NFSD usrgroup            0            0K            0
     vm_fictitious            0            0K            0
       NFSD string            0            0K            0
       NFSD V4lock            0            0K            0
      NFSD V4state            0            0K            0
       msdosfs_fat            0            0K            0
     msdosfs_mount            0            0K            0
      msdosfs_node            0            0K            0
           UMAHash            0            0K            0
            DEVFS4            0            0K            0
         vm_pgdata            0            0K            0
           jblocks            0            0K            0
          savedino            0            0K          474
          sentinel            0            0K            0
            jfsync            0            0K            0
            jtrunc            0            0K            0
             sbdep            0            0K            6
           jsegdep            0            0K            0
              jseg            0            0K            0
         jfreefrag            0            0K            0
          jfreeblk            0            0K            0
           jnewblk            0            0K            0
            jmvref            0            0K            0
           jremref            0            0K            0
           jaddref            0            0K            0
           freedep            0            0K            0
        allocindir            0            0K            0
       allocdirect            0            0K            0
          ufs_trim            0            0K            0
           mactemp            0            0K            0
     audit_trigger            0            0K            0
 audit_pipe_presel            0            0K            0
     audit_pipeent            0            0K            0
        audit_pipe            0            0K            0
      audit_evname            0            0K            0
         audit_bsm            0            0K            0
      audit_gidset            0            0K            0
        audit_text            0            0K            0
        audit_path            0            0K            0
        audit_data            0            0K            0
        audit_cred            0            0K            0
            DEVFS2            0            0K            0
            gntdev            0            0K            0
       privcmd_dev            0            0K            0
        evtchn_dev            0            0K            0
          xenstore            0            0K            0
         scsi_pass            0            0K            0
         ciss_data            0            0K            0
               xnb            0            0K            0
          xen_acpi            0            0K            0
              xbbd            0            0K            0
               xbd            0            0K            0
           Balloon            0            0K            0
          sysmouse            0            0K            0
            vtfont            0            0K            0
          ktls_ocf            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
      MLX5E_TLS_RX            0            0K            0
        MLX5EEPROM            0            0K            0
         MLX5E_TLS            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
            MLX5EN            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
          MLX5DUMP            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
        MLX5EEPROM            0            0K            0
       simple_attr            0            0K            0
          seq_file            0            0K            0
           lkpiskb            0            0K            0
             radix            0            0K            0
               idr            0            0K            0
          lkpindev            0            0K            0
           lkpimhi            0            0K            0
            lkpifw            0            0K            0
         lkpi80211            0            0K            0
               NLM            0            0K            0
    ipsec-spdcache            0            0K            0
         ipsec-reg            0            0K            0
        ipsec-misc            0            0K            0
      ipsecrequest            0            0K            0
            ip6opt            0            0K            3
       ip6_msource            0            0K            0
      ip6_moptions            0            0K            0
       in6_mfilter            0            0K            0
             frag6            0            0K            0
            tcplog            0            0K            0
        tcp_hwpace            0            0K            0
         BACKLIGHT            0            0K            0
               LRO            0            0K            0
        ip_msource            0            0K            0
       ip_moptions            0            0K            0
        in_mfilter            0            0K            0
              ipid            0            0K            0
         80211scan            0            0K            0
      80211ratectl            0            0K            0
        80211power            0            0K            0
       80211nodeie            0            0K            0
         80211node            0            0K            0
      80211mesh_gt            0            0K            0
      80211mesh_rt            0            0K            0
         80211perr            0            0K            0
         80211prep            0            0K            0
         80211preq            0            0K            0
          80211dfs            0            0K            0
       80211crypto            0            0K            0
          80211vap            0            0K            0
             iflib            0            0K            0
              vlan            0            0K            0
               gif            0            0K            0
           ifdescr            0            0K            0
              zlib            0            0K            0
           fadvise            0            0K            0
           VN POLL            0            0K            0
           ath_hal            0            0K            0
            statfs            0            0K         4354
     namei_tracker            0            0K            0
       export_host            0            0K            0
        cl_savebuf            0            0K            3
            athdev            0            0K            0
           ata_pci            0            0K            0
           ata_dma            0            0K            0
       ata_generic            0            0K            0
            pvscsi            0            0K            0
           scsi_da            0            0K           69
            ata_da            0            0K            0
           scsi_ch            0            0K            0
           scsi_cd            0            0K            0
       AHCI driver            0            0K            0
            USBdev            0            0K            0
               USB            0            0K            0
               agp            0            0K            0
           nvme_da            0            0K            0
           acpipwr            0            0K            0
         acpi_perf            0            0K            0
            twsbuf            0            0K            0
       tcp_log_dev            0            0K            0
               aio            0            0K            0
               lio            0            0K            0
               acl            0            0K            0
      midi buffers            0            0K            0
          mbuf_tag            0            0K            0
              ktls            0            0K            0
             mixer            0            0K            0
              ac97            0            0K            0
             hdacc            0            0K            0
              hdac            0            0K            0
              hdaa            0            0K            0
         acpicmbat            0            0K            0
       SIIS driver            0            0K            0
           CAM CCB            0            0K          523
               PUC            0            0K            0
          ppbusdev            0            0K            0
            sr_iov            0            0K            0
               OCS            0            0K            0
               OCS            0            0K            0
              nvme            0            0K            0
               nvd            0            0K            0
            netmap            0            0K            0
            mwldev            0            0K            0
        MVS driver            0            0K            0
     CAM ccb queue            0            0K            0
              accf            0            0K            0
               pts            0            0K            0
               iov            0            0K        13586
          ioctlops            0            0K           86
           eventfd            0            0K            0
           Witness            0            0K            0
             stack            0            0K            0
          mrsasbuf            0            0K            0
          mpt_user            0            0K            0
          mps_user            0            0K            0
            MPSSAS            0            0K            0
               mps            0            0K            0
              sbuf            0            0K          288
          mpr_user            0            0K            0
          firmware            0            0K            0
        compressor            0            0K            0
            MPRSAS            0            0K            0
              SWAP            0            0K            0
               mpr            0            0K            0
            mfibuf            0            0K            0
         sysctltmp            0            0K          645
            sysctl            0            0K            3
        md_sectors            0            0K            0
              ekcd            0            0K            0
            dumper            0            0K            0
          sendfile            0            0K            0
              rctl            0            0K            0
           md_disk            0            0K            0
           malodev            0            0K            0
               LED            0            0K            0
          ix_sriov            0            0K            0
             cache            0            0K            0
        aacraidcam            0            0K            0
          kcovinfo            0            0K            0
      prison_racct            0            0K            0
       Fail Points            0            0K            0
             sigio            0            0K            1
filedesc_to_leader            0            0K            0
               pwd            0            0K            0
       tty console            0            0K            0
                ix            0            0K            0
            ipsbuf            0            0K            0
       aacraid_buf            0            0K            0
            aaccam            0            0K            0
         boottrace            0            0K            0
            aacbuf            0            0K            0
              zstd            0            0K            0
            XZ_DEC            0            0K            0
            nvlist            0            0K            0
          SCSI ENC            0            0K            0
           SCSI sa            0            0K            0
        isofs_node            0            0K            0
       isofs_mount            0            0K            0
     tr_raid5_data            0            0K            0
    tr_raid1e_data            0            0K            0
     tr_raid1_data            0            0K            0
     tr_raid0_data            0            0K            0
    tr_concat_data            0            0K            0
       md_sii_data            0            0K            0
   md_promise_data            0            0K            0
    md_nvidia_data            0            0K            0
   md_jmicron_data            0            0K            0
db> show uma
              Zone   Size    Used    Free    Requests  Sleeps  Bucket  Total Mem    XFree
   mbuf_jumbo_page   4096    8320    1078       17484       0     254   38494208        0
              mbuf    256    8583    1079       32933       0     254    2473472        0
          BUF TRIE    144     196   11620         642       0      62    1701504        0
        malloc-384    384    4170      30        4171       0      30    1612800        0
        malloc-128    128   11436     158       15708       0     126    1484032        0
        malloc-384    384    3733      47        4261       0      30    1451520        0
       malloc-4096   4096     327       1         489       0       2    1343488        0
       UMA Slabs 0    112   10638      12       10638       0     126    1192800        0
      mbuf_cluster   2048     508       0         508       0     254    1040384        0
        malloc-256    256    3768      27       12973       0      62     971520        0
         vmem btag     56   16254      57       16254       0     254     913416        0
        malloc-128    128    4899      30       39202       0     126     630912        0
         FFS inode   1160     494      31        4676       0       8     609000        0
              pbuf   2624       0     198           0       0       2     519552        0
        RADIX NODE    144    3380     227      119731       0      62     519408        0
            socket    960      26     482        9628       0     254     487680        0
       malloc-2048   2048       7     193        1076       0       8     409600        0
          lkpicurr    168       2    2350           2       0      62     395136        0
      malloc-65536  65536       6       0           6       0       1     393216        0
            lkpimm    168       1    2327           1       0      62     391104        0
        256 Bucket   2048     125      19        1052       0       8     294912        0
            THREAD   1824     136      18        5096       0       8     280896        0
         malloc-64     64    3888     459       13733       0     254     278208        0
             VNODE    448     524      79        4708       0      30     270144        0
         VM OBJECT    264     956      64       64927       0      30     269280        0
        malloc-256    256     358     572        5365       0      62     238080        0
       malloc-4096   4096      53       5        4950       0       2     237568        0
         malloc-16     16   14387     113       22764       0     254     232000        0
            DEVCTL   1024       0     220         123       0       0     225280        0
         UMA Zones    768     239       0         239       0      16     183552        0
         malloc-32     32    5280     390        5307       0     254     181440        0
       malloc-1024   1024     131      29         140       0      16     163840        0
     FPU_save_area    832     138      42        5116       0      16     149760        0
        malloc-128    128    1029     118        5413       0     126     146816        0
       S VFS Cache    104     981     423        5177       0     126     146016        0
       FFS2 dinode    256     494      76        4675       0      62     145920        0
        malloc-256    256      11     559         535       0      62     145920        0
      malloc-65536  65536       0       2          54       0       1     131072        0
             unpcb    256       7     503        1154       0     254     130560        0
       mbuf_packet    256       0     508          98       0     254     130048        0
         MAP ENTRY     96     933     327      112857       0     126     120960        0
          ksiginfo    112      55     989          75       0     126     116928        0
      malloc-32768  32768       3       0           3       0       1      98304        0
              PROC   1376      44      22        4939       0       8      90816        0
          UMA Kegs    384     226       7         226       0      30      89472        0
        128 Bucket   1024      47      36         336       0      16      84992        0
        malloc-256    256     180     135        9689       0      62      80640        0
         filedesc0   1072      45      25        4940       0       8      75040        0
       malloc-8192   8192       8       1          10       0       1      73728        0
       malloc-4096   4096      18       0          29       0       2      73728        0
             g_bio    408       1     179        5094       0      30      73440        0
         malloc-64     64     515     556        1623       0     254      68544        0
        malloc-128    128     332     195        4640       0     126      67456        0
      malloc-65536  65536       1       0           1       0       1      65536        0
      malloc-65536  65536       0       1         126       0       1      65536        0
      malloc-65536  65536       1       0           1       0       1      65536        0
      malloc-65536  65536       1       0           1       0       1      65536        0
      malloc-32768  32768       2       0           2       0       1      65536        0
       malloc-8192   8192       6       1         107       0       1      57344        0
         64 Bucket    512      86      18        2840       0      30      53248        0
         malloc-64     64     474     345         910       0     254      52416        0
        malloc-256    256      47     148        4427       0      62      49920        0
         32 Bucket    256      55     140        3885       0      62      49920        0
           DIRHASH   1024      35      13          35       0      16      49152        0
             NAMEI   1024       0      48       28748       0      16      49152        0
      malloc-16384  16384       3       0           3       0       1      49152        0
       malloc-1024   1024      10      38        1421       0      16      49152        0
        malloc-384    384      96      24          99       0      30      46080        0
               PWD     40      18    1093        4264       0     254      44440        0
          syncache    168       0     264           5       0     254      44352        0
         tcp_inpcb   1304      11      22        8325       0       8      43032        0
       malloc-8192   8192       4       1          30       0       1      40960        0
           VMSPACE    520      28      47        4924       0      16      39000        0
         udp_inpcb    424       6      84         128       0      30      38160        0
            da_ccb    544       0      70        1407       0      16      38080        0
            pcpu-8      8    4287     321        4315       0     254      36864        0
         malloc-64     64      27     540          27       0     254      36288        0
         malloc-64     64      71     496       78221       0     254      36288        0
         malloc-64     64      59     508        5501       0     254      36288        0
         malloc-64     64     211     356         233       0     254      36288        0
         malloc-64     64     158     409         176       0     254      36288        0
        malloc-128    128       3     276           3       0     126      35712        0
        malloc-128    128      23     256          34       0     126      35712        0
        malloc-128    128      24     255         166       0     126      35712        0
        malloc-128    128     133     146         142       0     126      35712        0
     routing nhops    256      10     125          17       0      62      34560        0
           ttyoutq    256      72      63         160       0      62      34560        0
        malloc-256    256      28     107        8439       0      62      34560        0
        malloc-256    256      29     106         370       0      62      34560        0
        malloc-256    256      52      83        5129       0      62      34560        0
      malloc-32768  32768       0       1         120       0       1      32768        0
      malloc-16384  16384       2       0          12       0       1      32768        0
       malloc-4096   4096       7       1           7       0       2      32768        0
       malloc-2048   2048       4      12           4       0       8      32768        0
       malloc-2048   2048       2      14           2       0       8      32768        0
       malloc-2048   2048       6      10          15       0       8      32768        0
       malloc-2048   2048       7       9           7       0       8      32768        0
       malloc-1024   1024       0      32           4       0      16      32768        0
       malloc-1024   1024      11      21          15       0      16      32768        0
       malloc-1024   1024       4      28           8       0      16      32768        0
       malloc-1024   1024       4      28         151       0      16      32768        0
       malloc-1024   1024      10      22          10       0      16      32768        0
       malloc-1024   1024       1      31          16       0      16      32768        0
        malloc-512    512       2      62          20       0      30      32768        0
        malloc-512    512       4      60           4       0      30      32768        0
        malloc-512    512       0      64         128       0      30      32768        0
        malloc-512    512      11      53          60       0      30      32768        0
           pcpu-64     64     486      26         486       0     254      32768        0
    ertt_txseginfo     40       1     807        4328       0     254      32320        0
            ttyinq    160     135      65         300       0      62      32000        0
              PGRP    120      19     245          32       0     126      31680        0
            clpbuf   2624       0      12          27       0       4      31488        0
            cpuset    104       7     272           7       0     126      29016        0
        sctp_laddr     48       0     588           4       0     254      28224        0
   tcp_inpcb ports     32       9     873        8319       0     254      28224        0
         16 Bucket    144      56     140         302       0      62      28224        0
          4 Bucket     48       6     582           8       0     254      28224        0
         TURNSTILE    136     155      34         155       0      62      25704        0
       malloc-4096   4096       2       4        4906       0       2      24576        0
          rl_entry     40      30     576          30       0     254      24240        0
           rtentry    168      13     131          17       0      62      24192        0
              pipe    728       6      27         284       0      16      24024        0
             Files     80      75     225       19013       0     126      24000        0
          8 Bucket     80      65     235        1944       0     126      24000        0
        malloc-384    384       0      60          19       0      30      23040        0
        SLEEPQUEUE     88     155     101         155       0     126      22528        0
         hostcache     64       1     314           1       0     254      20160        0
   udp_inpcb ports     32       3     627          40       0     254      20160        0
              ertt     72      11     269        8325       0     126      20160        0
         malloc-32     32       6     624          20       0     254      20160        0
         malloc-32     32      25     605          70       0     254      20160        0
         malloc-32     32     368     262        9272       0     254      20160        0
         malloc-32     32      74     556        5125       0     254      20160        0
         malloc-32     32      38     592          81       0     254      20160        0
         malloc-32     32      52     578        3879       0     254      20160        0
         malloc-32     32      54     576        8499       0     254      20160        0
          2 Bucket     32      42     588         326       0     254      20160        0
 epoch_record pcpu    256       4      60           4       0      62      16384        0
      malloc-16384  16384       1       0           1       0       1      16384        0
      malloc-16384  16384       1       0           1       0       1      16384        0
      malloc-16384  16384       0       1         160       0       1      16384        0
      malloc-16384  16384       1       0           1       0       1      16384        0
       malloc-8192   8192       2       0           2       0       1      16384        0
       malloc-8192   8192       2       0           2       0       1      16384        0
       malloc-8192   8192       2       0           2       0       1      16384        0
       malloc-2048   2048       0       8          50       0       8      16384        0
       malloc-2048   2048       2       6         193       0       8      16384        0
       malloc-2048   2048       1       7           1       0       8      16384        0
        malloc-512    512       1      31           2       0      30      16384        0
        malloc-512    512       2      30           2       0      30      16384        0
           SMR CPU     32       7     504           7       0     254      16352        0
      vtnet_tx_hdr     24       0     668        6332       0     254      16032        0
              kenv    258      16      44        1033       0      30      15480        0
            mqnode    416       3      33           3       0      30      14976        0
              vmem   1856       1       7           1       0       8      14848        0
             ripcb    392       1      35           4       0      30      14112        0
        SMR SHARED     24       7     504           7       0     254      12264        0
             KNOTE    160       0      75           8       0      62      12000        0
         malloc-16     16      26     724         113       0     254      12000        0
         malloc-16     16      49     701         288       0     254      12000        0
         malloc-16     16     214     536        4290       0     254      12000        0
         malloc-16     16      28     722         150       0     254      12000        0
         malloc-16     16     278     472       26318       0     254      12000        0
         malloc-16     16      11     739          17       0     254      12000        0
        malloc-384    384       0      30           1       0      30      11520        0
        malloc-384    384       0      30         342       0      30      11520        0
        malloc-384    384       2      28           2       0      30      11520        0
        malloc-384    384       1      29           1       0      30      11520        0
       Mountpoints   2816       2       2           2       0       4      11264        0
       malloc-8192   8192       1       0           1       0       1       8192        0
       malloc-8192   8192       1       0           1       0       1       8192        0
       malloc-4096   4096       0       2           5       0       2       8192        0
         malloc-16     16       0     500           2       0     254       8000        0
           pcpu-16     16       4     252           4       0     254       4096        0
       UMA Slabs 1    176       8      14           8       0      62       3872        0
        KMAP ENTRY     96      12      27          14       0       0       3744        0
       FFS1 dinode    128       0       0           0       0     126          0        0
           ada_ccb    272       0       0           0       0      30          0        0
             swblk    136       0       0           0       0      62          0        0
          swpctrie    144       0       0           0       0      62          0        0
   pf state scrubs     40       0       0           0       0     254          0        0
   pf frag entries     40       0       0           0       0     254          0        0
          pf frags    248       0       0           0       0      62          0        0
  pf table entries    160       0       0           0       0     254          0        0
pf table entry counters     64       0       0           0       0     254          0        0
   pf source nodes    152       0       0           0       0     254          0        0
     pf state keys     88       0       0           0       0     126          0        0
         pf states    344       0       0           0       0     254          0        0
           pf tags    104       0       0           0       0     126          0        0
          pf mtags    184       0       0           0       0      62          0        0
       tcp_bbr_pcb    832       0       0           0       0      16          0        0
       tcp_bbr_map    128       0       0           0       0     126          0        0
      tcp_rack_pcb   1024       0       0           0       0      16          0        0
      tcp_rack_map    128       0       0           0       0     126          0        0
tfo_ccache_entries     80       0       0           0       0     126          0        0
               tfo      4       0       0           0       0     254          0        0
          sackhole     32       0       0           0       0     254          0        0
               ipq     56       0       0           0       0     254          0        0
   tcp_log_id_node    120       0       0           0       0     126          0        0
 tcp_log_id_bucket    176       0       0           0       0      62          0        0
           tcp_log    416       0       0           0       0     254          0        0
          tcpreass     48       0       0           0       0     254          0        0
   sctp_asconf_ack     48       0       0           0       0     254          0        0
       sctp_asconf     40       0       0           0       0     254          0        0
sctp_stream_msg_out    112       0       0           0       0     254          0        0
        sctp_readq    152       0       0           0       0     254          0        0
        sctp_chunk    152       0       0           0       0     254          0        0
        sctp_raddr    736       0       0           0       0     254          0        0
         sctp_asoc   2264       0       0           0       0     254          0        0
           sctp_ep   1176       0       0           0       0     254          0        0
       ripcb ports     32       0       0           0       0     254          0        0
udplite_inpcb ports     32       0       0           0       0     254          0        0
     udplite_inpcb    424       0       0           0       0      30          0        0
    IPsec SA lft_c     16       0       0           0       0     254          0        0
           netlink   2048       0       0           0       0       8          0        0
            itimer    352       0       0           0       0      30          0        0
            AIOLIO    272       0       0           0       0      30          0        0
             AIOCB    552       0       0           0       0      16          0        0
               AIO    208       0       0           0       0      62          0        0
           NCLNODE    608       0       0           0       0      16          0        0
        TMPFS node    232       0       0           0       0      62          0        0
        mqnotifier    216       0       0           0       0      62          0        0
            mvdata     64       0       0           0       0     254          0        0
            mqueue    248       0       0           0       0      62          0        0
     LTS VFS Cache    360       0       0           0       0      30          0        0
       L VFS Cache    320       0       0           0       0      30          0        0
     STS VFS Cache    144       0       0           0       0      62          0        0
           cryptop    280       0       0           0       0      30          0        0
  linux_dma_object     32       0       0           0       0     254          0        0
  linux_dma_pctrie    144       0       0           0       0      62          0        0
   IOMMU_MAP_ENTRY    104       0       0           0       0     126          0        0
    mbuf_jumbo_16k  16384       0       0           0       0     254          0        0
     mbuf_jumbo_9k   9216       0       0           0       0     254          0        0
      audit_record   1280       0       0           0       0       8          0        0
         domainset     40       0       0           0       0     254          0        0
        MAC labels     40       0       0           0       0     254          0        0
            vnpbuf   2624       0       0           0       0      16          0        0
           nfspbuf   2624       0       0           0       0       4          0        0
            swwbuf   2624       0       0           0       0       2          0        0
            swrbuf   2624       0       0           0       0       4          0        0
          umtx_shm     88       0       0           0       0     126          0        0
           umtx pi     96       0       0           0       0     126          0        0
rangeset pctrie nodes    144       0       0           0       0      62          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-65536  65536       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-32768  32768       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
      malloc-16384  16384       0       0           0       0       1          0        0
       malloc-4096   4096       0       0           0       0       2          0        0
       malloc-4096   4096       0       0           0       0       2          0        0
        malloc-512    512       0       0           0       0      30          0        0
        malloc-512    512       0       0           0       0      30          0        0
           pcpu-32     32       0       0           0       0     254          0        0
            pcpu-4      4       0       0           0       0     254          0        0
            fakepg    104       0       0           0       0     126          0        0
          UMA Hash    256       0       0           0       0      62          0        0

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/07/10 22:51 freebsd-src 8ab2da68283b ee50e71c console log report syz C ci-freebsd-main Fatal trap NUM: page fault in tcp_input_with_port
2023/07/01 09:18 freebsd-src 0631830a7a3c ee50e71c console log report syz ci-freebsd-main Fatal trap NUM: page fault in tcp_input_with_port
2023/07/10 21:10 freebsd-src 8ab2da68283b ee50e71c console log report ci-freebsd-main Fatal trap NUM: page fault in tcp_input_with_port
2023/07/01 08:17 freebsd-src 0631830a7a3c ee50e71c console log report ci-freebsd-main Fatal trap NUM: page fault in tcp_input_with_port
2023/06/01 04:09 freebsd-src 4f2cc73f34eb ee50e71c console log report ci-freebsd-main Fatal trap NUM: page fault in tcp_input_with_port
2023/06/15 18:30 freebsd-src 2b5dd8b89011 ee50e71c console log report ci-freebsd-i386 Fatal trap NUM: page fault in tcp_input_with_port
* Struck through repros no longer work on HEAD.