Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0xffffffff82c70d52 stack pointer = 0x28:0xfffffe0003640840 frame pointer = 0x28:0xfffffe0003640880 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock (0)) trap number = 9 panic: general protection fault cpuid = 0 time = 1604351138 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x47/frame 0xfffffe0003640510 vpanic() at vpanic+0x1c7/frame 0xfffffe0003640570 panic() at panic+0x43/frame 0xfffffe00036405d0 trap_fatal() at trap_fatal+0x4cd/frame 0xfffffe0003640650 trap() at trap+0xf7/frame 0xfffffe0003640770 calltrap() at calltrap+0x8/frame 0xfffffe0003640770 --- trap 0x9, rip = 0xffffffff82c70d52, rsp = 0xfffffe0003640840, rbp = 0xfffffe0003640880 --- sctp_timer_stop() at sctp_timer_stop+0x3a2/frame 0xfffffe0003640880 sctp_stop_association_timers() at sctp_stop_association_timers+0xf9/frame 0xfffffe00036408b0 sctp_free_assoc() at sctp_free_assoc+0x2b6/frame 0xfffffe0003640940 sctp_timeout_handler() at sctp_timeout_handler+0x122a/frame 0xfffffe0003640a00 softclock_call_cc() at softclock_call_cc+0x1f7/frame 0xfffffe0003640ad0 softclock() at softclock+0xb0/frame 0xfffffe0003640b10 ithread_loop() at ithread_loop+0x33f/frame 0xfffffe0003640bb0 fork_exit() at fork_exit+0xb3/frame 0xfffffe0003640bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0003640bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 12 tid 100022 ] Stopped at kdb_enter+0x67: movq $0,0x14760c6(%rip) db> db> set $lines = 0 db> set $maxwidth = 0 db> show registers cs 0x20 ds 0x3b es 0x3b fs 0x13 gs 0x1b ss 0x28 rax 0x12 rcx 0xffffffff810e9c30 vprintf+0x140 rdx 0x1 rbx 0 rsp 0xfffffe00036404f0 rbp 0xfffffe0003640510 rsi 0 rdi 0xffffffff810e9c66 vprintf+0x176 r8 0 r9 0xffffffff r10 0 r11 0x234e ll+0x232d r12 0xffffffff82066b10 ddb_dbbe r13 0 r14 0xffffffff8196ce02 r15 0xffffffff8196ce02 rip 0xffffffff810de1b7 kdb_enter+0x67 rflags 0x86 kdb_enter+0x67: movq $0,0x14760c6(%rip) db> show proc Process 12 (intr) at 0xfffff800042a4a40: state: NORMAL uid: 0 gids: 0 parent: pid 0 at 0xffffffff82506ab0 ABI: null reaper: 0xffffffff82506ab0 reapsubtree: 12 sigparent: 20 vmspace: 0xffffffff82507700 (map 0xffffffff82507700) (map.pmap 0xffffffff825077c0) (pmap 0xffffffff82507820) threads: 23 100012 I [swi5: fast taskq] 100016 I [swi6: task queue] 100017 I [swi6: Giant taskq] 100020 I [swi1: netisr 0] 100021 I [swi3: vm] 100022 Run CPU 0 [swi4: clock (0)] 100023 I [swi4: clock (1)] 100035 I [irq24: virtio_pci0] 100036 I [irq25: virtio_pci0] 100037 I [irq26: virtio_pci0] 100038 I [irq27: virtio_pci0] 100039 I [irq28: virtio_pci1] 100040 I [irq29: virtio_pci1] 100041 RunQ [irq30: virtio_pci1] 100042 I [irq31: virtio_pci1] 100043 I [irq32: virtio_pci1] 100048 I [irq10: virtio_pci2] 100050 I [irq1: atkbd0] 100051 I [irq12: psm0] 100052 I [swi0: uart uart++] 100060 I [swi1: pf send] 100073 I [swi1: hpts] 100074 I [swi1: hpts] db> ps pid ppid pgrp uid state wmesg wchan cmd 26088 25990 25990 0 R (threaded) syz-executor.2 100371 RunQ syz-executor.2 100133 S sbwait 0xfffff800465521d4 syz-executor.2 100693 S sbwait 0xfffff80046b7c1d4 syz-executor.2 100703 S uwait 0xfffff80046576c00 syz-executor.2 100708 S uwait 0xfffff800465eb700 syz-executor.2 101030 S uwait 0xfffff800465eb600 syz-executor.2 26086 25975 25975 0 R (threaded) syz-executor.0 101129 RunQ syz-executor.0 100687 S sbwait 0xfffff80030b5f1d4 syz-executor.0 100616 S sbwait 0xfffff80037cf6594 syz-executor.0 100699 S accept 0xfffff80018f87168 syz-executor.0 100706 S uwait 0xfffff8004667be80 syz-executor.0 100726 S uwait 0xfffff8000495e100 syz-executor.0 26083 25915 25915 0 R (threaded) syz-executor.1 101089 Run CPU 1 syz-executor.1 100689 S connec 0xfffff80030d34870 syz-executor.1 100621 S connec 0xfffff80030da0c30 syz-executor.1 101034 RunQ syz-executor.1 101035 RunQ syz-executor.1 101036 RunQ syz-executor.1 25990 762 25990 0 Rs syz-executor.2 25975 762 25975 0 Rs syz-executor.0 25971 25961 25971 0 Ss select 0xfffff800469848c0 dhclient 25968 1 25968 0 Ss select 0xfffff800465c7240 dhclient 25961 25949 424 65 S select 0xfffff800464f7c40 dhclient 25949 424 424 0 S wait 0xfffff80046653000 sh 25915 762 25915 0 Rs syz-executor.1 21712 1 21712 65 Ss select 0xfffff80046984340 dhclient 21226 1 21226 0 Ss select 0xfffff800464f7b40 dhclient 21223 1 21223 0 Ss select 0xfffff80004bf36c0 dhclient 20729 762 20729 0 Ss piperd 0xfffff800463e82f8 syz-executor.3 762 760 760 0 S (threaded) syz-fuzzer 100099 S uwait 0xfffff8000495e880 syz-fuzzer 100105 S uwait 0xfffff80004b8e680 syz-fuzzer 100106 S uwait 0xfffff800463dab80 syz-fuzzer 100107 S uwait 0xfffff800463dac80 syz-fuzzer 100108 S uwait 0xfffff80004b8ec80 syz-fuzzer 100109 S uwait 0xfffff80004b8ed80 syz-fuzzer 100110 S kqread 0xfffff80004becb00 syz-fuzzer 100111 S uwait 0xfffff800463daa80 syz-fuzzer 100613 S uwait 0xfffff8004667b680 syz-fuzzer 760 758 760 0 Ss pause 0xfffff800460225c8 csh 758 682 758 0 Ss select 0xfffff80004bf3940 sshd 742 1 742 0 Ss+ ttyin 0xfffff80004706cb0 getty 741 1 741 0 Ss+ ttyin 0xfffff800049fc0b0 getty 740 1 740 0 Ss+ ttyin 0xfffff800049fc4b0 getty 739 1 739 0 Ss+ ttyin 0xfffff800049fc8b0 getty 738 1 738 0 Ss+ ttyin 0xfffff800049fccb0 getty 737 1 737 0 Ss+ ttyin 0xfffff800049f40b0 getty 736 1 736 0 Ss+ ttyin 0xfffff800049f44b0 getty 735 1 735 0 Ss+ ttyin 0xfffff800049f48b0 getty 734 1 734 0 Ss+ ttyin 0xfffff800049f4cb0 getty 686 1 686 0 Ss nanslp 0xffffffff825279b1 cron 682 1 682 0 Ss select 0xfffff80004bf3dc0 sshd 495 1 495 0 Ss select 0xfffff80004936840 syslogd 424 1 424 0 Ss wait 0xfffff80004b89000 devd 423 1 423 65 Ss select 0xfffff800049366c0 dhclient 338 1 338 0 Ss select 0xfffff80004bf3ec0 dhclient 335 1 335 0 Ss select 0xfffff800049367c0 dhclient 23 0 0 0 DL syncer 0xffffffff82614d78 [syncer] 22 0 0 0 DL vlruwt 0xfffff80004a52a40 [vnlru] 21 0 0 0 DL (threaded) [bufdaemon] 100070 D qsleep 0xffffffff82613e50 [bufdaemon] 100077 D - 0xffffffff8200ac00 [bufspacedaemon-0] 100088 D sdflush 0xfffff800044be0e8 [/ worker] 20 0 0 0 DL psleep 0xffffffff8263b308 [vmdaemon] 19 0 0 0 DL (threaded) [pagedaemon] 100068 D psleep 0xffffffff8262f778 [dom0] 100075 D launds 0xffffffff8262f784 [laundry: dom0] 100076 D umarcl 0xffffffff814e58d0 [uma] 18 0 0 0 DL - 0xffffffff8235b108 [rand_harvestq] 17 0 0 0 DL waiting 0xffffffff82c951d8 [sctp_iterator] 16 0 0 0 DL pftm 0xffffffff82e47390 [pf purge] 15 0 0 0 DL - 0xffffffff8261342c [soaiod4] 9 0 0 0 DL - 0xffffffff8261342c [soaiod3] 8 0 0 0 DL - 0xffffffff8261342c [soaiod2] 7 0 0 0 DL - 0xffffffff8261342c [soaiod1] 6 0 0 0 DL (threaded) [cam] 100034 D - 0xffffffff82232e40 [doneq0] 100067 D - 0xffffffff82232d10 [scanner] 5 0 0 0 DL crypto_ 0xfffff800042b6b90 [crypto returns 1] 4 0 0 0 DL crypto_ 0xfffff800042b6b30 [crypto returns 0] 3 0 0 0 DL crypto_ 0xffffffff8262cc70 [crypto] 14 0 0 0 DL seqstat 0xfffff800042ea488 [sequencer 00] 13 0 0 0 DL (threaded) [geom] 100025 D - 0xffffffff82506520 [g_event] 100026 D - 0xffffffff82506528 [g_up] 100027 D - 0xffffffff82506530 [g_down] 2 0 0 0 DL (threaded) [KTLS] 100018 D - 0xfffff800042a6b00 [thr_0] 100019 D - 0xfffff800042a6b80 [thr_1] 12 0 0 0 RL (threaded) [intr] 100012 I [swi5: fast taskq] 100016 I [swi6: task queue] 100017 I [swi6: Giant taskq] 100020 I [swi1: netisr 0] 100021 I [swi3: vm] 100022 Run CPU 0 [swi4: clock (0)] 100023 I [swi4: clock (1)] 100035 I [irq24: virtio_pci0] 100036 I [irq25: virtio_pci0] 100037 I [irq26: virtio_pci0] 100038 I [irq27: virtio_pci0] 100039 I [irq28: virtio_pci1] 100040 I [irq29: virtio_pci1] 100041 RunQ [irq30: virtio_pci1] 100042 I [irq31: virtio_pci1] 100043 I [irq32: virtio_pci1] 100048 I [irq10: virtio_pci2] 100050 I [irq1: atkbd0] 100051 I [irq12: psm0] 100052 I [swi0: uart uart++] 100060 I [swi1: pf send] 100073 I [swi1: hpts] 100074 I [swi1: hpts] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 1 0 1 0 SLs wait 0xfffff80004287520 [init] 10 0 0 0 DL audit_w 0xffffffff8262d190 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff82506ab0 [swapper] 100005 D - 0xfffff80004185700 [if_io_tqg_0] 100006 D - 0xfffff80004185500 [if_io_tqg_1] 100007 D - 0xfffff80004185300 [if_config_tqg_0] 100008 D - 0xfffff80004185100 [softirq_0] 100009 D - 0xfffff800042a3e00 [softirq_1] 100010 D - 0xfffff800042a3c00 [inm_free taskq] 100011 D - 0xfffff800042a3a00 [thread taskq] 100013 D - 0xfffff800042a3600 [kqueue_ctx taskq] 100014 D - 0xfffff800042a3400 [aiod_kick taskq] 100015 D - 0xfffff800042a3200 [in6m_free taskq] 100024 D - 0xfffff800042a6400 [firmware taskq] 100029 D - 0xfffff800042ecc00 [crypto_0] 100030 D - 0xfffff800042ecc00 [crypto_1] 100044 D - 0xfffff800044ca200 [vtnet0 rxq 0] 100045 D - 0xfffff800044ca100 [vtnet0 txq 0] 100046 D - 0xfffff800044ca000 [vtnet0 rxq 1] 100047 D - 0xfffff800046f2e00 [vtnet0 txq 1] 100049 D vtbslp 0xfffff800046f1400 [virtio_balloon] 100053 D - 0xfffff800046fc200 [mca taskq] 100056 D - 0xffffffff81d14b01 [deadlkres] 100063 D - 0xfffff80004923300 [acpi_task_0] 100064 D - 0xfffff80004923300 [acpi_task_1] 100065 D - 0xfffff80004923300 [acpi_task_2] 100066 D - 0xfffff800042ec400 [CAM taskq] db> show all locks Process 26083 (syz-executor.1) thread 0xfffffe002586b100 (101034) exclusive rw tcpinp (tcpinp) r = 0 (0xfffff80046c587c0) locked @ /syzkaller/managers/main/kernel/sys/netinet/tcp_usrreq.c:984 exclusive sx so_snd_sx (so_snd_sx) r = 0 (0xfffff80030da0dd0) locked @ /syzkaller/managers/main/kernel/sys/kern/uipc_sockbuf.c:467 Process 12 (intr) thread 0xfffffe00048b1c00 (100022) exclusive sleep mutex sctp-send-tcb (tcbs) r = 0 (0xfffffe0025a9d3a0) locked @ /syzkaller/managers/main/kernel/sys/netinet/sctp_pcb.c:4694 exclusive sleep mutex sctp-tcb (tcb) r = 0 (0xfffffe0025a9d380) locked @ /syzkaller/managers/main/kernel/sys/netinet/sctputil.c:1762 db> show malloc Type InUse MemUse Requests sctp_stro 119 23208K 9162 pf_hash 5 11524K 5 devbuf 4214 4854K 4242 tcp_hpts 5 3201K 5 vtbuf 24 1968K 46 sysctloid 28692 1673K 28756 kobj 336 1344K 496 newblk 12 1027K 60789 vfscache 3 1025K 3 pcb 267 807K 39985 inodedep 23 523K 29609 ufs_quota 1 512K 1 vfs_hash 1 512K 1 callout 2 512K 2 intr 4 472K 4 subproc 139 269K 26171 acpica 1674 185K 52444 vnet_data 1 168K 1 filedesc 20 137K 48841 pagedep 9 130K 25035 tfo_ccache 1 128K 1 sctp_atcl 240 120K 28280 sem 4 106K 4 DEVFS1 105 105K 122 linker 254 99K 438 bus 979 79K 3032 mtx_pool 2 72K 2 syncache 1 68K 1 acpitask 1 64K 1 ddb_capture 1 64K 1 module 507 64K 507 ifaddr 135 56K 245 BPF 30 53K 254 umtx 378 48K 378 kdtrace 192 38K 131024 lockf 316 36K 24124 temp 35 33K 9574 hostcache 1 32K 1 shm 1 32K 21 DEVFS3 124 31K 134 msg 4 30K 4 DEVFS_RULE 56 27K 56 vmem 3 26K 6 gtaskqueue 18 26K 18 kbdmux 6 22K 6 lltable 48 18K 992 ufs_mount 5 17K 6 proc 3 17K 3 tty 16 16K 16 tidhash 1 16K 1 ithread 99 16K 99 sctp_atky 369 16K 38990 ether_multi 172 14K 1510 bus-sc 30 14K 1413 KTRACE 100 13K 100 ifnet 7 13K 7 select 94 12K 94 kenv 92 12K 92 eventhandler 132 12K 132 sctp_timw 42 11K 42 in6_multi 89 11K 754 pfs_nodes 20 10K 20 GEOM 60 10K 489 rman 82 10K 423 bmsafemap 3 9K 29717 UART 12 9K 12 devstat 4 9K 4 rpc 2 8K 2 shmfd 1 8K 13 pfs_vncache 1 8K 1 audit_evclass 233 8K 291 sctp_ifa 53 7K 161 CAM DEV 3 6K 510 pf_osfp 54 6K 54 kqueue 59 6K 26095 cred 23 6K 940 pf_rule 10 6K 816 pf_ifnet 15 6K 736 plimit 22 6K 2110 vt 11 6K 11 ufs_dirhash 27 6K 27 sglist 5 6K 5 CAM queue 5 6K 1528 taskqueue 48 6K 48 DEVFSP 74 5K 998 UMA 249 5K 249 memdesc 1 4K 1 MCA 32 4K 32 kcovinfo 64 4K 884 evdev 4 4K 4 sctp_athm 240 4K 29351 sctp_map 238 4K 18210 routetbl 20 4K 1552 session 28 4K 177 pgrp 28 4K 179 dirrem 13 4K 27258 hhook 13 4K 13 terminal 11 3K 11 acpisem 22 3K 22 proc-args 47 3K 2037 uidinfo 4 3K 128 local_apic 1 2K 1 io_apic 1 2K 1 ipsec-saq 2 2K 2 ip6ndp 12 2K 115 Unitno 30 2K 155536 CAM XPT 22 2K 543 freefile 12 2K 27064 in_multi 6 2K 106 freework 6 2K 39545 freeblks 5 2K 26653 ipsecpolicy 2 2K 2 acpidev 20 2K 20 msi 9 2K 9 clone 9 2K 9 tun 7 2K 7 softdep 1 1K 1 mkdir 8 1K 48676 sahead 1 1K 1 secasvar 1 1K 1 nhops 6 1K 26 vnodemarker 2 1K 2748 NFSD session 1 1K 1 CAM periph 4 1K 271 ipsec 3 1K 3 sctp_ifn 6 1K 161 mld 6 1K 6 ip6opt 4 1K 249 igmp 6 1K 6 toponodes 6 1K 6 isadev 6 1K 6 mount 16 1K 89 pci_link 10 1K 10 inpcbpolicy 20 1K 63552 encap_export_host 12 1K 12 crypto 3 1K 3 sctp_stri 1 1K 2327 newdirblk 4 1K 24338 diradd 4 1K 27496 indirdep 2 1K 18405 pfil 4 1K 4 chacha20random 1 1K 1 CAM SIM 2 1K 2 cdev 2 1K 2 sctp_aadr 7 1K 1296 ip6_msource 5 1K 81 osd 3 1K 9 vnodes 1 1K 296 ktls 1 1K 1 NFSD lckfile 1 1K 1 NFSD V4client 1 1K 1 DEVFS 9 1K 10 feeder 7 1K 7 iov 4 1K 74632 loginclass 3 1K 6 prison 6 1K 6 soname 7 1K 22990 CAM dev queue 2 1K 2 CAM I/O Scheduler 1 1K 1 apmdev 1 1K 1 atkbddev 2 1K 2 CAM path 4 1K 1034 tcpfunc 2 1K 2 ip_msource 2 1K 64 pmchooks 1 1K 1 nexusdev 5 1K 5 sctp_vrf 1 1K 1 entropy 2 1K 130 vnet 1 1K 1 pmc 1 1K 1 acpiintr 1 1K 1 filecaps 3 1K 319 cpus 2 1K 2 vnet_data_free 1 1K 1 Per-cpu 1 1K 1 p1003.1b 1 1K 1 pf_table 0 0K 370 pf_altq 0 0K 0 pf_temp 0 0K 0 sctp_mcore 0 0K 0 sctp_socko 0 0K 12829 sctp_iter 0 0K 216 sctp_mvrf 0 0K 0 sctp_cpal 0 0K 46 sctp_cmsg 0 0K 0 sctp_stre 0 0K 0 sctp_athi 0 0K 0 sctp_a_it 0 0K 174 nvlist 0 0K 0 SCSI ENC 0 0K 0 SCSI sa 0 0K 0 madt_table 0 0K 2 scsi_pass 0 0K 0 ciss_data 0 0K 0 smartpqi 0 0K 0 pvscsi 0 0K 0 ath_hal 0 0K 0 athdev 0 0K 0 ata_pci 0 0K 0 ata_dma 0 0K 0 ata_generic 0 0K 0 amr 0 0K 0 scsi_da 0 0K 69 iavf 0 0K 0 ixl 0 0K 0 ata_da 0 0K 0 scsi_ch 0 0K 0 scsi_cd 0 0K 0 USBdev 0 0K 0 USB 0 0K 0 AHCI driver 0 0K 0 ice-resmgr 0 0K 0 ice-osdep 0 0K 0 ice 0 0K 0 axgbe 0 0K 0 fpukern_ctx 0 0K 0 agp 0 0K 0 xen_intr 0 0K 0 xen_hvm 0 0K 0 legacydrv 0 0K 0 qpidrv 0 0K 0 nvme_da 0 0K 0 dmar_idpgtbl 0 0K 0 dmar_dom 0 0K 0 dmar_ctx 0 0K 0 acpipwr 0 0K 0 twsbuf 0 0K 0 twe_commands 0 0K 0 isci 0 0K 0 iommu_dmamap 0 0K 0 twa_commands 0 0K 0 hyperv_socket 0 0K 0 bxe_ilt 0 0K 0 xenbus 0 0K 0 tcp_log_dev 0 0K 113 midi buffers 0 0K 0 mixer 0 0K 0 vm_fictitious 0 0K 0 ac97 0 0K 0 hdacc 0 0K 0 hdac 0 0K 0 hdaa 0 0K 0 acpi_perf 0 0K 0 acpicmbat 0 0K 0 SIIS driver 0 0K 0 CAM CCB 0 0K 179175 UMAHash 0 0K 0 vm_pgdata 0 0K 0 jblocks 0 0K 0 savedino 0 0K 13367 sentinel 0 0K 0 jfsync 0 0K 0 jtrunc 0 0K 0 sbdep 0 0K 1370 jsegdep 0 0K 0 jseg 0 0K 0 jfreefrag 0 0K 0 jfreeblk 0 0K 0 jnewblk 0 0K 0 jmvref 0 0K 0 jremref 0 0K 0 jaddref 0 0K 0 freedep 0 0K 0 freefrag 0 0K 118 allocindir 0 0K 0 allocdirect 0 0K 0 ufs_trim 0 0K 0 mactemp 0 0K 0 audit_trigger 0 0K 0 audit_pipe_presel 0 0K 0 audit_pipeent 0 0K 0 audit_pipe 0 0K 0 audit_evname 0 0K 0 audit_bsm 0 0K 0 audit_gidset 0 0K 0 audit_text 0 0K 0 audit_path 0 0K 0 audit_data 0 0K 0 audit_cred 0 0K 0 xform 0 0K 0 NLM 0 0K 0 ipsec-spdcache 0 0K 0 ipsec-reg 0 0K 0 ipsec-misc 0 0K 0 ipsecrequest 0 0K 0 ip6_moptions 0 0K 132 in6_mfilter 0 0K 324 frag6 0 0K 0 tcplog 0 0K 0 PUC 0 0K 0 LRO 0 0K 0 newreno data 0 0K 0 ip_moptions 0 0K 118 in_mfilter 0 0K 228 ipid 0 0K 0 80211scan 0 0K 0 80211ratectl 0 0K 0 80211power 0 0K 0 80211nodeie 0 0K 0 80211node 0 0K 0 80211mesh_gt 0 0K 0 80211mesh_rt 0 0K 0 80211perr 0 0K 0 80211prep 0 0K 0 80211preq 0 0K 0 80211dfs 0 0K 0 80211crypto 0 0K 0 80211vap 0 0K 0 iflib 0 0K 0 vlan 0 0K 0 gif 0 0K 0 ifdescr 0 0K 0 zlib 0 0K 0 fadvise 0 0K 20 VN POLL 0 0K 64 ppbusdev 0 0K 0 statfs 0 0K 24469 export_host 0 0K 0 cl_savebuf 0 0K 36 agtiapi_MemAlloc malloc 0 0K 0 osti_cacheable 0 0K 0 tempbuff 0 0K 0 tempbuff 0 0K 0 ag_tgt_map_t malloc 0 0K 0 ag_slr_map_t malloc 0 0K 0 lDevFlags * malloc 0 0K 0 tiDeviceHandle_t * malloc 0 0K 0 ag_portal_data_t malloc 0 0K 0 ag_device_t malloc 0 0K 0 STLock malloc 0 0K 0 CCB List 0 0K 0 sr_iov 0 0K 0 OCS 0 0K 0 OCS 0 0K 0 nvme 0 0K 0 nvd 0 0K 0 netmap 0 0K 0 mwldev 0 0K 0 MVS driver 0 0K 0 biobuf 0 0K 0 aios 0 0K 0 lio 0 0K 0 acl 0 0K 0 mbuf_tag 0 0K 2145 accf 0 0K 0 pts 0 0K 0 ioctlops 0 0K 2946 Witness 0 0K 0 stack 0 0K 0 CAM ccb queue 0 0K 0 mrsasbuf 0 0K 0 mpt_user 0 0K 0 mps_user 0 0K 0 MPSSAS 0 0K 0 sbuf 0 0K 288 mps 0 0K 0 firmware 0 0K 0 compressor 0 0K 0 mpr_user 0 0K 0 SWAP 0 0K 0 MPRSAS 0 0K 0 mpr 0 0K 0 sysctltmp 0 0K 1796 sysctl 0 0K 1 mfibuf 0 0K 0 ekcd 0 0K 0 dumper 0 0K 0 sendfile 0 0K 72 rctl 0 0K 0 md_sectors 0 0K 0 md_disk 0 0K 0 malodev 0 0K 0 LED 0 0K 0 cache 0 0K 0 ix_sriov 0 0K 0 prison_racct 0 0K 0 Fail Points 0 0K 0 sigio 0 0K 6 filedesc_to_leader 0 0K 0 pwd 0 0K 0 tty console 0 0K 0 aacraidcam 0 0K 0 ix 0 0K 0 ipsbuf 0 0K 0 iirbuf 0 0K 0 aacraid_buf 0 0K 0 aaccam 0 0K 0 isofs_node 0 0K 0 isofs_mount 0 0K 0 tr_raid5_data 0 0K 0 tr_raid1e_data 0 0K 0 tr_raid1_data 0 0K 0 tr_raid0_data 0 0K 0 tr_concat_data 0 0K 0 md_sii_data 0 0K 0 md_promise_data 0 0K 0 md_nvidia_data 0 0K 0 md_jmicron_data 0 0K 0 md_intel_data 0 0K 0 md_ddf_data 0 0K 0 raid_data 0 0K 72 geom_flashmap 0 0K 0 tmpfs dir 0 0K 0 tmpfs name 0 0K 0 tmpfs mount 0 0K 0 NFS FHA 0 0K 0 newnfsmnt 0 0K 0 newnfsclient_req 0 0K 0 NFSCL layrecall 0 0K 0 NFSCL session 0 0K 0 NFSCL sockreq 0 0K 0 NFSCL devinfo 0 0K 0 NFSCL flayout 0 0K 0 NFSCL layout 0 0K 0 NFSD rollback 0 0K 0 NFSCL diroffdiroff 0 0K 0 NEWdirectio 0 0K 0 NEWNFSnode 0 0K 0 NFSCL lck 0 0K 0 NFSCL lckown 0 0K 0 NFSCL client 0 0K 0 NFSCL deleg 0 0K 0 NFSCL open 0 0K 0 NFSCL owner 0 0K 0 NFS fh 0 0K 0 NFS req 0 0K 0 NFSD usrgroup 0 0K 0 NFSD string 0 0K 0 NFSD V4lock 0 0K 0 NFSD V4state 0 0K 0 NFSD srvcache 0 0K 0 msdosfs_fat 0 0K 0 msdosfs_mount 0 0K 0 msdosfs_node 0 0K 0 DEVFS4 0 0K 0 DEVFS2 0 0K 0 gntdev 0 0K 0 privcmd_dev 0 0K 0 evtchn_dev 0 0K 0 xenstore 0 0K 0 aacbuf 0 0K 0 xnb 0 0K 0 xbbd 0 0K 0 xbd 0 0K 0 Balloon 0 0K 0 sysmouse 0 0K 0 vtfont 0 0K 0 zstd 0 0K 0 db> show uma Zone Size Used Free Requests Sleeps Bucket Total Mem XFree mbuf_jumbo_page 4096 6372 6372 47877 0 254 52199424 0 mbuf_cluster 2048 9173 3 9178 0 254 18792448 0 tcp_log 416 0 15093 290248 0 254 6278688 0 RADIX NODE 144 13916 13101 1244310 0 63 3890448 0 mbuf 256 7029 6606 1932915 0 254 3490560 0 mbuf_packet 256 8252 793 294564 0 254 2315520 0 malloc-512 512 4158 66 7693 0 30 2162688 0 UMA Slabs 0 112 19243 14 19243 0 126 2156784 0 BUF TRIE 144 492 12976 135353 0 62 1939392 0 malloc-4096 4096 395 14 27039 0 2 1675264 0 sctp_asoc 2288 119 456 9105 0 254 1315600 0 malloc-128 128 9710 55 10327 0 126 1249920 0 sctp_ep 1280 120 519 17653 0 254 817920 0 256 Bucket 2048 277 63 24833 0 8 696320 0 pbuf 832 0 781 0 0 2 649792 0 tcp_bbr_map 128 1733 3320 429902 0 126 646784 0 socket 960 39 521 89320 0 254 537600 0 VNODE 496 859 69 28286 0 30 460288 0 sctp_raddr 736 126 490 12756 0 254 453376 0 VM OBJECT 264 1529 166 433055 0 30 447480 0 malloc-2048 2048 121 89 17654 0 8 430080 0 malloc-65536 65536 2 4 1686 0 1 393216 0 THREAD 1792 159 30 78927 0 8 338688 0 malloc-256 256 40 1145 102532 0 62 303360 0 malloc-65536 65536 4 0 4 0 1 262144 0 malloc-2048 2048 3 125 179973 0 8 262144 0 udplite_inpcb 488 8 512 56134 0 254 253760 0 malloc-16384 16384 10 5 24400 0 1 245760 0 malloc-32768 32768 0 7 332 0 1 229376 0 malloc-32768 32768 0 7 16366 0 1 229376 0 malloc-512 512 266 182 31690 0 30 229376 0 g_bio 408 0 560 714358 0 31 228480 0 malloc-1024 1024 114 106 13353 0 16 225280 0 FFS2 dinode 256 822 18 28247 0 62 215040 0 malloc-64 64 3215 124 29673 0 254 213696 0 DEVCTL 1024 5 203 221 0 0 212992 0 MAP ENTRY 96 1456 602 1681547 0 126 197568 0 malloc-65536 65536 0 3 310 0 1 196608 0 malloc-16 16 12213 37 13510 0 254 196000 0 malloc-128 128 1208 249 26118 0 126 186496 0 malloc-4096 4096 31 14 228 0 2 184320 0 UMA Zones 768 224 3 224 0 16 174336 0 sctp_chunk 152 113 979 35796 0 254 165984 0 malloc-16384 16384 4 6 125 0 1 163840 0 malloc-32 32 4788 252 5525 0 254 161280 0 FFS inode 160 822 153 28247 0 62 156000 0 malloc-512 512 25 271 30859 0 30 151552 0 128 Bucket 1024 99 48 13735 0 16 150528 0 S VFS Cache 104 1150 293 29496 0 126 150072 0 malloc-1024 1024 122 18 260 0 16 143360 0 malloc-65536 65536 0 2 118 0 1 131072 0 malloc-65536 65536 2 0 2 0 1 131072 0 VMSPACE 2536 34 17 26067 0 4 129336 0 vmem btag 56 2104 44 2268 0 254 120288 0 ksiginfo 112 66 978 1492 0 126 116928 0 malloc-256 256 140 310 73603 0 62 115200 0 malloc-128 128 730 107 3519 0 126 107136 0 PROC 1312 56 25 26088 0 8 106272 0 tcpcb 1048 7 92 5070 0 254 103752 0 malloc-256 256 260 145 137682 0 62 103680 0 vnpbuf 832 0 124 7 0 62 103168 0 64 Bucket 512 108 76 15120 0 30 94208 0 clpbuf 832 0 112 72826 0 16 93184 0 filedesc0 1080 57 27 26089 0 8 90720 0 malloc-256 256 200 145 9854 0 62 88320 0 malloc-256 256 69 276 14210 0 62 88320 0 malloc-512 512 158 10 492 0 30 86016 0 UMA Kegs 384 210 6 210 0 30 82944 0 malloc-8192 8192 8 2 99 0 1 81920 0 malloc-128 128 271 318 47251 0 126 75392 0 malloc-65536 65536 1 0 1 0 1 65536 0 malloc-32768 32768 1 1 113 0 1 65536 0 malloc-8192 8192 3 5 304 0 1 65536 0 tcp_inpcb 488 7 121 5070 0 254 62464 0 malloc-128 128 188 277 76714 0 126 59520 0 malloc-64 64 385 497 58969 0 254 56448 0 malloc-128 128 399 35 524 0 126 55552 0 32 Bucket 256 58 152 101196 0 62 53760 0 malloc-512 512 89 15 199 0 30 53248 0 malloc-64 64 289 530 13499 0 254 52416 0 malloc-4096 4096 8 4 840 0 2 49152 0 malloc-2048 2048 11 13 2166 0 8 49152 0 unpcb 256 16 164 7777 0 254 46080 0 pipe 760 21 39 2862 0 16 45600 0 malloc-64 64 533 160 1807 0 254 44352 0 Files 80 215 335 144234 0 126 44000 0 DIRHASH 1024 36 4 36 0 16 40960 0 malloc-2048 2048 8 12 324 0 8 40960 0 64 pcpu 8 4453 667 12589 0 254 40960 0 malloc-128 128 77 233 25262 0 126 39680 0 malloc-256 256 15 135 25298 0 62 38400 0 tcp_bbr_pcb 832 2 43 1241 0 16 37440 0 NAMEI 1024 0 36 149139 0 16 36864 0 malloc-2048 2048 2 16 372 0 8 36864 0 selfd 64 179 388 199750 0 254 36288 0 16 Bucket 144 104 148 25410 0 62 36288 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-32768 32768 1 0 1 0 1 32768 0 malloc-8192 8192 1 3 29 0 1 32768 0 malloc-8192 8192 4 0 4 0 1 32768 0 sctp_stream_msg_out 112 38 250 2424 0 254 32256 0 malloc-256 256 92 28 2228 0 62 30720 0 mt_stats_zone 64 448 0 448 0 254 28672 0 TURNSTILE 136 190 20 190 0 62 28560 0 sctp_laddr 48 58 530 7465 0 254 28224 0 malloc-32 32 344 538 76665 0 254 28224 0 malloc-32 32 295 587 51889 0 254 28224 0 KNOTE 160 28 147 260180 0 62 28000 0 ttyinq 160 135 40 300 0 62 28000 0 8 Bucket 80 75 275 69546 0 126 28000 0 malloc-8192 8192 3 0 3 0 1 24576 0 malloc-4096 4096 4 2 12 0 2 24576 0 malloc-4096 4096 1 5 24470 0 2 24576 0 PWD 32 19 737 24321 0 254 24192 0 ttyoutq 256 72 18 160 0 62 23040 0 Mountpoints 2816 2 6 2 0 4 22528 0 udpcb 32 10 620 57431 0 254 20160 0 4 Bucket 48 17 403 7625 0 254 20160 0 2 Bucket 32 81 549 32354 0 254 20160 0 malloc-128 128 116 39 128 0 126 19840 0 SLEEPQUEUE 88 190 34 190 0 126 19712 0 ripcb 488 3 37 1051 0 254 19520 0 routing nhops 256 24 51 238 0 62 19200 0 malloc-16384 16384 1 0 1 0 1 16384 0 malloc-16384 16384 0 1 136 0 1 16384 0 malloc-8192 8192 2 0 2 0 1 16384 0 malloc-1024 1024 16 0 16 0 16 16384 0 rtentry 176 31 61 238 0 62 16192 0 malloc-64 64 122 130 978 0 254 16128 0 malloc-32 32 50 454 155700 0 254 16128 0 vtnet_tx_hdr 24 2 666 1066739 0 254 16032 0 malloc-16 16 249 751 27748 0 254 16000 0 malloc-16 16 288 712 30250 0 254 16000 0 malloc-256 256 7 53 849 0 62 15360 0 malloc-4096 4096 1 2 4 0 2 12288 0 malloc-2048 2048 3 3 25 0 8 12288 0 malloc-1024 1024 2 10 290 0 16 12288 0 malloc-1024 1024 10 2 10 0 16 12288 0 malloc-1024 1024 7 5 173 0 16 12288 0 malloc-1024 1024 7 5 662 0 16 12288 0 malloc-512 512 0 24 187 0 30 12288 0 malloc-64 64 33 156 1220 0 254 12096 0 malloc-64 64 58 131 26198 0 254 12096 0 mt_zone 24 448 53 448 0 254 12024 0 udp_inpcb 488 2 22 1297 0 254 11712 0 malloc-2048 2048 3 1 194 0 8 8192 0 malloc-1024 1024 6 2 6 0 16 8192 0 pf tags 104 0 78 22 0 126 8112 0 rl_entry 40 105 97 105 0 254 8080 0 malloc-64 64 6 120 3874 0 254 8064 0 malloc-32 32 27 225 3412 0 254 8064 0 malloc-32 32 18 234 837 0 254 8064 0 malloc-32 32 10 242 768 0 254 8064 0 malloc-32 32 26 226 3666 0 254 8064 0 rentr 24 0 334 624 0 254 8016 0 malloc-16 16 31 469 22655 0 254 8000 0 malloc-16 16 6 494 265 0 254 8000 0 malloc-16 16 4 496 5 0 254 8000 0 malloc-16 16 246 254 246 0 254 8000 0 malloc-16 16 14 486 23675 0 254 8000 0 tcp_log_node 120 0 66 16 0 126 7920 0 sctp_readq 152 0 52 939 0 254 7904 0 procdesc 136 3 55 72 0 62 7888 0 kenv 258 3 27 1102 0 30 7740 0 FPU_save_area 832 1 8 1 0 16 7488 0 cpuset 104 7 55 7 0 126 6448 0 epoch_record pcpu 256 4 12 4 0 62 4096 0 malloc-2048 2048 1 1 18 0 8 4096 0 malloc-512 512 2 6 2 0 30 4096 0 tcp_log_bucket 176 0 23 8 0 62 4048 0 sctp_asconf_ack 48 1 83 38 0 254 4032 0 sackhole 32 0 126 1 0 254 4032 0 hostcache 96 2 40 2 0 254 4032 0 syncache 168 0 24 4 0 254 4032 0 ipq 56 0 72 2 0 254 4032 0 UMA Slabs 1 176 10 12 10 0 62 3872 0 KMAP ENTRY 96 12 27 12 0 126 3744 0 vmem 1856 1 1 1 0 8 3712 0 int pcpu 4 34 478 34 0 254 2048 0 SMR CPU 32 2 29 2 0 254 992 0 SMR SHARED 24 2 29 2 0 254 744 0 FFS1 dinode 128 0 0 0 0 126 0 0 swblk 136 0 0 0 0 62 0 0 swpctrie 144 0 0 0 0 62 0 0 sctp_asconf 40 0 0 0 0 254 0 0 pf state scrubs 40 0 0 0 0 254 0 0 pf frag entries 40 0 0 0 0 254 0 0 pf frags 248 0 0 0 0 62 0 0 pf table entries 160 0 0 0 0 62 0 0 pf table entry counters 64 0 0 0 0 254 0 0 pf source nodes 136 0 0 0 0 254 0 0 pf state keys 88 0 0 0 0 126 0 0 pf states 296 0 0 0 0 0 0 0 pf mtags 48 0 0 0 0 254 0 0 tcpreass 48 0 0 0 0 254 0 0 tfo_ccache_entries 80 0 0 0 0 126 0 0 tfo 4 0 0 0 0 254 0 0 tcptw 88 0 0 0 0 254 0 0 IPsec SA lft_c 16 0 0 0 0 254 0 0 itimer 352 0 0 0 0 30 0 0 AIOLIO 280 0 0 0 0 30 0 0 AIOCB 752 0 0 0 0 16 0 0 AIOP 32 0 0 0 0 254 0 0 AIO 208 0 0 0 0 62 0 0 TMPFS node 232 0 0 0 0 62 0 0 NCLNODE 592 0 0 0 0 16 0 0 LTS VFS Cache 360 0 0 0 0 30 0 0 L VFS Cache 320 0 0 0 0 30 0 0 STS VFS Cache 144 0 0 0 0 62 0 0 crypto_session 72 0 0 0 0 126 0 0 cryptop 280 0 0 0 0 30 0 0 nvme_request 128 0 0 0 0 126 0 0 IOMMU_MAP_ENTRY 120 0 0 0 0 126 0 0 ktls_session 192 0 0 0 0 62 0 0 mbuf_jumbo_16k 16384 0 0 0 0 254 0 0 mbuf_jumbo_9k 9216 0 0 0 0 254 0 0 audit_record 1280 0 0 0 0 8 0 0 domainset 40 0 0 0 0 254 0 0 MAC labels 40 0 0 0 0 254 0 0 mdpbuf 832 0 0 0 0 4 0 0 nfspbuf 832 0 0 0 0 16 0 0 swwbuf 832 0 0 0 0 8 0 0 swrbuf 832 0 0 0 0 16 0 0 umtx_shm 88 0 0 0 0 126 0 0 umtx pi 96 0 0 0 0 126 0 0 rangeset pctrie nodes 144 0 0 0 0 62 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-65536 65536 0 0 0 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-32768 32768 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-16384 16384 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-8192 8192 0 0 0 0 1 0 0 malloc-4096 4096 0 0 0 0 2 0 0 malloc-4096 4096 0 0 0 0 2 0 0 malloc-512 512 0 0 0 0 30 0 0 fakepg 104 0 0 0 0 126 0 0 UMA Hash 256 0 0 0 0 62 0 0