syzbot


witness: reversal: &cq->cq_mtx &sched_lock

Status: upstream: reported on 2025/04/22 03:22
Reported-by: syzbot+8f5c1beabd11c2de64bb@syzkaller.appspotmail.com
First crash: 274d, last: 2h13m

Sample crash report:
witness: lock order reversal:
 1st 0xffffffff83819e30 &cq->cq_mtx (&cq->cq_mtx)
 2nd 0xffffffff838b8ee8 &sched_lock (&sched_lock)
lock order [1] &cq->cq_mtx (&cq->cq_mtx) -> [2] &sched_lock (&sched_lock)
#0  mtx_enter+0x95 sys/kern/kern_lock.c:406
#1  sleep_setup+0xed sys/kern/kern_synch.c:304
#2  msleep_nsec+0x10b sys/kern/kern_synch.c:217
#3  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:589
#4  dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#4  dtclose+0x109 sys/dev/dt/dt_dev.c:239
#5  spec_close+0x466 sys/kern/spec_vnops.c:-1
#6  VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
#7  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#7  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#8  fdrop+0x121 sys/kern/kern_descrip.c:1280
#9  closef+0x192 sys/kern/kern_descrip.c:1264
#10 syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#10 syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#11 Xsyscall+0x128
lock order [2] &sched_lock (&sched_lock) -> [1] &cq->cq_mtx (&cq->cq_mtx)
#0  mtx_enter+0x95 sys/kern/kern_lock.c:406
#1  clockintr_cancel+0x2f sys/kern/kern_clockintr.c:323
#2  mi_switch+0x187 sys/kern/sched_bsd.c:376
#3  sleep_finish+0x2d8 sys/kern/kern_synch.c:369
#4  sys_futex+0xb90 futex_wait sys/kern/sys_futex.c:306 [inline]
#4  sys_futex+0xb90 sys/kern/sys_futex.c:155
#5  syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#5  syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#6  Xsyscall+0x128
Stopped at      db_enter+0x25:  addq    $0x8,%rsp
ddb{1}> 
ddb{1}> set $lines = 0
ddb{1}> set $maxwidth = 0
ddb{1}> show panic
the kernel did not panic
ddb{1}> trace
db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
witness_checkorder(ffffffff838b8ee8,9,0) at witness_checkorder+0x10d1 sys/kern/subr_witness.c:-1
mtx_enter(ffffffff838b8ed8) at mtx_enter+0x95 sys/kern/kern_lock.c:406
sleep_setup(ffffffff83819e80,220,ffffffff83433ce6) at sleep_setup+0xed sys/kern/kern_synch.c:304
msleep_nsec(ffffffff83819e80,ffffffff83819e20,220,ffffffff83433ce6,ffffffffffffffff) at msleep_nsec+0x10b sys/kern/kern_synch.c:217
dt_ioctl_record_stop(ffff80000150b000) at dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:589
dtclose(21e5f,81,2000,ffff8000ffff2d10) at dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
dtclose(21e5f,81,2000,ffff8000ffff2d10) at dtclose+0x109 sys/dev/dt/dt_dev.c:239
spec_close(ffff8000357f5190) at spec_close+0x466 sys/kern/spec_vnops.c:-1
VOP_CLOSE(fffffd8061554da0,81,fffffd80097fd5b0,ffff8000ffff2d10) at VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
vn_closefile(fffffd806c8ea860,ffff8000ffff2d10) at vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
vn_closefile(fffffd806c8ea860,ffff8000ffff2d10) at vn_closefile+0x12b sys/kern/vfs_vnops.c:615
fdrop(fffffd806c8ea860,ffff8000ffff2d10) at fdrop+0x121 sys/kern/kern_descrip.c:1280
closef(fffffd806c8ea860,ffff8000ffff2d10) at closef+0x192 sys/kern/kern_descrip.c:1264
syscall(ffff8000357f53f0) at syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
syscall(ffff8000357f53f0) at syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x1f813cb62b0, count: -14
ddb{1}> show registers
rdi                                0
rsi                          0x80000    acpi_pdirpa+0x6be71
rbp               0xffff8000357f4e60
rbx                                0
rdx               0xffff800001593600
rcx               0xffff8000ffff2d10
rax                          0x7ffff    acpi_pdirpa+0x6be70
r8                0xffff8000357f4d40
r9                0x8080808080808080
r10               0xddbfe979aab29d49
r11               0xe9ff4bf0d1346258
r12               0xfffffd8003f50e40
r13               0xfffffd800473ed58
r14                              0x3
r15               0xffffffff8341feeb    substchar+0xbf4f
rip               0xffffffff81595d95    db_enter+0x25
cs                               0x8
rflags                         0x246
rsp               0xffff8000357f4e50
ss                              0x10
db_enter+0x25:  addq    $0x8,%rsp
ddb{1}> show proc
PROC (syz-executor) tid=206264 pid=97209 tcnt=2 stat=onproc
    flags process=0 proc=4000000<THREAD>
    runpri=32, usrpri=50, slppri=32, nice=20
    wchan=0x0, wmesg=, ps_single=0x0 scnt=0 ecnt=0
    forw=0xffffffffffffffff, list=0xffff8000ffff27e0,0xffff8000ffff2558
    process=0xffff80003c41eb70 user=0xffff8000357f0000, vmspace=0xfffffd806c9657b8
    estcpu=36, cpticks=0, pctcpu=0.0, user=0, sys=0, intr=0
ddb{1}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 12961  182238   4778      0  2           0                syz-executor
 34928  212334  48432      0  7           0                syz-executor
 34928  277681  48432      0  2   0x4000000                syz-executor
 97209  232347  22969      0  2           0                syz-executor
*97209  206264  22969      0  7   0x4000000                syz-executor
 89965  459748  19205      0  2           0                syz-executor
 89965  371096  19205      0  3   0x4000080  fsleep        syz-executor
 89965  117292  19205      0  3   0x4000080  fsleep        syz-executor
 59739   45855  77714      0  3        0x80  nanoslp       syz-executor
 59739  376847  77714      0  3   0x4000080  sbwait        syz-executor
 59739  432639  77714      0  3   0x4000080  sbwait        syz-executor
 59739  212555  77714      0  3   0x4000080  fsleep        syz-executor
 41952   74031  78002      0  3      0x3000  suspend       syz-executor
 41952  106634  78002      0  3   0x4081000  biowait       syz-executor
 41952  488487  78002      0  3   0x4081000  fltagain2     syz-executor
 41952  211515  78002      0  3   0x4081000  pipecl        syz-executor
 22969    9023  27071      0  3        0x82  nanoslp       syz-executor
 87608  360496      1      0  3    0x100083  ttyopn        getty
 26642  427730      0      0  3     0x14200  acct          acct
  4778  287640  27071      0  3        0x82  nanoslp       syz-executor
 48432  427954  27071      0  3        0x82  nanoslp       syz-executor
  9405  520798  27071      0  3        0x82  nanoslp       syz-executor
 78002  307068  27071      0  3        0x82  nanoslp       syz-executor
 44793  315320  27071      0  2         0x3                syz-executor
 15542  319441  18621      0  3    0x100082  sbwait        arp
 18621  492939      1      0  3    0x10008a  sigsusp       sh
 77714  314572  27071      0  3        0x82  nanoslp       syz-executor
 19205  285466  27071      0  3        0x82  nanoslp       syz-executor
 27071  430932  40216      0  3        0x82  kqread        syz-executor
 40216   53126  45752      0  3    0x10008a  sigsusp       ksh
 45752  272629  13866      0  3        0x98  kqread        sshd-session
 13866  514510  31634      0  3        0x92  kqread        sshd-session
 31634  517260      1      0  3        0x88  kqread        sshd
 42497  214620  82756     74  3   0x1100092  bpf           pflogd
 82756  450690      1      0  3        0x80  sbwait        pflogd
 87776  169649  54246     73  3   0x1100090  kqread        syslogd
 54246  466875      1      0  3    0x100082  sbwait        syslogd
 57911   83012      1      0  3    0x100080  kqread        resolvd
 73601    1762      0      0  3     0x14200  bored         smr
 73729  276910      0      0  2     0x14200                zerothread
  8512  377591      0      0  3     0x14200  aiodoned      aiodoned
  3205  480082      0      0  3     0x14200  syncer        update
 83656   86754      0      0  3     0x14200  cleaner       cleaner
 44096  423173      0      0  3     0x14200  reaper        reaper
 95690  508261      0      0  3     0x14200  pgdaemon      pagedaemon
 97892  213181      0      0  3     0x14200  bored         viomb
 56133  124730      0      0  3  0x40014200  acpi0         acpi0
 37499   86437      0      0  3  0x40014200                idle1
 13262  248541      0      0  3     0x14200  bored         softnet1
 62696  355868      0      0  3     0x14200  bored         softnet0
 29958  448825      0      0  3     0x14200  bored         systqmp
 53237  264490      0      0  3     0x14200  bored         systq
 43564  424835      0      0  3     0x14200  tmoslp        softclockmp
 84508  306018      0      0  3  0x40014200  tmoslp        softclock
 46348  354346      0      0  3  0x40014200                idle0
     1  435221      0      0  3     0x80082  wait          init
     0       0     -1      0  3  0x10010200  scheduler     swapper
ddb{1}> show all locks
CPU 1:
exclusive mutex &cq->cq_mtx r = 0 (0xffffffff83819e30)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  mtx_enter+0x4b4 sys/kern/kern_lock.c:487
#2  clockintr_unbind+0x56 sys/kern/kern_clockintr.c:375
#3  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:589
#4  dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#4  dtclose+0x109 sys/dev/dt/dt_dev.c:239
#5  spec_close+0x466 sys/kern/spec_vnops.c:-1
#6  VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
#7  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#7  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#8  fdrop+0x121 sys/kern/kern_descrip.c:1280
#9  closef+0x192 sys/kern/kern_descrip.c:1264
#10 syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#10 syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#11 Xsyscall+0x128
Process 34928 (syz-executor) thread 0xffff800031fd0550 (277681)
exclusive rwlock vmmaplk r = 0 (0xfffffd806f5cf4d8)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  rw_do_enter_read+0x3e8 sys/kern/kern_rwlock.c:413
#2  uvmfault_lookup+0x122 sys/uvm/uvm_fault.c:1880
#3  uvm_fault_check+0x4f sys/uvm/uvm_fault.c:693
#4  uvm_fault+0x106 sys/uvm/uvm_fault.c:627
#5  upageflttrap+0xa9 sys/arch/amd64/amd64/trap.c:192
#6  usertrap+0x42f sys/arch/amd64/amd64/trap.c:632
#7  recall_trap+0x8
Process 97209 (syz-executor) thread 0xffff8000ffff2d10 (206264)
exclusive rwlock dtlk r = 0 (0xffffffff83872ef8)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  rw_do_enter_write+0x419 sys/kern/kern_rwlock.c:320
#2  dt_ioctl_record_stop+0x2e sys/dev/dt/dt_dev.c:572
#3  dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#3  dtclose+0x109 sys/dev/dt/dt_dev.c:239
#4  spec_close+0x466 sys/kern/spec_vnops.c:-1
#5  VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
#6  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#6  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#7  fdrop+0x121 sys/kern/kern_descrip.c:1280
#8  closef+0x192 sys/kern/kern_descrip.c:1264
#9  syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#9  syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#10 Xsyscall+0x128
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff839f0d40)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  vn_closefile+0x41 sys/kern/vfs_vnops.c:608
#2  fdrop+0x121 sys/kern/kern_descrip.c:1280
#3  closef+0x192 sys/kern/kern_descrip.c:1264
#4  syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#4  syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#5  Xsyscall+0x128
exclusive mutex &cq->cq_mtx r = 0 (0xffffffff83819e30)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  mtx_enter+0x4b4 sys/kern/kern_lock.c:487
#2  clockintr_unbind+0x56 sys/kern/kern_clockintr.c:375
#3  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:589
#4  dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#4  dtclose+0x109 sys/dev/dt/dt_dev.c:239
#5  spec_close+0x466 sys/kern/spec_vnops.c:-1
#6  VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
#7  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#7  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#8  fdrop+0x121 sys/kern/kern_descrip.c:1280
#9  closef+0x192 sys/kern/kern_descrip.c:1264
#10 syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#10 syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#11 Xsyscall+0x128
Process 41952 (syz-executor) thread 0xffff800031fd5cb8 (106634)
exclusive rrwlock inode r = 0 (0xfffffd807c33a220)
#0  witness_lock+0x5f1 stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5f1 sys/kern/subr_witness.c:1160
#1  rw_do_enter_write+0x419 sys/kern/kern_rwlock.c:320
#2  rrw_enter+0xc6 sys/kern/kern_rwlock.c:621
#3  VOP_LOCK+0xbd sys/kern/vfs_vops.c:527
#4  vn_lock+0xa4 sys/kern/vfs_vnops.c:570
#5  vn_write+0x18f sys/kern/vfs_vnops.c:405
#6  dofilewritev+0x242 sys/kern/sys_generic.c:380
#7  sys_write+0xa2 sys/kern/sys_generic.c:300
#8  syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#8  syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
#9  Xsyscall+0x128
ddb{1}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf 11091  12179K   12427K 166960K     13901        0
            pcb    17     15K      16K 166960K       688        0
         rtable   268     15K      15K 166960K      1105        0
             pf    44     19K      81K 166960K       380        0
         ifaddr    46      9K       9K 166960K       248        0
        ifgroup    69      3K       3K 166960K       441        0
         sysctl     4      1K       9K 166960K        29        0
       counters    78     37K      39K 166960K       670        0
       ioctlops     0      0K       4K 166960K      2420        0
            iov     0      0K      24K 166960K       258        0
          mount     1      1K       1K 166960K         1        0
            log     0      0K       0K 166960K         4        0
         vnodes  1343     84K      85K 166960K      3323        0
      UFS quota     1     32K      32K 166960K         1        0
      UFS mount     5     36K      36K 166960K         5        0
            shm     2      1K       9K 166960K        12        0
         VM map     2      1K       1K 166960K         2        0
            sem    15      0K       0K 166960K       262        0
        dirhash    12      2K       2K 166960K        69        0
           ACPI  1692    195K     286K 166960K     12470        0
      file desc    19     69K     252K 166960K      3030        0
          sigio     0      0K       0K 166960K        65        0
           proc    68     83K     180K 166960K      1102        0
        subproc    81      5K       5K 166960K       163        0
    NFS srvsock     1      0K       0K 166960K         1        0
     NFS daemon     1     16K      16K 166960K         1        0
    ip_moptions     0      0K       0K 166960K       592        0
       in_multi    79      5K       6K 166960K       297        0
    ether_multi     1      0K       0K 166960K        21        0
            mrt     1      0K       0K 166960K        28        0
    ISOFS mount     1     32K      32K 166960K         1        0
  MSDOSFS mount     1     16K      16K 166960K         1        0
           ttys   103    466K     466K 166960K       103        0
           exec     0      0K       1K 166960K      1012        0
   fusefs mount     1     32K      32K 166960K         1        0
     pfkey data     0      0K       0K 166960K        10        0
            tdb     3      0K       0K 166960K         3        0
        VM swap     8     62K      64K 166960K        10        0
       UVM amap   234    131K     176K 166960K     29288        0
       UVM aobj   124     16K      18K 166960K       135        0
     pinsyscall    39     78K     107K 166960K      4376        0
        memdesc     1      4K       4K 166960K         1        0
    crypto data     1      1K       1K 166960K         1        0
    ip6_options     0      0K       1K 166960K       223        0
            NDP    15      0K       1K 166960K       180        0
           temp    83   8684K    8812K 166960K    128903        0
         kqueue     8     14K      32K 166960K       613        0
      SYN cache     2     16K      16K 166960K         2        0
ddb{1}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache    128       26    0        0     1     0     1     1     0     8    0
rtpcb      120      426    0      424     5     4     1     3     0     8    0
rtentry    176      292    0      195     5     0     5     5     0     8    0
unpcb      144     2187    0     2175    16    14     2     6     0     8    1
syncache   336       10    0       10     4     4     0     1     0     8    0
tcpqe       32       26    0       26     5     5     0     1     0     8    0
tcpcb      736      812    0      803    12     9     3     7     0     8    0
arp        136       43    0       26     1     0     1     1     0     8    0
inpcb      328     3300    0     3290    32    27     5    13     0     8    2
nd6        152       60    0       36     1     0     1     1     0     8    0
pkpcb       40       26    0       26     6     6     0     1     0     8    0
kcovpl      48       18    0        9     1     0     1     1     0     8    0
ppxss      1192     251    0      250     4     3     1     1     0     8    0
pppxif     1504      24    0       23     6     5     1     1     0     8    0
pffrag     232       61    0       50     1     0     1     1     0   482    0
pffrnode    88       58    0       48     1     0     1     1     0     8    0
pffrent     40      111    0       99     1     0     1     1     0     8    0
pfosfp      40     1428    0     1005     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfrktable  1344       1    0        1     1     1     0     1     0     8    0
pfanchor   1288       4    0        0     1     0     1     1     0     8    0
pftag       88        1    0        1     1     1     0     1     0     8    0
pfstitem    24        4    0        0     1     0     1     1     0     8    0
pfstkey    128        6    0        3     1     0     1     1     0     8    0
pfstate    448        3    0        1     1     0     1     1     0     8    0
pfrule     1360      18    0        4     2     0     2     2     0     8    0
rttmr      136        6    0        6     4     3     1     1     0     8    1
art_heap8  4096       7    0        1     7     1     6     6     0     8    0
art_heap4  256     1327    0      907    38    11    27    27     0     8    0
art_table   40     1334    0      908     5     0     5     5     0     8    0
art_node    32      289    0      204     1     0     1     1     0     8    0
sysvmsgpl   40        2    0        2     2     1     1     1     0     8    1
semupl     112        3    0        3     3     3     0     1     0     8    0
semapl     112      254    0      241     1     0     1     1     0     8    0
shmpl      112      122    0        6     5     1     4     4     0     8    0
dirhash    1024      55    0       38     3     0     3     3     0     8    0
dino2pl    256     7354    0     5821    97     0    97    97     0     8    0
ffsino     296     7354    0     5821   120     1   119   119     0     8    0
nchpl      144    11444    0     9718    65     0    65    65     0     8    0
rtmask      32       28    0       28     3     3     0     1     0     8    0
vnodes     216     4383    0        0   244     0   244   244     0     8    0
namei      1024   39824    0    39824     4     3     1     2     0     8    1
percpumem   16      350    0      296     1     0     1     1     0     8    0
vcpupl     3968       6    0        1     1     0     1     1     0     8    0
vmpool     848        9    0        4     1     0     1     1     0     8    0
kstatmem   264      292    0      258     4     1     3     3     0     8    0
acpiwqpl    32        1    0        1     1     0     1     1     1     8    1
scsiplug    72       11    0       11     6     5     1     1     0     8    1
scxspl     216    69247    0    69246    12    11     1     8     1     8    0
plimitpl   152      754    0      736     1     0     1     1     0     8    0
sigapl     424     3296    0     3249     8     0     8     8     0     8    0
knotepl    120      956    0        0    27     0    27    27     0     8    0
kqueuepl   224     1286    0     1278    18    16     2     5     0     8    1
pipepl     344      578    0      547    12     8     4     9     0     8    0
fdescpl    528     3250    0     3220     3     0     3     3     0     8    0
filepl     160    22509    0    22291    38    24    14    18     0     8    1
lockfpl    104     1138    0     1137     2     1     1     2     0     8    0
lockfspl    48      404    0      403     1     0     1     1     0     8    0
sessionpl  144       38    0       30     1     0     1     1     0     8    0
pgrppl      48      102    0       85     1     0     1     1     0     8    0
ucredpl    104     4012    0     4001     1     0     1     1     0     8    0
zombiepl   144     4770    0     4768     1     0     1     1     0     8    0
processpl  1232    3296    0     3249     6     1     5     6     0     8    0
procpl     664     8243    0     8186     9     3     6     8     0     8    0
sosppl     176       16    0       16     5     4     1     1     0     8    1
sockpl     752     6173    0     6149    56    44    12    18     0     8    7
mcl64k     65536     22    0        0     3     1     2     3     0     8    0
mcl16k     16384      4    0        0     1     0     1     1     0     8    0
mcl12k     12288      2    0        0     1     0     1     1     0     8    0
mcl9k      9216       3    0        0     1     0     1     1     0     8    0
mcl8k      8192       3    0        0     1     0     1     1     0     8    0
mcl4k      4096     130    0        0    17     0    17    17     0     8    0
mcl2k2     2112       1    0        0     1     0     1     1     0     8    0
mcl2k      2048      43    0        0     5     0     5     5     0     8    0
mtagpl      96       10    0        0     1     0     1     1     0     8    0
mbufpl     256      808    0        0    50     0    50    50     0     8    0
bufpl      280    28354    0    22218   439     0   439   439     0     8    0
anonpl      32    14371    0        0   116     0   116   116     0   246    0
amapchunkpl 152  102494    0   101909    64    28    36    38     0   158    7
amappl16   200    10627    0    10441    76    62    14    31     0     8    0
amappl15   192        1    0        0     1     0     1     1     0     8    0
amappl14   184        4    0        4     2     1     1     1     0     8    1
amappl13   176      553    0      551     1     0     1     1     0     8    0
amappl12   168     3675    0     3637     3     0     3     3     0     8    0
amappl11   160        4    0        4     2     2     0     1     0     8    0
amappl10   152       47    0       39     1     0     1     1     0     8    0
amappl9    144      258    0      257     1     0     1     1     0     8    0
amappl8    136       23    0       20     1     0     1     1     0     8    0
amappl7    128      114    0      113     1     0     1     1     0     8    0
amappl6    120      394    0      381     1     0     1     1     0     8    0
amappl5    112       78    0       69     1     0     1     1     0     8    0
amappl4    104      497    0      466     2     1     1     2     0     8    0
amappl3     96    17923    0    17835     4     1     3     3     0     8    0
amappl2     88     3358    0     3297     2     0     2     2     0     8    0
amappl1     80    22610    0    22070    17     2    15    15     0     8    0
amappl      88    27898    0    27728     6     1     5     5     0    92    0
uvmvnodes   80      187    0        0     4     0     4     4     0     8    0
dma65536   65536      2    0        2     2     2     0     1     0     8    0
dma8192    8192       2    0        2     2     2     0     1     0     8    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma2048    2048       1    0        1     1     1     0     1     0     8    0
dma1024    1024       1    0        0     1     0     1     1     0     8    0
dma512     512        2    0        2     2     2     0     1     0     8    0
dma256     256        8    0        8     3     2     1     1     0     8    1
dma128     128      255    0      255     3     3     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       19    0       18     1     0     1     1     0     8    0
aobjpl      72      134    0       11     3     0     3     3     0     8    0
uaddrrnd    24     3250    0     3220     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24     3250    0     3220     1     0     1     1     0     8    0
vmmpekpl   168    26510    0    26445     4     0     4     4     0     8    0
vmmpepl    168   209490    0   207551   149    44   105   117     0   357    5
vmsppl     488     3249    0     3220     5     0     5     5     0     8    0
rwobjpl     80    54844    0    53522    43     6    37    39     0     8    0
pdppl      4096    6525    0     6453   125    45    80    85     0     8    8
pvpl        32    21980    0        0   177     0   177   177     0   265    0
pmappl     256     3258    0     3224     3     0     3     3     0     8    0
extentpl    40       45    0       27     1     0     1     1     0     8    0
phpool     112      385    0       84     9     0     9     9     0     8    0
ddb{1}> machine ddbcpu 0
Stopped at      x86_ipi_db+0x27:        addq    $0x8,%rsp
ddb{0}> trace
x86_ipi_db(ffffffff83818ff0) at x86_ipi_db+0x27 sys/arch/amd64/amd64/db_interface.c:394
x86_ipi_handler() at x86_ipi_handler+0xd9 sys/arch/amd64/amd64/ipi.c:106
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
mtx_enter(ffffffff83819e20) at mtx_enter+0x377 sys/kern/kern_lock.c:454
clockintr_dispatch(ffff80003c40d570) at clockintr_dispatch+0x401 sys/kern/kern_clockintr.c:215
lapic_clockintr(0,0) at lapic_clockintr+0x51 sys/arch/amd64/amd64/lapic.c:482
Xresume_lapic_ltimer() at Xresume_lapic_ltimer+0x2a
end of kernel
end trace frame: 0x7b334acecde0, count: -7
ddb{0}> machine ddbcpu 1
Stopped at      db_enter+0x25:  addq    $0x8,%rsp
ddb{1}> trace
db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
witness_checkorder(ffffffff838b8ee8,9,0) at witness_checkorder+0x10d1 sys/kern/subr_witness.c:-1
mtx_enter(ffffffff838b8ed8) at mtx_enter+0x95 sys/kern/kern_lock.c:406
sleep_setup(ffffffff83819e80,220,ffffffff83433ce6) at sleep_setup+0xed sys/kern/kern_synch.c:304
msleep_nsec(ffffffff83819e80,ffffffff83819e20,220,ffffffff83433ce6,ffffffffffffffff) at msleep_nsec+0x10b sys/kern/kern_synch.c:217
dt_ioctl_record_stop(ffff80000150b000) at dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:589
dtclose(21e5f,81,2000,ffff8000ffff2d10) at dtclose+0x109 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
dtclose(21e5f,81,2000,ffff8000ffff2d10) at dtclose+0x109 sys/dev/dt/dt_dev.c:239
spec_close(ffff8000357f5190) at spec_close+0x466 sys/kern/spec_vnops.c:-1
VOP_CLOSE(fffffd8061554da0,81,fffffd80097fd5b0,ffff8000ffff2d10) at VOP_CLOSE+0x132 sys/kern/vfs_vops.c:156
vn_closefile(fffffd806c8ea860,ffff8000ffff2d10) at vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
vn_closefile(fffffd806c8ea860,ffff8000ffff2d10) at vn_closefile+0x12b sys/kern/vfs_vnops.c:615
fdrop(fffffd806c8ea860,ffff8000ffff2d10) at fdrop+0x121 sys/kern/kern_descrip.c:1280
closef(fffffd806c8ea860,ffff8000ffff2d10) at closef+0x192 sys/kern/kern_descrip.c:1264
syscall(ffff8000357f53f0) at syscall+0xbd4 mi_syscall sys/sys/syscall_mi.h:176 [inline]
syscall(ffff8000357f53f0) at syscall+0xbd4 sys/arch/amd64/amd64/trap.c:775
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x1f813cb62b0, count: -14

Crashes (110):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/01/21 10:28 openbsd 5c0c839c3d8f 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/20 14:07 openbsd 7d7bacacae3d 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/19 22:33 openbsd 665aeba05e03 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/18 15:29 openbsd 4409e1965d65 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/18 10:54 openbsd 4409e1965d65 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/17 00:20 openbsd 8105fbd98be2 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/16 02:16 openbsd c600931321c7 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/15 01:26 openbsd ac9e191703f1 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/14 21:10 openbsd bf518416d64f 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/14 19:41 openbsd bf518416d64f 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/14 18:05 openbsd bf518416d64f 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/14 13:39 openbsd bf518416d64f 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/13 15:31 openbsd 67ddd48175b9 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/12 22:41 openbsd a9df5e8d0934 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/11 17:39 openbsd 294bdc40ca4d 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/11 13:29 openbsd 294bdc40ca4d 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/11 09:14 openbsd 294bdc40ca4d 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/10 03:01 openbsd ee14140d0567 56f88057 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/09 12:40 openbsd 0f50c032e950 c1f5c7d0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/09 00:23 openbsd b9d9e3fc96bc c1f5c7d0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/08 14:34 openbsd b9d9e3fc96bc c1f5c7d0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/08 03:45 openbsd d77c41cf8f5a c1f5c7d0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/08 01:30 openbsd d77c41cf8f5a c1f5c7d0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/07 06:22 openbsd c7117ed52e15 15f6fd08 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/07 04:28 openbsd c7117ed52e15 15f6fd08 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/07 01:26 openbsd c7117ed52e15 15f6fd08 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/06 12:45 openbsd 4459f914e6b0 15f6fd08 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/05 20:21 openbsd 97968167a1e0 917fe602 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/05 14:09 openbsd 97968167a1e0 917fe602 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/05 11:25 openbsd 97968167a1e0 917fe602 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/05 10:09 openbsd 97968167a1e0 917fe602 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/04 17:22 openbsd 079c76c73fb6 e3fbc05e .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/03 05:04 openbsd 287000ffa12f e3fbc05e .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/02 19:57 openbsd e29e388b5d06 af6b70b9 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/02 13:17 openbsd e29e388b5d06 af6b70b9 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2026/01/01 13:49 openbsd 6a1433bd6d39 2733a373 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/30 23:20 openbsd e93b8ae725cf d32f762a .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/30 20:04 openbsd e93b8ae725cf d32f762a .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/30 11:08 openbsd 163664d81c58 17e6c961 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/30 07:51 openbsd 163664d81c58 17e6c961 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/29 16:31 openbsd 4420759a09aa 9831da75 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/27 10:43 openbsd fed22d3a29d8 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/27 07:06 openbsd 5ce3a7f811ea 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/26 15:27 openbsd 97e1364f97a2 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/26 06:21 openbsd 97e1364f97a2 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/25 05:03 openbsd 37a2fc69fcf0 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/24 10:14 openbsd 1dc30a6de584 5dc09de1 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/21 00:59 openbsd df8f251ef5f9 e14dbeb9 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/12/19 02:16 openbsd 3f23c8878a83 e14dbeb9 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/04/22 03:22 openbsd e8eeea37c7e6 552876f8 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
* Struck through repros no longer work on HEAD.