syzbot


witness: reversal: &cq->cq_mtx &sched_lock

Status: upstream: reported on 2025/04/22 03:22
Reported-by: syzbot+8f5c1beabd11c2de64bb@syzkaller.appspotmail.com
First crash: 63d, last: 22h44m

Sample crash report:
witness: lock order reversal:
 1st 0xffff8000299dee00 &cq->cq_mtx (&cq->cq_mtx)
 2nd 0xffffffff83944b58 &sched_lock (&sched_lock)
lock order [1] &cq->cq_mtx (&cq->cq_mtx) -> [2] &sched_lock (&sched_lock)
#0  mtx_enter+0x4a sys/kern/kern_lock.c:257
#1  sleep_setup+0xc6 sys/kern/kern_synch.c:302
#2  msleep_nsec+0x119 sys/kern/kern_synch.c:217
#3  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:576
#4  dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#4  dtclose+0x105 sys/dev/dt/dt_dev.c:232
#5  spec_close+0x45f sys/kern/spec_vnops.c:-1
#6  VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
#7  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#7  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#8  fdrop+0x126 sys/kern/kern_descrip.c:1267
#9  closef+0x192 sys/kern/kern_descrip.c:1251
#10 syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#10 syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#11 Xsyscall+0x128
lock order [2] &sched_lock (&sched_lock) -> [1] &cq->cq_mtx (&cq->cq_mtx)
#0  mtx_enter+0x4a sys/kern/kern_lock.c:257
#1  clockintr_cancel+0x2f sys/kern/kern_clockintr.c:323
#2  mi_switch+0x172 x86_atomic_clearbits_u32 machine/atomic.h:-1 [inline]
#2  mi_switch+0x172 sys/kern/sched_bsd.c:374
#3  sleep_finish+0x2da sys/kern/kern_synch.c:366
#4  sys_futex+0xa24 futex_wait sys/kern/sys_futex.c:301 [inline]
#4  sys_futex+0xa24 sys/kern/sys_futex.c:151
#5  syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#5  syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#6  Xsyscall+0x128
Stopped at      db_enter+0x25:  addq    $0x8,%rsp
ddb{0}> 
ddb{0}> set $lines = 0
ddb{0}> set $maxwidth = 0
ddb{0}> show panic
the kernel did not panic
ddb{0}> trace
db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
witness_checkorder(ffffffff83944b58,9,0) at witness_checkorder+0x10fb sys/kern/subr_witness.c:-1
mtx_enter(ffffffff83944b48) at mtx_enter+0x4a sys/kern/kern_lock.c:257
sleep_setup(ffff8000299dee50,220,ffffffff833af742) at sleep_setup+0xc6 sys/kern/kern_synch.c:302
msleep_nsec(ffff8000299dee50,ffff8000299dedf0,220,ffffffff833af742,ffffffffffffffff) at msleep_nsec+0x119 sys/kern/kern_synch.c:217
dt_ioctl_record_stop(ffff80000149a000) at dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:576
dtclose(11e5f,81,2000,ffff8000fffe82b8) at dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
dtclose(11e5f,81,2000,ffff8000fffe82b8) at dtclose+0x105 sys/dev/dt/dt_dev.c:232
spec_close(ffff80002a3060e0) at spec_close+0x45f sys/kern/spec_vnops.c:-1
VOP_CLOSE(fffffd805e2506d8,81,fffffd80097fb2d8,ffff8000fffe82b8) at VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
vn_closefile(fffffd805cc3a088,ffff8000fffe82b8) at vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
vn_closefile(fffffd805cc3a088,ffff8000fffe82b8) at vn_closefile+0x12b sys/kern/vfs_vnops.c:615
fdrop(fffffd805cc3a088,ffff8000fffe82b8) at fdrop+0x126 sys/kern/kern_descrip.c:1267
closef(fffffd805cc3a088,ffff8000fffe82b8) at closef+0x192 sys/kern/kern_descrip.c:1251
syscall(ffff80002a306340) at syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
syscall(ffff80002a306340) at syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0xa0c37359210, count: -14
ddb{0}> show registers
rdi                                0
rsi                          0x80000    acpi_pdirpa+0x6be71
rbp               0xffff80002a305de0
rbx               0xfffffd8004702330
rdx               0xffff80000147fa40
rcx               0xffff8000fffe82b8
rax                          0x7ffff    acpi_pdirpa+0x6be70
r8                0xffff80002a305cc0
r9                0x8080808080808080
r10               0x930f9eca61f48d02
r11                0x73b9fbab8b54bf2
r12                                0
r13               0xfffffd8003f55300
r14                              0x3
r15                       0xffffffff
rip               0xffffffff8232ace5    db_enter+0x25
cs                               0x8
rflags                         0x246
rsp               0xffff80002a305dd0
ss                              0x10
db_enter+0x25:  addq    $0x8,%rsp
ddb{0}> show proc
PROC (syz-executor) tid=245065 pid=28672 tcnt=2 stat=onproc
    flags process=0 proc=4000000<THREAD>
    runpri=32, usrpri=50, slppri=32, nice=20
    wchan=0x0, wmesg=, ps_single=0x0 scnt=0 ecnt=0
    forw=0xffffffffffffffff, list=0xffff80002a285728,0xffff8000ffffca58
    process=0xffff8000fffdd868 user=0xffff80002a301000, vmspace=0xfffffd806d472d30
    estcpu=36, cpticks=0, pctcpu=0.0, user=0, sys=0, intr=0
ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 64247  263409  14942     -1  2        0x10                syz-executor
 64247  440013  14942     -1  3   0x4000090  sbwait        syz-executor
 14559  100993  80652     -1  3        0x90  nanoslp       syz-executor
 14559  212186  80652     -1  3   0x4000090  fsleep        syz-executor
 32243  491070  47349      0  2           0                syz-executor
 32243    7983  47349      0  3   0x4000080  kqread        syz-executor
 32243   99729  47349      0  3   0x4000080  fsleep        syz-executor
 28672  114196  49353      0  2           0                syz-executor
*28672  245065  49353      0  7   0x4000000                syz-executor
 46838  447177  13472     -1  3        0x90  nanoslp       syz-executor
 46838  503710  13472     -1  3   0x4000090  kqsel         syz-executor
 46838  233140  13472     -1  3   0x4000090  fsleep        syz-executor
 38427  441038  81271      0  3        0x80  nanoslp       syz-executor
 38427  513948  81271      0  3   0x4000080  ttyin         syz-executor
 38427  228336  81271      0  3   0x4000080  fsleep        syz-executor
 38427   22366  81271      0  3   0x4000080  fsleep        syz-executor
 42210  408391  63339      0  3        0x80  nanoslp       syz-executor
 42210  433248  63339      0  3   0x4000080  kqsel         syz-executor
 42210  221878  63339      0  3   0x4000080  fsleep        syz-executor
 47873  387135      0      0  3     0x14200  acct          acct
 47349  430007  47724      0  3        0x82  nanoslp       syz-executor
 80328  311698  47724      0  3         0x2  biowait       syz-executor
 84052  210966      1      0  3    0x100083  ttyin         getty
 14942  130870  47724      0  3        0x82  nanoslp       syz-executor
 36863  377711      0      0  3     0x14200  bored         sosplice
 49353   32163  47724      0  3        0x82  nanoslp       syz-executor
 13472  256470  47724      0  3        0x82  nanoslp       syz-executor
 63339  489368  47724      0  3        0x82  nanoslp       syz-executor
 80652  499398  47724      0  3        0x82  nanoslp       syz-executor
 81271   51113  47724      0  3        0x82  nanoslp       syz-executor
 47724  210534  36594      0  3        0x82  kqread        syz-executor
 36594  194588  13379      0  3    0x10008a  sigsusp       ksh
 13379  196769  38301      0  3        0x98  kqread        sshd-session
 38301  345864  88184      0  3        0x92  kqread        sshd-session
 88184  134643      1      0  3        0x88  kqread        sshd
 54488  277307  56327     74  3   0x1100092  bpf           pflogd
 56327  306007      1      0  3        0x80  sbwait        pflogd
 70228   37148  87168     73  3   0x1100090  kqread        syslogd
 87168  109600      1      0  3    0x100082  sbwait        syslogd
 71943  354864      1      0  3    0x100080  kqread        resolvd
 91744  363224   6620     77  3    0x100092  kqread        dhcpleased
 33129  415894   6620     77  3    0x100092  kqread        dhcpleased
  6620  118281      1      0  3        0x80  kqread        dhcpleased
 53721  117493      0      0  3     0x14200  bored         smr
 55804  419168      0      0  2     0x14200                zerothread
 94680   38423      0      0  3     0x14200  aiodoned      aiodoned
 25912  421113      0      0  3     0x14200  syncer        update
 70619  368150      0      0  3     0x14200  cleaner       cleaner
 42569  478395      0      0  3     0x14200  reaper        reaper
 38832  320776      0      0  3     0x14200  pgdaemon      pagedaemon
 24296  514891      0      0  3     0x14200  bored         viomb
 55645  201765      0      0  3  0x40014200  acpi0         acpi0
 80621  246109      0      0  7  0x40014200                idle1
 74582  134857      0      0  3     0x14200  bored         softnet3
  3487  179295      0      0  3     0x14200  bored         softnet2
  6666   76195      0      0  3     0x14200  bored         softnet1
 87737  298238      0      0  3     0x14200  bored         softnet0
 95559  326604      0      0  3     0x14200  bored         systqmp
 50150  450937      0      0  3     0x14200  bored         systq
 40957  500059      0      0  3     0x14200  tmoslp        softclockmp
 83216  229223      0      0  3  0x40014200  tmoslp        softclock
 27380  211635      0      0  3  0x40014200                idle0
     1  308654      0      0  3        0x82  wait          init
     0       0     -1      0  3  0x10010200  scheduler     swapper
ddb{0}> show all locks
CPU 0:
exclusive mutex &cq->cq_mtx r = 0 (0xffff8000299dee00)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  mtx_enter_try+0x1ad sys/kern/kern_lock.c:301
#2  mtx_enter+0x62 sys/kern/kern_lock.c:258
#3  clockintr_unbind+0x56 sys/kern/kern_clockintr.c:375
#4  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:576
#5  dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#5  dtclose+0x105 sys/dev/dt/dt_dev.c:232
#6  spec_close+0x45f sys/kern/spec_vnops.c:-1
#7  VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
#8  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#8  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#9  fdrop+0x126 sys/kern/kern_descrip.c:1267
#10 closef+0x192 sys/kern/kern_descrip.c:1251
#11 syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#11 syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#12 Xsyscall+0x128
Process 28672 (syz-executor) thread 0xffff8000fffe82b8 (245065)
exclusive rwlock dtlk r = 0 (0xffffffff83854ba8)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  rw_do_enter_write+0x377 sys/kern/kern_rwlock.c:310
#2  dt_ioctl_record_stop+0x68 sys/dev/dt/dt_dev.c:-1
#3  dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#3  dtclose+0x105 sys/dev/dt/dt_dev.c:232
#4  spec_close+0x45f sys/kern/spec_vnops.c:-1
#5  VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
#6  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#6  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#7  fdrop+0x126 sys/kern/kern_descrip.c:1267
#8  closef+0x192 sys/kern/kern_descrip.c:1251
#9  syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#9  syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#10 Xsyscall+0x128
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff83976530)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  vn_closefile+0x41 sys/kern/vfs_vnops.c:608
#2  fdrop+0x126 sys/kern/kern_descrip.c:1267
#3  closef+0x192 sys/kern/kern_descrip.c:1251
#4  syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#4  syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#5  Xsyscall+0x128
exclusive mutex &cq->cq_mtx r = 0 (0xffff8000299dee00)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  mtx_enter_try+0x1ad sys/kern/kern_lock.c:301
#2  mtx_enter+0x62 sys/kern/kern_lock.c:258
#3  clockintr_unbind+0x56 sys/kern/kern_clockintr.c:375
#4  dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:576
#5  dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
#5  dtclose+0x105 sys/dev/dt/dt_dev.c:232
#6  spec_close+0x45f sys/kern/spec_vnops.c:-1
#7  VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
#8  vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
#8  vn_closefile+0x12b sys/kern/vfs_vnops.c:615
#9  fdrop+0x126 sys/kern/kern_descrip.c:1267
#10 closef+0x192 sys/kern/kern_descrip.c:1251
#11 syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#11 syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
#12 Xsyscall+0x128
Process 80328 (syz-executor) thread 0xffff80003c410f98 (311698)
exclusive rrwlock inode r = 0 (0xfffffd806e389538)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  rw_do_enter_write+0x377 sys/kern/kern_rwlock.c:310
#2  rrw_enter+0xc6 sys/kern/kern_rwlock.c:605
#3  VOP_LOCK+0xa6 sys/kern/vfs_vops.c:527
#4  vn_lock+0xa4 sys/kern/vfs_vnops.c:570
#5  vget+0x2bd sys/kern/vfs_subr.c:693
#6  ufs_ihashget+0x185 sys/ufs/ufs/ufs_ihash.c:98
#7  ffs_vget+0x8c sys/ufs/ffs/ffs_vfsops.c:1203
#8  ufs_lookup+0x19f8 sys/ufs/ufs/ufs_lookup.c:478
#9  VOP_LOOKUP+0x6e sys/kern/vfs_vops.c:85
#10 vfs_lookup+0x8fa sys/kern/vfs_lookup.c:566
#11 namei+0x7aa sys/kern/vfs_lookup.c:250
#12 dounlinkat+0xc1 sys/kern/vfs_syscalls.c:1864
#13 syscall+0xb08 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#13 syscall+0xb08 sys/arch/amd64/amd64/trap.c:579
#14 Xsyscall+0x128
exclusive rrwlock inode r = 0 (0xfffffd806c57f200)
#0  witness_lock+0x5bb stacktrace_save sys/sys/stacktrace.h:37 [inline]
#0  witness_lock+0x5bb sys/kern/subr_witness.c:1160
#1  rw_do_enter_write+0x377 sys/kern/kern_rwlock.c:310
#2  rrw_enter+0xc6 sys/kern/kern_rwlock.c:605
#3  VOP_LOCK+0xa6 sys/kern/vfs_vops.c:527
#4  vn_lock+0xa4 sys/kern/vfs_vnops.c:570
#5  vget+0x2bd sys/kern/vfs_subr.c:693
#6  cache_lookup+0x36e sys/kern/vfs_cache.c:222
#7  ufs_lookup+0x21b sys/ufs/ufs/ufs_lookup.c:160
#8  VOP_LOOKUP+0x6e sys/kern/vfs_vops.c:85
#9  vfs_lookup+0x8fa sys/kern/vfs_lookup.c:566
#10 namei+0x7aa sys/kern/vfs_lookup.c:250
#11 dounlinkat+0xc1 sys/kern/vfs_syscalls.c:1864
#12 syscall+0xb08 mi_syscall sys/sys/syscall_mi.h:176 [inline]
#12 syscall+0xb08 sys/arch/amd64/amd64/trap.c:579
#13 Xsyscall+0x128
ddb{0}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf 10252  11061K   11473K 166960K     13445        0
            pcb    18     16K      18K 166960K       741        0
         rtable   243     12K      12K 166960K       800        0
             pf    40     18K   67485K 166960K       256        0
         ifaddr    45      8K       9K 166960K       175        0
        ifgroup    64      2K       3K 166960K       313        0
         sysctl     4      1K       9K 166960K        43        0
       counters    72     37K      38K 166960K       418        0
       ioctlops     0      0K       4K 166960K      2015        0
            iov     0      0K      32K 166960K       173        0
          mount     1      1K       1K 166960K         1        0
            log     0      0K       0K 166960K         4        0
         vnodes  1512     95K      96K 166960K      3549        0
      UFS quota     1     32K      32K 166960K         1        0
      UFS mount     5     36K      36K 166960K         5        0
            shm     2     28K      36K 166960K        45        0
         VM map     2      1K       1K 166960K         2        0
            sem    12      0K       0K 166960K       163        0
        dirhash    12      2K       3K 166960K        78        0
           ACPI  1692    195K     286K 166960K     12470        0
      file desc    17     61K     257K 166960K      2786        0
          sigio     0      0K       0K 166960K        83        0
           proc    76    115K     128K 166960K       908        0
        subproc    72      4K       4K 166960K       101        0
    NFS srvsock     1      0K       0K 166960K         1        0
     NFS daemon     1     16K      16K 166960K         1        0
    ip_moptions     0      0K       0K 166960K       416        0
       in_multi    96      6K       7K 166960K       273        0
    ether_multi     1      0K       0K 166960K        32        0
            mrt     3      0K       0K 166960K        22        0
    ISOFS mount     1     32K      32K 166960K         1        0
  MSDOSFS mount     1     16K      16K 166960K         1        0
           ttys   265   1182K    1182K 166960K       265        0
           exec     0      0K       1K 166960K       819        0
   fusefs mount     1     32K      32K 166960K         1        0
     pfkey data     0      0K       0K 166960K         7        0
            tdb     3      0K       0K 166960K         3        0
        VM swap     8     62K      64K 166960K        10        0
       UVM amap   256    168K     189K 166960K     26638        0
       UVM aobj    41      5K       5K 166960K        44        0
     pinsyscall    42     84K     102K 166960K      4014        0
        memdesc     1      4K       4K 166960K         1        0
    crypto data     1      1K       1K 166960K         1        0
    ip6_options     0      0K       0K 166960K       151        0
            NDP    14      0K       2K 166960K       116        0
           temp    82   8696K    8952K 166960K    143185        0
         kqueue    14     22K      32K 166960K       488        0
      SYN cache     2     16K      16K 166960K         2        0
ddb{0}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache    128       26    0        0     1     0     1     1     0     8    0
rtpcb      120      392    0      389     5     4     1     3     0     8    0
rtentry    176      229    0      127     6     0     6     6     0     8    0
unpcb      144     2106    0     2081    20    19     1     4     0     8    0
syncache   336       24    0       24     7     7     0     1     0     8    0
tcpqe       32       11    0       11     6     6     0     1     0     8    0
tcpcb      736     1229    0     1223    22    18     4     7     0     8    2
arp        128       34    0       13     1     0     1     1     0     8    0
inpcb      328     3745    0     3732    34    27     7    10     0     8    3
nd6        144       39    0       13     2     1     1     2     0     8    0
pkpcb       40       39    0       39     6     5     1     1     0     8    1
kcovpl      48       11    0        3     1     0     1     1     0     8    0
mppekey    1024       4    0        4     3     3     0     1     0     8    0
ppxss      1192     139    0      139     4     3     1     1     0     8    1
pppxif     1504      54    0       54     7     6     1     1     0     8    1
pfstscr     40        1    0        0     1     0     1     1     0     8    0
pffrag     232       23    0       16     1     0     1     1     0   482    0
pffrnode    88       22    0       15     1     0     1     1     0     8    0
pffrent     40       41    0       34     1     0     1     1     0     8    0
pfosfp      40     1428    0     1005     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfanchor   1288       4    0        2     1     0     1     1     0     8    0
pftag       88        3    0        0     1     0     1     1     0     8    0
pfstitem    24      220    0      105     1     0     1     1     0     8    0
pfstkey    128      220    0      105     5     0     5     5     0     8    0
pfstate    384      219    0      105    13     0    13    13     0     8    0
pfrule     1344      31    0       23     2     1     1     2     0     8    0
rttmr      136        7    0        7     5     5     0     1     0     8    0
art_heap8  4096       4    0        0     4     0     4     4     0     8    0
art_heap4  256      972    0      537    33     4    29    31     0     8    0
art_table   32      976    0      537     4     0     4     4     0     8    0
art_node    16      214    0      126     1     0     1     1     0     8    0
sysvmsgpl   40       20    0       14     1     0     1     1     0     8    0
semapl     112      152    0      142     1     0     1     1     0     8    0
shmpl      112       41    0        3     2     0     2     2     0     8    0
dirhash    1024      62    0       45     3     0     3     3     0     8    0
dino2pl    256     6937    0     5428    96     0    96    96     0     8    0
ffsino     288     6937    0     5428   109     0   109   109     0     8    0
nchpl      144    10824    0     9117    64     0    64    64     0     8    0
rtmask      32       19    0       19     6     5     1     1     0     8    1
uvmvnodes   80     5926    0        0   121     0   121   121     0     8    0
vnodes     216     5926    0        0   330     0   330   330     0     8    0
namei      1024   39882    0    39882     3     2     1     1     0     8    1
percpumem   16      224    0      173     1     0     1     1     0     8    0
kstatmem   264      192    0      160     3     0     3     3     0     8    0
acpiwqpl    32        4    0        4     1     0     1     1     1     8    1
scsiplug    72        9    0        9     5     5     0     1     0     8    0
scxspl     216    50553    0    50552    18    17     1     8     1     8    0
plimitpl   152      639    0      620     1     0     1     1     0     8    0
sigapl     424     3101    0     3050    10     2     8     9     0     8    0
knotepl    120      620    0        0    18     0    18    18     0     8    0
kqueuepl   224     1024    0     1012    11     9     2     5     0     8    1
pipepl     336      442    0      415     5     0     5     5     0     8    2
fdescpl    520     3054    0     3023     3     0     3     3     0     8    0
filepl     160    23665    0    23432    46    32    14    20     0     8    2
lockfpl    104     1117    0     1115     3     2     1     2     0     8    0
lockfspl    48      444    0      442     1     0     1     1     0     8    0
sessionpl  144       43    0       34     1     0     1     1     0     8    0
pgrppl      48      161    0      144     1     0     1     1     0     8    0
ucredpl    104     4217    0     4200     1     0     1     1     0     8    0
zombiepl   144     3050    0     3050     3     2     1     1     0     8    1
processpl  1240    3101    0     3050     7     1     6     6     0     8    0
procpl     656     7399    0     7336     8     2     6     8     0     8    0
srpgc       96        8    0        8     3     3     0     1     0     8    0
sosppl     168        8    0        8     6     6     0     1     0     8    0
sockpl     728     6374    0     6333    57    48     9    19     0     8    2
mcl64k     65536      7    0        0     1     0     1     1     0     8    0
mcl16k     16384      3    0        0     1     0     1     1     0     8    0
mcl12k     12288      1    0        0     1     0     1     1     0     8    0
mcl9k      9216       1    0        0     1     0     1     1     0     8    0
mcl8k      8192       3    0        0     1     0     1     1     0     8    0
mcl4k      4096     122    0        0    15     0    15    15     0     8    0
mcl2k      2048      34    0        0     4     0     4     4     0     8    0
mtagpl      96       26    0        0     1     0     1     1     0     8    0
mbufpl     256      297    0        0    17     0    17    17     0     8    0
bufpl      280    19579    0    13437   439     0   439   439     0     8    0
anonpl      32    14984    0        0   122     1   121   121     0   246    0
amapchunkpl 152  104117    0   103458    83    52    31    51     0   158    3
amappl16   200     9178    0     9057    86    67    19    32     0     8    6
amappl15   192        7    0        7     1     1     0     1     0     8    0
amappl14   184      154    0      142     1     0     1     1     0     8    0
amappl13   176        8    0        8     1     1     0     1     0     8    0
amappl12   168     3762    0     3731     3     1     2     2     0     8    0
amappl11   160       94    0       79     1     0     1     1     0     8    0
amappl10   152       15    0       14     2     1     1     1     0     8    0
amappl9    144      247    0      245     1     0     1     1     0     8    0
amappl8    136       34    0       31     1     0     1     1     0     8    0
amappl7    128      140    0      127     1     0     1     1     0     8    0
amappl6    120      250    0      245     1     0     1     1     0     8    0
amappl5    112      156    0      146     1     0     1     1     0     8    0
amappl4    104      411    0      391     1     0     1     1     0     8    0
amappl3     96    18998    0    18882     5     1     4     4     0     8    0
amappl2     88      827    0      764     2     0     2     2     0     8    0
amappl1     80    22806    0    22203    16     1    15    15     0     8    0
amappl      88    25329    0    25146     5     0     5     5     0    92    0
dma65536   65536      2    0        2     1     1     0     1     0     8    0
dma32768   32768      1    0        1     1     0     1     1     0     8    1
dma4096    4096       2    0        2     2     2     0     1     0     8    0
dma1024    1024       1    0        0     1     0     1     1     0     8    0
dma256     256       10    0       10     3     3     0     1     0     8    0
dma128     128      257    0      257     5     5     0     1     0     8    0
dma64       64        7    0        7     2     2     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       22    0       21     1     0     1     1     0     8    0
aobjpl      72       43    0        3     1     0     1     1     0     8    0
uaddrrnd    24     3054    0     3023     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24     3054    0     3023     1     0     1     1     0     8    0
vmmpekpl   168    24086    0    24032     3     0     3     3     0     8    0
vmmpepl    168   197517    0   195397   145    41   104   115     0   357    2
vmsppl     480     3053    0     3023     7     2     5     5     0     8    0
rwobjpl     72    56420    0    49400   138     3   135   135     0     8    0
pdppl      4096    6115    0     6046   149    78    71    83     0     8    2
pvpl        32    21818    0        0   178     2   176   176     0   265    0
pmappl     256     3053    0     3023     3     0     3     3     0     8    0
extentpl    40       45    0       27     1     0     1     1     0     8    0
phpool     112      484    0      118    11     0    11    11     0     8    0
ddb{0}> machine ddbcpu 0
Invalid cpu 0
ddb{0}> trace
db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
witness_checkorder(ffffffff83944b58,9,0) at witness_checkorder+0x10fb sys/kern/subr_witness.c:-1
mtx_enter(ffffffff83944b48) at mtx_enter+0x4a sys/kern/kern_lock.c:257
sleep_setup(ffff8000299dee50,220,ffffffff833af742) at sleep_setup+0xc6 sys/kern/kern_synch.c:302
msleep_nsec(ffff8000299dee50,ffff8000299dedf0,220,ffffffff833af742,ffffffffffffffff) at msleep_nsec+0x119 sys/kern/kern_synch.c:217
dt_ioctl_record_stop(ffff80000149a000) at dt_ioctl_record_stop+0xbc sys/dev/dt/dt_dev.c:576
dtclose(11e5f,81,2000,ffff8000fffe82b8) at dtclose+0x105 dt_pcb_purge sys/dev/dt/dt_dev.c:-1 [inline]
dtclose(11e5f,81,2000,ffff8000fffe82b8) at dtclose+0x105 sys/dev/dt/dt_dev.c:232
spec_close(ffff80002a3060e0) at spec_close+0x45f sys/kern/spec_vnops.c:-1
VOP_CLOSE(fffffd805e2506d8,81,fffffd80097fb2d8,ffff8000fffe82b8) at VOP_CLOSE+0x133 sys/kern/vfs_vops.c:156
vn_closefile(fffffd805cc3a088,ffff8000fffe82b8) at vn_closefile+0x12b vn_close sys/kern/vfs_vnops.c:292 [inline]
vn_closefile(fffffd805cc3a088,ffff8000fffe82b8) at vn_closefile+0x12b sys/kern/vfs_vnops.c:615
fdrop(fffffd805cc3a088,ffff8000fffe82b8) at fdrop+0x126 sys/kern/kern_descrip.c:1267
closef(fffffd805cc3a088,ffff8000fffe82b8) at closef+0x192 sys/kern/kern_descrip.c:1251
syscall(ffff80002a306340) at syscall+0xbc6 mi_syscall sys/sys/syscall_mi.h:176 [inline]
syscall(ffff80002a306340) at syscall+0xbc6 sys/arch/amd64/amd64/trap.c:579
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0xa0c37359210, count: -14
ddb{0}> machine ddbcpu 1
Stopped at      x86_ipi_db+0x27:        addq    $0x8,%rsp
ddb{1}> trace
x86_ipi_db(ffff8000299ddff0) at x86_ipi_db+0x27 sys/arch/amd64/amd64/db_interface.c:394
x86_ipi_handler() at x86_ipi_handler+0xd9 sys/arch/amd64/amd64/ipi.c:106
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
__sanitizer_cov_trace_pc() at __sanitizer_cov_trace_pc+0x1a kd_curproc sys/dev/kcov.c:580 [inline]
__sanitizer_cov_trace_pc() at __sanitizer_cov_trace_pc+0x1a sys/dev/kcov.c:153
mtx_enter(ffff8000299dedf0) at mtx_enter+0xf5 sys/kern/kern_lock.c:261
clockintr_dispatch(ffff80002a228cd0) at clockintr_dispatch+0x3b4 sys/kern/kern_clockintr.c:215
lapic_clockintr(0,0) at lapic_clockintr+0x51 sys/arch/amd64/amd64/lapic.c:489
Xresume_lapic_ltimer() at Xresume_lapic_ltimer+0x2a
acpicpu_idle() at acpicpu_idle+0x41e sys/dev/acpi/acpicpu_x86.c:1218
sched_idle(ffff8000299ddff0) at sched_idle+0x4d8 sys/kern/kern_sched.c:191
end trace frame: 0x0, count: -10

Crashes (12):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/06/23 15:56 openbsd bfd2b5c81cd0 d6cdfb8a .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/06/21 02:58 openbsd f62159b6a996 d6cdfb8a .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/06/20 03:26 openbsd c7ecc8a6a85e ed3e87f7 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/06/15 19:21 openbsd 1a751a5f99c1 5f4b362d .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/06/08 08:16 openbsd 44b12aca1c11 4826c28e .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/06/01 13:48 openbsd ae460101462c 3d2f584d .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/05/27 16:45 openbsd 34392d87e208 874a1386 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/05/20 01:47 openbsd 03d4002980cf b84f0537 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/05/19 17:25 openbsd 2fc058fc36e6 27f68995 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/05/18 13:54 openbsd 6decc3d5f4d6 f41472b0 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/05/09 20:18 openbsd 0dc0a232ede0 77908e5f .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
2025/04/22 03:22 openbsd e8eeea37c7e6 552876f8 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore witness: reversal: &cq->cq_mtx &sched_lock
* Struck through repros no longer work on HEAD.