syzbot


witness: reversal: vmmaplk inode

Status: fixed on 2019/12/18 19:18
Reported-by: syzbot+0ab93519af8f3dbfb136@syzkaller.appspotmail.com
Fix commit: Use separate rwlock initializations for userland ("vmspace") and kernel maps. This lets witness know that these really are different classes avoiding false positives when detecting lock order reversals.
First crash: 1341d, last: 1013d
duplicates (1):
Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
uvm_fault.c:LINE C 260 1341d 1342d 0/3 closed as dup on 2019/01/25 07:22
similar bugs (2):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
openbsd witness: reversal: vmmaplk inode (3) C 210 1d18h 132d 0/3 upstream: reported C repro on 2022/05/18 16:04
openbsd witness: reversal: vmmaplk inode (2) C 46771 699d 1013d 0/3 closed as invalid on 2020/10/27 19:49

Sample crash report:
login: witness: lock order reversal:
 1st 0xfffffd806e7b68c0 vmmaplk (&map->lock)
 2nd 0xfffffd807bbd65f8 inode (&ip->i_lock)
lock order "&ip->i_lock"(rrwlock) -> "&map->lock"(rwlock) first seen at:
#0  witness_checkorder+0x69c witness_lock_order_check sys/kern/subr_witness.c:2441 [inline]
#0  witness_checkorder+0x69c sys/kern/subr_witness.c:872
#1  rw_enter+0xd4
#2  vm_map_lock_ln+0x10d sys/uvm/uvm_map.c:5387
#3  uvm_map+0x2da sys/uvm/uvm_map.c:1226
#4  km_alloc+0x18b sys/uvm/uvm_km.c:908
#5  pool_multi_alloc_ni+0xd0 sys/kern/subr_pool.c:1692
#6  pool_p_alloc+0x74 pool_allocator_alloc sys/kern/subr_pool.c:1606 [inline]
#6  pool_p_alloc+0x74 sys/kern/subr_pool.c:927
#7  pool_do_get+0x10f sys/kern/subr_pool.c:712
#8  pool_get+0xeb sys/kern/subr_pool.c:581
#9  ufsdirhash_build+0x3ea sys/ufs/ufs/ufs_dirhash.c:180
#10 ufs_lookup+0x2a1 sys/ufs/ufs/ufs_lookup.c:200
#11 VOP_LOOKUP+0x5b sys/kern/vfs_vops.c:91
#12 vfs_lookup+0x7a6 sys/kern/vfs_lookup.c:568
#13 namei+0x63c sys/kern/vfs_lookup.c:249
#14 start_init+0xc9 check_console sys/kern/init_main.c:596 [inline]
#14 start_init+0xc9 sys/kern/init_main.c:636
#15 proc_trampoline+0x1c
lock order "&map->lock"(rwlock) -> "&ip->i_lock"(rrwlock) first seen at:
#0  witness_checkorder+0x69c witness_lock_order_check sys/kern/subr_witness.c:2441 [inline]
#0  witness_checkorder+0x69c sys/kern/subr_witness.c:872
#1  rw_enter+0xd4
#2  rrw_enter+0x88 sys/kern/kern_rwlock.c:453
#3  VOP_LOCK+0xf0 sys/kern/vfs_vops.c:615
#4  vn_lock+0x81 sys/kern/vfs_vnops.c:571
#5  uvn_io+0x2c8 sys/uvm/uvm_vnode.c:1189
#6  uvn_get+0x226 sys/uvm/uvm_vnode.c:1049
#7  uvm_fault+0x11cc sys/uvm/uvm_fault.c:1023
#8  uvm_fault_wire+0x70 sys/uvm/uvm_fault.c:1295
#9  uvm_map_pageable_wire+0x2dd sys/uvm/uvm_map.c:2349
#10 sys_mlock+0x185 sys/uvm/uvm_mmap.c:765
#11 syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
#11 syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
#12 Xsyscall+0x128
Stopped at      db_enter+0x18:  addq    $0x8,%rsp
ddb{0}> 
ddb{0}> set $lines = 0
ddb{0}> set $maxwidth = 0
ddb{0}> show panic
the kernel did not panic
ddb{0}> trace
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:398
witness_checkorder(fffffd807bbd65f8,9,0) at witness_checkorder+0x1009 witness_debugger sys/kern/subr_witness.c:2504 [inline]
witness_checkorder(fffffd807bbd65f8,9,0) at witness_checkorder+0x1009 sys/kern/subr_witness.c:1086
rw_enter(fffffd807bbd65e8,81) at rw_enter+0xd4
rrw_enter(fffffd807bbd65e8,81) at rrw_enter+0x88 sys/kern/kern_rwlock.c:453
VOP_LOCK(fffffd807bbd7020,81) at VOP_LOCK+0xf0 sys/kern/vfs_vops.c:615
vn_lock(fffffd807bbd7020,81) at vn_lock+0x81 sys/kern/vfs_vnops.c:571
uvn_io(fffffd806dc8e488,ffff800020b13598,1,2,0) at uvn_io+0x2c8 sys/uvm/uvm_vnode.c:1189
uvn_get(fffffd806dc8e488,0,ffff800020b137f0,ffff800020b13784,0,1) at uvn_get+0x226 sys/uvm/uvm_vnode.c:1049
uvm_fault(fffffd806e7b68a8,20002000,2,1) at uvm_fault+0x11cc sys/uvm/uvm_fault.c:1023
uvm_fault_wire(fffffd806e7b68a8,20002000,20005000,1) at uvm_fault_wire+0x70 sys/uvm/uvm_fault.c:1295
uvm_map_pageable_wire(fffffd806e7b68a8,fffffd806f0d5ee8,fffffd806f0d5d98,0,ffffffff81697ed7,0) at uvm_map_pageable_wire+0x2dd sys/uvm/uvm_map.c:2349
sys_mlock(ffff800020ad0010,ffff800020b139e8,ffff800020b13a30) at sys_mlock+0x185 sys/uvm/uvm_mmap.c:765
syscall(ffff800020b13ab0) at syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
syscall(ffff800020b13ab0) at syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7fffff8f80, count: -14
ddb{0}> show registers
rdi                              0x3
rsi               0xffffffff82481ba0    __sancov_gen_cov_switch_values.129
rbp               0xffff800020b13210
rbx                              0x3
rdx                             0x8b
rcx                              0x3
rax                              0x3
r8                0xffffffff81695cd2    witness_checkorder+0xfe2
r9                               0x5
r10               0x24d4ffb43c64efd7
r11               0x70357cb678c9c64a
r12               0xfffffd80029761c0
r13                                0
r14               0xffffffff825a94d0    w_lodata+0x4f3c0
r15               0xffffffff825af1a0    w_lodata+0x55090
rip               0xffffffff81dfc018    db_enter+0x18
cs                               0x8
rflags                         0x246
rsp               0xffff800020b13200
ss                              0x10
db_enter+0x18:  addq    $0x8,%rsp
ddb{0}> show proc
PROC (syz-executor3123) pid=289160 stat=onproc
    flags process=2<EXEC> proc=0
    pri=50, usrpri=50, nice=20
    forw=0xffffffffffffffff, list=0xffff800020ad1158,0xffffffff8262c650
    process=0xffff800020a9b190 user=0xffff800020b0e000, vmspace=0xfffffd806e7b68a8
    estcpu=0, cpticks=1, pctcpu=0.0
    user=0, sys=0, intr=0
ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
*93634  289160  48500      0  7         0x2                syz-executor3123
 48500  226038  42850      0  3    0x10008a  pause         ksh
 42850  438301  66890      0  3        0x92  select        sshd
  5978  423571      1      0  3    0x100083  ttyin         getty
 66890   40837      1      0  3        0x80  select        sshd
 24355  409436   9850     74  3    0x100092  bpf           pflogd
  9850  153672      1      0  3        0x80  netio         pflogd
 98776  213776  22504     73  3    0x100090  kqread        syslogd
 22504  231816      1      0  3    0x100082  netio         syslogd
 19784  140010      1     77  3    0x100090  poll          dhclient
 19335  492206      1      0  3        0x80  poll          dhclient
 90723   91973      0      0  3     0x14200  pgzero        zerothread
 13340  353420      0      0  3     0x14200  aiodoned      aiodoned
  8164  385459      0      0  3     0x14200  syncer        update
 56747  459072      0      0  3     0x14200  cleaner       cleaner
 65553  376222      0      0  3     0x14200  reaper        reaper
 98989   72977      0      0  3     0x14200  pgdaemon      pagedaemon
 71336  189637      0      0  3     0x14200  bored         crynlk
 88357  398910      0      0  3     0x14200  bored         crypto
 70987   79073      0      0  3  0x40014200  acpi0         acpi0
 69614  219293      0      0  7  0x40014200                idle1
 27410  432895      0      0  3     0x14200  bored         softnet
 66686  524027      0      0  3     0x14200  bored         systqmp
 38505  297139      0      0  3     0x14200  bored         systq
 73210  423173      0      0  3  0x40014200  bored         softclock
  2798  113912      0      0  3  0x40014200                idle0
 24400  496656      0      0  3     0x14200  bored         smr
     1  103949      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{0}> show all locks
Process 93634 (syz-executor3123) thread 0xffff800020ad0010 (289160)
shared rwlock vmmaplk r = 0 (0xfffffd806e7b68c0)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  rw_enter+0x453 sys/kern/kern_rwlock.c:309
#2  vm_map_lock_ln+0x10d sys/uvm/uvm_map.c:5387
#3  uvm_map_pageable+0x11d sys/uvm/uvm_map.c:2445
#4  sys_mlock+0x185 sys/uvm/uvm_mmap.c:765
#5  syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
#5  syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
#6  Xsyscall+0x128
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff82626000)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  syscall+0x400 mi_syscall sys/sys/syscall_mi.h:83 [inline]
#1  syscall+0x400 sys/arch/amd64/amd64/trap.c:555
#2  Xsyscall+0x128
ddb{0}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf  9461   6393K    6394K  78643K     10552        0
            pcb    13      8K       8K  78643K        13        0
         rtable    61      2K       2K  78643K       127        0
         ifaddr    29      8K       8K  78643K        30        0
       counters    39     33K      33K  78643K        39        0
       ioctlops     0      0K       4K  78643K      1467        0
          mount     1      1K       1K  78643K         1        0
         vnodes  1181     74K      74K  78643K      1186        0
      UFS quota     1     32K      32K  78643K         1        0
      UFS mount     5     36K      36K  78643K         5        0
            shm     2      1K       1K  78643K         2        0
         VM map     2      1K       1K  78643K         2        0
            sem     2      0K       0K  78643K         2        0
        dirhash    12      2K       2K  78643K        12        0
           ACPI  1809    196K     290K  78643K     12766        0
      file desc     1      0K       0K  78643K         1        0
           proc    59     63K      71K  78643K       358        0
    NFS srvsock     1      0K       0K  78643K         1        0
     NFS daemon     1     16K      16K  78643K         1        0
       in_multi    11      0K       0K  78643K        11        0
    ether_multi     1      0K       0K  78643K         1        0
    ISOFS mount     1     32K      32K  78643K         1        0
  MSDOSFS mount     1     16K      16K  78643K         1        0
           ttys    18     79K      79K  78643K        18        0
           exec     0      0K       1K  78643K       177        0
        pagedep     1      8K       8K  78643K         1        0
       inodedep     1     32K      32K  78643K         1        0
         newblk     1      0K       0K  78643K         1        0
        VM swap     7     26K      26K  78643K         7        0
       UVM amap    67      3K       4K  78643K       816        0
       UVM aobj     2      2K       2K  78643K         2        0
        memdesc     1      4K       4K  78643K         1        0
    crypto data     1      1K       1K  78643K         1        0
            NDP     4      0K       0K  78643K         4        0
           temp    27   3003K    3067K  78643K      1978        0
      SYN cache     2     16K      16K  78643K         2        0
ddb{0}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
arp         64        2    0        0     1     0     1     1     0     8    0
plcache    128       20    0        0     1     0     1     1     0     8    0
rtpcb       80       15    0       13     1     0     1     1     0     8    0
rtentry    112       23    0        1     1     0     1     1     0     8    0
unpcb      120       29    0       19     1     0     1     1     0     8    0
syncache   264        5    0        5     1     0     1     1     0     8    1
tcpcb      544        8    0        5     1     0     1     1     0     8    0
inpcb      280       29    0       23     1     0     1     1     0     8    0
pfosfp      40      846    0      423     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfstitem    24        8    0        0     1     0     1     1     0     8    0
pfstkey    112        8    0        0     1     0     1     1     0     8    0
pfstate    328        8    0        0     1     0     1     1     0     8    0
pfrule     1360      21    0       16     2     1     1     2     0     8    0
art_heap8  4096       1    0        0     1     0     1     1     0     8    0
art_heap4  256       96    0        0     6     0     6     6     0     8    0
art_table   32       97    0        0     1     0     1     1     0     8    0
art_node    16       22    0        2     1     0     1     1     0     8    0
dirhash    1024      17    0        0     3     0     3     3     0     8    0
dino1pl    128     1403    0       16    45     0    45    45     0     8    0
ffsino     272     1403    0       16    93     0    93    93     0     8    0
nchpl      144     1578    0       34    58     0    58    58     0     8    0
uvmvnodes   72     1413    0        0    26     0    26    26     0     8    0
vnodes     208     1413    0        0    75     0    75    75     0     8    0
namei      1024    3774    0     3774     1     0     1     1     0     8    1
percpumem   16       30    0        0     1     0     1     1     0     8    0
scxspl     192     2524    0     2524     2     1     1     2     0     8    1
plimitpl   152       14    0        8     1     0     1     1     0     8    0
sigapl     432      204    0      192     2     0     2     2     0     8    0
knotepl    112        5    0        0     1     0     1     1     0     8    0
kqueuepl   104        1    0        0     1     0     1     1     0     8    0
pipepl     160      128    0      121     1     0     1     1     0     8    0
fdescpl    488      205    0      192     3     0     3     3     0     8    0
filepl     152      997    0      948     2     0     2     2     0     8    0
lockfpl    104        5    0        4     1     0     1     1     0     8    0
lockfspl    48        3    0        2     1     0     1     1     0     8    0
sessionpl  112       18    0        9     1     0     1     1     0     8    0
pgrppl      48       18    0        9     1     0     1     1     0     8    0
ucredpl     96       52    0       43     1     0     1     1     0     8    0
zombiepl   144      192    0      192     1     0     1     1     0     8    1
processpl  896      220    0      192     4     0     4     4     0     8    0
procpl     632      220    0      192     3     0     3     3     0     8    0
sockpl     384       73    0       55     2     0     2     2     0     8    0
mcl4k      4096       3    0        0     1     0     1     1     0     8    0
mcl2k      2048      56    0        0     7     0     7     7     0     8    0
mtagpl      80        1    0        0     1     0     1     1     0     8    0
mbufpl     256       81    0        0     5     0     5     5     0     8    0
bufpl      280     2200    0      274   138     0   138   138     0     8    0
anonpl      16    19206    0    17967     7     1     6     7     0   125    0
amapchunkpl 152     522    0      485     2     0     2     2     0   158    0
amappl16   192       30    0       28     1     0     1     1     0     8    0
amappl15   184       50    0       46     1     0     1     1     0     8    0
amappl14   176       15    0       14     1     0     1     1     0     8    0
amappl12   160        3    0        3     1     0     1     1     0     8    1
amappl11   152       51    0       35     1     0     1     1     0     8    0
amappl10   144        5    0        2     1     0     1     1     0     8    0
amappl9    136      419    0      418     1     0     1     1     0     8    0
amappl8    128       68    0       64     1     0     1     1     0     8    0
amappl7    120       76    0       68     1     0     1     1     0     8    0
amappl6    112       47    0       45     1     0     1     1     0     8    0
amappl5    104      126    0      112     1     0     1     1     0     8    0
amappl4     96      459    0      434     1     0     1     1     0     8    0
amappl3     88      109    0      102     1     0     1     1     0     8    0
amappl2     80      820    0      765     2     0     2     2     0     8    0
amappl1     72    14029    0    13606    15     5    10    15     0     8    0
amappl      80      408    0      387     1     0     1     1     0    84    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma256     256        6    0        6     1     1     0     1     0     8    0
dma128     128      253    0      253     1     1     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       18    0       17     1     0     1     1     0     8    0
aobjpl      64        1    0        0     1     0     1     1     0     8    0
uaddrrnd    24      205    0      192     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24      205    0      192     1     0     1     1     0     8    0
vmmpekpl   168     5344    0     5323     1     0     1     1     0     8    0
vmmpepl    168    29611    0    28762    49     8    41    49     0   357    1
vmsppl     368      204    0      192     2     0     2     2     0     8    0
pdppl      4096     417    0      384     5     0     5     5     0     8    0
pvpl        32    79469    0    76468    29     0    29    29     0   265    2
pmappl     232      204    0      192     1     0     1     1     0     8    0
extentpl    40       46    0       29     1     0     1     1     0     8    0
phpool     112      129    0        3     4     0     4     4     0     8    0

Crashes (103124):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-openbsd-multicore 2019/12/03 22:50 openbsd afd2bec52a56 4b0a22b9 .config log report syz C
ci-openbsd-multicore 2019/10/31 13:44 openbsd 6f77cf22e25c a41ca8fa .config log report syz C
ci-openbsd-multicore 2019/10/03 20:43 openbsd 0a65a5060995 fc17ba49 .config log report syz C
ci-openbsd-setuid 2019/06/27 05:31 openbsd ab90d8c35e80 7509bf36 .config log report syz C
ci-openbsd-setuid 2019/06/05 07:25 openbsd 759317bf8c07 bfb4a51e .config log report syz C
ci-openbsd-multicore 2019/05/01 09:24 openbsd 4d23fe49f8a8 618456b4 .config log report syz C
ci-openbsd-multicore 2019/04/17 00:01 openbsd 5c9c6d70fcb3 505ab413 .config log report syz C
ci-openbsd-multicore 2019/04/11 12:10 openbsd f3736a26b567 e955ac50 .config log report syz C
ci-openbsd-setuid 2019/04/03 00:09 openbsd 5bc82f070685 dfd3394d .config log report syz C
ci-openbsd-multicore 2019/03/27 12:20 openbsd 690d147bdb68 70d776a2 .config log report syz C
ci-openbsd-multicore 2019/03/19 18:41 openbsd a5f4ee4e3f8f e4549234 .config log report syz C
ci-openbsd-setuid 2019/01/25 07:35 openbsd 6f9e9be94f66 b5d78bce .config log report syz C
ci-openbsd-setuid 2019/12/18 18:13 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 16:58 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 15:54 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 15:52 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 14:42 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 13:38 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 13:11 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 12:04 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 11:07 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 10:07 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 08:43 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-multicore 2019/12/18 07:35 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 07:29 openbsd 52ed6215f160 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 06:14 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 05:08 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 04:06 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 03:26 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-multicore 2019/12/18 02:20 openbsd e749f6f89611 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 01:20 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/18 00:36 openbsd d0d1e83d5ece 64ca0a37 .config log report
ci-openbsd-multicore 2019/12/17 23:35 openbsd e749f6f89611 64ca0a37 .config log report
ci-openbsd-setuid 2019/12/17 22:02 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-multicore 2019/12/17 21:42 openbsd e749f6f89611 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 20:20 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 20:12 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 18:53 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 17:53 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 17:14 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 16:12 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 15:32 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 14:28 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 13:18 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 12:29 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 11:20 openbsd d0d1e83d5ece 7b9de469 .config log report
ci-openbsd-setuid 2019/12/17 10:05 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 09:40 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 08:36 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 08:11 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 07:09 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-multicore 2019/12/17 06:22 openbsd e749f6f89611 d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 06:20 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-setuid 2019/12/17 05:19 openbsd d0d1e83d5ece d13d7958 .config log report
ci-openbsd-pf 2019/03/05 16:07 openbsd 41e5a6229bec 3419571c .config log report
ci-openbsd-setuid 2019/01/25 07:15 openbsd 6f9e9be94f66 b5d78bce .config log report
* Struck through repros no longer work on HEAD.