syzbot


panic: pmap_unwire: invalid PDE

Status: upstream: reported syz repro on 2022/08/11 20:34
Reported-by: syzbot+922c2ec14d832dfa1311@syzkaller.appspotmail.com
First crash: 832d, last: 189d
Last patch testing requests (10)
Created Duration User Patch Repo Result
2024/11/12 06:13 1m retest repro openbsd error
2024/09/03 06:09 1m retest repro openbsd error
2024/07/25 03:12 15m retest repro openbsd OK log
2024/06/25 06:03 3m retest repro openbsd error
2024/05/16 02:49 16m retest repro openbsd report log
2024/04/11 05:08 0m retest repro openbsd error
2024/03/07 02:33 16m retest repro openbsd log
2024/02/01 04:58 9m retest repro openbsd error
2023/11/23 04:48 8m retest repro openbsd error
2023/09/14 04:33 8m retest repro openbsd error

Sample crash report:
panic: pmap_unwire: invalid PDE
Stopped at      db_enter+0x18:  addq    $0x8,%rsp
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
*384095  92143      0           0  0x4000000    0K syz-executor.7
   5954  69064      0           0  0x4000000    1  syz-executor.3
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:437
panic(ffffffff826560a8) at panic+0x177 sys/kern/subr_prf.c:198
pmap_unwire(fffffd8069898108,20000000) at pmap_unwire+0x1d9 sys/arch/amd64/amd64/pmap.c:2189
uvm_fault_unwire_locked(fffffd80701ba748,20000000,20002000) at uvm_fault_unwire_locked+0x1fc sys/uvm/uvm_fault.c:1678
uvm_map_pageable(fffffd80701ba748,20000000,20002000,1,0) at uvm_map_pageable+0x597 uvm_map_pageable_pgon sys/uvm/uvm_map.c:2089 [inline]
uvm_map_pageable(fffffd80701ba748,20000000,20002000,1,0) at uvm_map_pageable+0x597 sys/uvm/uvm_map.c:2330
sys_munlock(ffff800021403508,ffff800021488e58,ffff800021488ea0) at sys_munlock+0x100 sys/uvm/uvm_mmap.c:868
syscall(ffff800021488f20) at syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
syscall(ffff800021488f20) at syscall+0x435 sys/arch/amd64/amd64/trap.c:585
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x66615977a40, count: 7
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{0}> 
ddb{0}> set $lines = 0
ddb{0}> set $maxwidth = 0
ddb{0}> show panic
*cpu0: pmap_unwire: invalid PDE
ddb{0}> trace
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:437
panic(ffffffff826560a8) at panic+0x177 sys/kern/subr_prf.c:198
pmap_unwire(fffffd8069898108,20000000) at pmap_unwire+0x1d9 sys/arch/amd64/amd64/pmap.c:2189
uvm_fault_unwire_locked(fffffd80701ba748,20000000,20002000) at uvm_fault_unwire_locked+0x1fc sys/uvm/uvm_fault.c:1678
uvm_map_pageable(fffffd80701ba748,20000000,20002000,1,0) at uvm_map_pageable+0x597 uvm_map_pageable_pgon sys/uvm/uvm_map.c:2089 [inline]
uvm_map_pageable(fffffd80701ba748,20000000,20002000,1,0) at uvm_map_pageable+0x597 sys/uvm/uvm_map.c:2330
sys_munlock(ffff800021403508,ffff800021488e58,ffff800021488ea0) at sys_munlock+0x100 sys/uvm/uvm_mmap.c:868
syscall(ffff800021488f20) at syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
syscall(ffff800021488f20) at syscall+0x435 sys/arch/amd64/amd64/trap.c:585
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x66615977a40, count: -8
ddb{0}> show registers
rdi                                0
rsi                              0x1
rbp               0xffff800021488bc0
rbx               0xffffffff82974b8f    cpu_info_full_primary+0x2b8f
rdx                            0x3fd
rcx                                0
rax                             0x20
r8                 0x101010101010101
r9                0x8080808080808080
r10               0x53f60bafbcae6c1a
r11               0xee0748f6c31234f4
r12               0xffffffff82974990    cpu_info_full_primary+0x2990
r13                                0
r14                                0
r15                              0x1
rip               0xffffffff81a07d98    db_enter+0x18
cs                               0x8
rflags                         0x246
rsp               0xffff800021488bb0
ss                              0x10
db_enter+0x18:  addq    $0x8,%rsp
ddb{0}> show proc
PROC (syz-executor.7) pid=384095 stat=onproc
    flags process=0 proc=4000000<THREAD>
    pri=32, usrpri=76, nice=20
    forw=0xffffffffffffffff, list=0xffff800021301ce0,0xffffffff82b66800
    process=0xffff8000ffff0018 user=0xffff800021483000, vmspace=0xfffffd80701ba748
    estcpu=36, cpticks=0, pctcpu=0.0
    user=0, sys=0, intr=0
ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 92143  494755  58860      0  3           0  vmmaplk       syz-executor.7
 92143  290819  58860      0  2   0x4000000                syz-executor.7
*92143  384095  58860      0  7   0x4000000                syz-executor.7
  2736  252222   9354      0  2           0                syz-executor.5
  2736  257796   9354      0  3   0x4000080  fsleep        syz-executor.5
  2736  305853   9354      0  3   0x4000080  fsleep        syz-executor.5
 68440   68340  84422      0  2           0                syz-executor.6
 68440  519315  84422      0  3   0x4000080  fsleep        syz-executor.6
 68440  462964  84422      0  3   0x4000080  fsleep        syz-executor.6
 68440  168745  84422      0  2   0x4000000                syz-executor.6
 94265  466706  10818      0  2           0                syz-executor.0
 94265   90082  10818      0  3   0x4000080  fsleep        syz-executor.0
 94265  228439  10818      0  3   0x4000080  fsleep        syz-executor.0
  4181  281154  89419      0  2           0                syz-executor.2
  4181  122982  89419      0  3   0x4000080  fsleep        syz-executor.2
  4181  524016  89419      0  2   0x4000000                syz-executor.2
 20313  333986  69580      0  2           0                syz-executor.1
 20313  168492  69580      0  3   0x4000080  fsleep        syz-executor.1
 20313  263845  69580      0  2   0x4000000                syz-executor.1
 69064  432904  48094      0  2           0                syz-executor.3
 69064    5954  48094      0  7   0x4000000                syz-executor.3
 69064  232736  48094      0  3   0x4000080  fsleep        syz-executor.3
 48094  162502   9344      0  3        0x82  nanoslp       syz-executor.3
 50477   93129   9344      0  3        0x82  nanoslp       syz-executor.4
  9354  165625   9344      0  3        0x82  nanoslp       syz-executor.5
 84422  373408   9344      0  3        0x82  nanoslp       syz-executor.6
 58860  298973   9344      0  3        0x82  nanoslp       syz-executor.7
 10818  294965   9344      0  3        0x82  nanoslp       syz-executor.0
 89419  342884   9344      0  2         0x2                syz-executor.2
 69580   75068   9344      0  3        0x82  nanoslp       syz-executor.1
  9344  330067  33188      0  3        0x82  thrsleep      syz-execprog
  9344  407351  33188      0  3   0x4000082  thrsleep      syz-execprog
  9344  114270  33188      0  3   0x4000082  wait          syz-execprog
  9344  125435  33188      0  3   0x4000082  wait          syz-execprog
  9344  307537  33188      0  3   0x4000082  wait          syz-execprog
  9344  123726  33188      0  3   0x4000082  wait          syz-execprog
  9344   21853  33188      0  3   0x4000082  thrsleep      syz-execprog
  9344   68083  33188      0  3   0x4000082  wait          syz-execprog
  9344  225110  33188      0  3   0x4000082  thrsleep      syz-execprog
  9344  459730  33188      0  3   0x4000082  wait          syz-execprog
  9344  206726  33188      0  3   0x4000082  kqread        syz-execprog
  9344  258188  33188      0  3   0x4000082  wait          syz-execprog
  9344  212476  33188      0  3   0x4000082  wait          syz-execprog
  9344  377186  33188      0  3   0x4000082  thrsleep      syz-execprog
  9344  360818  33188      0  3   0x4000082  thrsleep      syz-execprog
  9344  247948  33188      0  3   0x4000082  thrsleep      syz-execprog
 33188  306139  81967      0  3    0x10008a  sigsusp       ksh
 81967  238456  10582      0  3        0x9a  kqread        sshd
 40998  193937      1      0  3    0x100083  ttyin         getty
 10582  174087      1      0  3        0x88  kqread        sshd
 35253  510135  64644     74  3   0x1100092  bpf           pflogd
 64644  113101      1      0  3        0x80  netio         pflogd
 23509  405767  32411     73  3   0x1100090  kqread        syslogd
 32411   39246      1      0  3    0x100082  netio         syslogd
 48827  163000      1      0  3    0x100080  kqread        resolvd
 45926  176695  52183     77  3    0x100092  kqread        dhcpleased
 47238  344415  52183     77  3    0x100092  kqread        dhcpleased
 52183  455738      1      0  3        0x80  kqread        dhcpleased
 27663   35522      0      0  3     0x14200  bored         smr
 15602    5013      0      0  2     0x14200                zerothread
 13694  505047      0      0  3     0x14200  aiodoned      aiodoned
 77447  138870      0      0  3     0x14200  syncer        update
 28521  468836      0      0  3     0x14200  cleaner       cleaner
 76533   45402      0      0  3     0x14200  reaper        reaper
 17052  455841      0      0  3     0x14200  pgdaemon      pagedaemon
 89006  305303      0      0  3     0x14200  bored         viomb
 35499  191843      0      0  3  0x40014200  acpi0         acpi0
 31911  311589      0      0  3  0x40014200                idle1
 74700  513084      0      0  3     0x14200  bored         softnet
 25429  411299      0      0  3     0x14200  bored         softnet
 28908  266583      0      0  3     0x14200  bored         softnet
 16720  110222      0      0  3     0x14200  bored         softnet
 24344  116393      0      0  3     0x14200  bored         systqmp
  1244   75854      0      0  3     0x14200  bored         systq
 20999  316001      0      0  3  0x40014200  bored         softclock
 87658  392755      0      0  3  0x40014200                idle0
     1  207142      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{0}> show all locks
Process 92143 (syz-executor.7) thread 0xffff800021301ce0 (290819)
exclusive rrwlock inode r = 0 (0xfffffd80707ec4d8)
#0  witness_lock+0x44d
#1  rw_enter+0x3e1 sys/kern/kern_rwlock.c:310
#2  rrw_enter+0x8b sys/kern/kern_rwlock.c:465
#3  VOP_LOCK+0x87 sys/kern/vfs_vops.c:518
#4  vn_lock+0x84 sys/kern/vfs_vnops.c:564
#5  vget+0x1fc sys/kern/vfs_subr.c:676
#6  ufs_ihashget+0x121 sys/ufs/ufs/ufs_ihash.c:119
#7  ffs_vget+0x7c sys/ufs/ffs/ffs_vfsops.c:1324
#8  ufs_lookup+0x122c sys/ufs/ufs/ufs_lookup.c:582
#9  VOP_LOOKUP+0x58 sys/kern/vfs_vops.c:85
#10 vfs_lookup+0x6e5 sys/kern/vfs_lookup.c:560
#11 namei+0x36a sys/kern/vfs_lookup.c:244
#12 vn_open+0x188 sys/kern/vfs_vnops.c:107
#13 doopenat+0x26a sys/kern/vfs_syscalls.c:1127
#14 syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
#14 syscall+0x435 sys/arch/amd64/amd64/trap.c:585
#15 Xsyscall+0x128
Process 92143 (syz-executor.7) thread 0xffff800021403508 (384095)
exclusive rwlock uobjlk r = 0 (0xfffffd807e9bac20)
#0  witness_lock+0x44d
#1  rw_enter+0x3e1 sys/kern/kern_rwlock.c:310
#2  uvm_fault_unwire_locked+0x1c6 sys/uvm/uvm_fault.c:1669
#3  uvm_map_pageable+0x597 uvm_map_pageable_pgon sys/uvm/uvm_map.c:2089 [inline]
#3  uvm_map_pageable+0x597 sys/uvm/uvm_map.c:2330
#4  sys_munlock+0x100 sys/uvm/uvm_mmap.c:868
#5  syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
#5  syscall+0x435 sys/arch/amd64/amd64/trap.c:585
#6  Xsyscall+0x128
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff82a17850)
#0  witness_lock+0x44d
#1  __mp_acquire_count+0x48 sys/kern/kern_lock.c:227
#2  mi_switch+0x3bb sys/kern/sched_bsd.c:415
#3  sleep_finish+0x180 sys/kern/kern_synch.c:417
#4  rw_enter+0x35a sys/kern/kern_rwlock.c:286
#5  uvm_fault_unwire_locked+0x1c6 sys/uvm/uvm_fault.c:1669
#6  uvm_map_pageable+0x597 uvm_map_pageable_pgon sys/uvm/uvm_map.c:2089 [inline]
#6  uvm_map_pageable+0x597 sys/uvm/uvm_map.c:2330
#7  sys_munlock+0x100 sys/uvm/uvm_mmap.c:868
#8  syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
#8  syscall+0x435 sys/arch/amd64/amd64/trap.c:585
#9  Xsyscall+0x128
exclusive rwlock vmmaplk r = 0 (0xfffffd80701ba760)
#0  witness_lock+0x44d
#1  rw_enter+0x3e1 sys/kern/kern_rwlock.c:310
#2  vm_map_lock_ln+0xda sys/uvm/uvm_map.c:5276
#3  uvm_map_pageable+0x11a sys/uvm/uvm_map.c:2263
#4  sys_munlock+0x100 sys/uvm/uvm_mmap.c:868
#5  syscall+0x435 mi_syscall sys/sys/syscall_mi.h:101 [inline]
#5  syscall+0x435 sys/arch/amd64/amd64/trap.c:585
#6  Xsyscall+0x128
ddb{0}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf 10188   6474K    6474K  78643K     11278        0
            pcb    13      8K       8K  78643K        13        0
         rtable   234      6K       6K  78643K       347        0
         ifaddr    87     17K      17K  78643K        92        0
       counters    56     35K      35K  78643K        56        0
       ioctlops     0      0K       4K  78643K      1487        0
            iov     0      0K      24K  78643K       470        0
          mount     1      1K       1K  78643K         1        0
            log     0      0K       0K  78643K         4        0
         vnodes  1167     73K      73K  78643K      1180        0
      UFS quota     1     32K      32K  78643K         1        0
      UFS mount     5     36K      36K  78643K         5        0
            shm     2      1K       1K  78643K         2        0
         VM map     2      1K       1K  78643K         2        0
            sem     2      0K       0K  78643K         2        0
        dirhash    12      2K       2K  78643K        12        0
           ACPI  1697    195K     286K  78643K     12548        0
      file desc    17     61K      93K  78643K       587        0
           proc    67     91K     115K  78643K       506        0
    NFS srvsock     1      0K       0K  78643K         1        0
     NFS daemon     1     16K      16K  78643K         1        0
       in_multi    99      6K       6K  78643K        99        0
    ether_multi     1      0K       0K  78643K         1        0
    ISOFS mount     1     32K      32K  78643K         1        0
  MSDOSFS mount     1     16K      16K  78643K         1        0
           ttys    25    122K     122K  78643K        25        0
           exec     0      0K       2K  78643K       647        0
            tdb     3      0K       0K  78643K         3        0
        pagedep     1      8K       8K  78643K         1        0
       inodedep     1     32K      32K  78643K         1        0
         newblk     1      0K       0K  78643K         1        0
        VM swap     8     62K      64K  78643K        10        0
       UVM amap   231     74K      74K  78643K      5716        0
       UVM aobj     3      2K       2K  78643K         3        0
        memdesc     1      4K       4K  78643K         1        0
    crypto data     1      1K       1K  78643K         1        0
            NDP    12      0K       2K  78643K        28        0
           temp    56   4718K    4782K  78643K      4671        0
         kqueue    12     18K      18K  78643K        25        0
      SYN cache     2     16K      16K  78643K         2        0
ddb{0}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache    128       22    0        0     1     0     1     1     0     8    0
rtpcb      120       33    0       30     1     0     1     1     0     8    0
rtentry    112      111    0        1     4     0     4     4     0     8    0
unpcb      144       35    0       20     1     0     1     1     0     8    0
syncache   296        5    0        5     2     2     0     1     0     8    0
tcpcb      768        8    0        5     1     0     1     1     0     8    0
arp        120       18    0        0     1     0     1     1     0     8    0
inpcb      368       64    0       58     1     0     1     1     0     8    0
nd6         48       24    0        0     1     0     1     1     0     8    0
pfosfp      40     1428    0     1005     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfstitem    24       23    0        2     1     0     1     1     0     8    0
pfstkey    120       23    0        2     1     0     1     1     0     8    0
pfstate    336       23    0        2     2     0     2     2     0     8    0
pfrule     1360      21    0       16     2     1     1     2     0     8    0
art_heap8  4096       1    0        0     1     0     1     1     0     8    0
art_heap4  256      452    0        0    29     0    29    29     0     8    0
art_table   32      453    0        0     4     0     4     4     0     8    0
art_node    16      110    0       10     1     0     1     1     0     8    0
dirhash    1024      17    0        0     3     0     3     3     0     8    0
dino2pl    256     2382    0      937    91     0    91    91     0     8    0
ffsino     272     2382    0      937    97     0    97    97     0     8    0
nchpl      144     3538    0     1850    63     0    63    63     0     8    0
uvmvnodes   80     2392    0        0    49     0    49    49     0     8    0
vnodes     216     2392    0        0   133     0   133   133     0     8    0
namei      1024    9580    0     9580     3     1     2     2     0     8    2
percpumem   16       40    0        0     1     0     1     1     0     8    0
kstatmem   264       24    0        0     2     0     2     2     0     8    0
scxspl     216    10460    0    10460     9     1     8     8     0     8    8
plimitpl   152       25    0        9     1     0     1     1     0     8    0
sigapl     424      894    0      846     8     1     7     7     0     8    1
futexpl     64     3397    0     3388     1     0     1     1     0     8    0
knotepl    120      102    0        0     4     0     4     4     0     8    0
kqueuepl   216       21    0       13     1     0     1     1     0     8    0
pipepl     320      134    0      106     4     1     3     3     0     8    0
fdescpl    496      877    0      847     5     0     5     5     0     8    0
filepl     152     2958    0     2817     6     0     6     6     0     8    0
lockfpl    104      905    0      898     1     0     1     1     0     8    0
lockfspl    48      456    0      449     1     0     1     1     0     8    0
sessionpl  144       26    0        9     1     0     1     1     0     8    0
pgrppl      48       26    0        9     1     0     1     1     0     8    0
ucredpl    104       69    0       57     1     0     1     1     0     8    0
zombiepl   144      847    0      846     2     1     1     1     0     8    0
processpl  1064     894    0      846     4     0     4     4     0     8    0
procpl     672     1945    0     1867     8     1     7     7     0     8    0
sockpl     488      132    0      108     5     1     4     4     0     8    0
mcl8k      8192       4    0        0     1     0     1     1     0     8    0
mcl4k      4096       5    0        0     1     0     1     1     0     8    0
mcl2k      2048      81    0        0    10     0    10    10     0     8    0
mtagpl      96        2    0        0     1     0     1     1     0     8    0
mbufpl     256      222    0        0    13     0    13    13     0     8    0
bufpl      288     4889    0      135   340     0   340   340     0     8    0
anonpl      24   149442    0   142903    52     6    46    47     0   186    5
amapchunkpl 152   13118    0    12606    22     1    21    21     0   158    1
amappl16   200     1152    0     1044     6     0     6     6     0     8    0
amappl15   192      105    0       94     1     0     1     1     0     8    0
amappl14   184       16    0       13     1     0     1     1     0     8    0
amappl13   176      252    0      245     2     1     1     1     0     8    0
amappl12   168       10    0        9     2     1     1     1     0     8    0
amappl11   160      175    0      156     1     0     1     1     0     8    0
amappl10   152       77    0       73     2     1     1     1     0     8    0
amappl9    144      979    0      970     3     2     1     1     0     8    0
amappl8    136      643    0      600     2     0     2     2     0     8    0
amappl7    128      183    0      167     1     0     1     1     0     8    0
amappl6    120      297    0      275     2     0     2     2     0     8    1
amappl5    112      550    0      532     1     0     1     1     0     8    0
amappl4    104     1059    0     1027     2     0     2     2     0     8    0
amappl3     96     2076    0     2015     2     0     2     2     0     8    0
amappl2     88     1119    0     1045     4     1     3     3     0     8    1
amappl1     80    24430    0    23604    25     2    23    23     0     8    5
amappl      88     4803    0     4652     4     0     4     4     0    92    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma1024    1024       1    0        0     1     0     1     1     0     8    0
dma256     256        6    0        6     1     1     0     1     0     8    0
dma128     128      253    0      253     1     1     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       18    0       17     1     0     1     1     0     8    0
aobjpl      72        2    0        0     1     0     1     1     0     8    0
uaddrrnd    24      878    0      848     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24      878    0      848     1     0     1     1     0     8    0
vmmpekpl   168    12799    0    12765     2     0     2     2     0     8    0
vmmpepl    168    83425    0    81108   108     3   105   105     0   357    0
vmsppl     368      877    0      848     4     0     4     4     0     8    0
rwobjpl     56    26261    0    22610    52     0    52    52     0     8    0
pdppl      4096    1763    0     1696   100    31    69    83     0     8    2
pvpl        32   394867    0   382892   275     4   271   271     0   265  173
pmappl     248      877    0      848     3     0     3     3     0     8    0
extentpl    40       56    0       38     1     0     1     1     0     8    0
phpool     112      701    0       35    20     0    20    20     0     8    0
ddb{0}> 

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/10/09 02:33 openbsd 721f596233a1 aea5da89 .config console log report syz [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore panic: pmap_unwire: invalid PDE
2024/02/22 02:31 openbsd 8788436e7766 9e510875 .config console log report syz [disk image] [bsd.gdb] [kernel image] ci-openbsd-setuid panic: pmap_unwire: invalid PDE
2024/02/22 01:58 openbsd 8788436e7766 9e510875 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-setuid panic: pmap_unwire: invalid PDE
2022/10/09 02:10 openbsd 721f596233a1 aea5da89 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore panic: pmap_unwire: invalid PDE
2022/08/20 02:18 openbsd 24b6f6bc8d78 26a13b38 .config console log report ci-openbsd-multicore panic: pmap_unwire: invalid PDE
2022/08/11 20:34 openbsd a004629d760f 787ed7e0 .config console log report ci-openbsd-setuid panic: pmap_unwire: invalid PDE
* Struck through repros no longer work on HEAD.