syzbot


pool: free list modified: aobjpl

Status: fixed on 2020/03/04 22:54
Reported-by: syzbot+0fc1766671a9461de8a5@syzkaller.appspotmail.com
Fix commit: c5a231fb6c1e Grab a reference for the shared memory segment before calling uvm_map() as the same function could end up putting the thread to sleep. Allowing another thread to free the shared memory segment, which in turns causes a use-after-free.
First crash: 1108d, last: 1095d
Patch testing requests:
Created Duration User Patch Repo Result
2020/02/22 15:45 19m anton@basename.se https://github.com/mptre/openbsd-src syzkaller OK

Sample crash report:
login: panic: pool_do_get: aobjpl free list modified: page 0xfffffd807eca2000; item addr 0xfffffd807eca2d80; offset 0x14=0xdead410f
Stopped at      db_enter+0x18:  addq    $0x8,%rsp
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
 477596  14252      0           0          0    0  syz-executor5844
*221866  14252      0           0  0x4000000    1K syz-executor5844
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:398
panic(ffffffff8220d3b4) at panic+0x15c sys/kern/subr_prf.c:207
pool_do_get(ffffffff82651ab8,1,ffff800020b3fa18) at pool_do_get+0x472 sys/kern/subr_pool.c:744
pool_get(ffffffff82651ab8,1) at pool_get+0xeb sys/kern/subr_pool.c:581
uao_create(1000,0) at uao_create+0x7c sys/uvm/uvm_aobj.c:728
shmget_allocate_segment(ffff800020ac18c8,ffff800020b3fc08,100,ffff800020b3fc50) at shmget_allocate_segment+0x352 sys/kern/sysv_shm.c:430
sys_shmget(ffff800020ac18c8,ffff800020b3fc08,ffff800020b3fc50) at sys_shmget+0x13f sys/kern/sysv_shm.c:468
syscall(ffff800020b3fcd0) at syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
syscall(ffff800020b3fcd0) at syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0xa984a216190, count: 6
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{1}> 
ddb{1}> set $lines = 0
ddb{1}> set $maxwidth = 0
ddb{1}> show panic
pool_do_get: aobjpl free list modified: page 0xfffffd807eca2000; item addr 0xfffffd807eca2d80; offset 0x14=0xdead410f
ddb{1}> trace
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:398
panic(ffffffff8220d3b4) at panic+0x15c sys/kern/subr_prf.c:207
pool_do_get(ffffffff82651ab8,1,ffff800020b3fa18) at pool_do_get+0x472 sys/kern/subr_pool.c:744
pool_get(ffffffff82651ab8,1) at pool_get+0xeb sys/kern/subr_pool.c:581
uao_create(1000,0) at uao_create+0x7c sys/uvm/uvm_aobj.c:728
shmget_allocate_segment(ffff800020ac18c8,ffff800020b3fc08,100,ffff800020b3fc50) at shmget_allocate_segment+0x352 sys/kern/sysv_shm.c:430
sys_shmget(ffff800020ac18c8,ffff800020b3fc08,ffff800020b3fc50) at sys_shmget+0x13f sys/kern/sysv_shm.c:468
syscall(ffff800020b3fcd0) at syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
syscall(ffff800020b3fcd0) at syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0xa984a216190, count: -9
ddb{1}> show registers
rdi                                0
rsi                              0x1
rbp               0xffff800020b3f860
rbx               0xffff800020b3f910
rdx                             0x8b
rcx                              0x2
rax                              0x1
r8                0xffffffff81cd657f    kprintf+0x16f
r9                               0x1
r10               0x2fb1d95634206333
r11               0xca503300e927441c
r12                     0x3000000008
r13               0xffff800020b3f870
r14                            0x100
r15                              0x1
rip               0xffffffff813ee2e8    db_enter+0x18
cs                               0x8
rflags                         0x246
rsp               0xffff800020b3f850
ss                              0x10
db_enter+0x18:  addq    $0x8,%rsp
ddb{1}> show proc
PROC (syz-executor5844) pid=221866 stat=onproc
    flags process=0 proc=4000000<THREAD>
    pri=53, usrpri=53, nice=20
    forw=0xffffffffffffffff, list=0xffff800020ac1160,0xffffffff82617120
    process=0xffff800020aeca80 user=0xffff800020b3a000, vmspace=0xfffffd807f0085c0
    estcpu=3, cpticks=1, pctcpu=0.0
    user=0, sys=1, intr=0
ddb{1}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 60170  227293  99918      0  3           0  vmmaplk       syz-executor5844
 60170  514709  99918      0  3   0x4000000  physio        syz-executor5844
 14252  477596  18168      0  7           0                syz-executor5844
 14252  434676  18168      0  3   0x4000080  fsleep        syz-executor5844
*14252  221866  18168      0  7   0x4000000                syz-executor5844
 99918  470932  58792      0  3        0x80  nanosleep     syz-executor5844
 18168  129280  58792      0  3        0x80  nanosleep     syz-executor5844
 58792  487161  65099      0  3        0x82  nanosleep     syz-executor5844
 65099  153185  67304      0  3    0x10008a  pause         ksh
 67304   67565  70790      0  3        0x92  select        sshd
 45212  336726      1      0  3    0x100083  ttyin         getty
 70790   68170      1      0  3        0x80  select        sshd
 95697  416915  32515     74  3    0x100092  bpf           pflogd
 32515  436230      1      0  3        0x80  netio         pflogd
  7748   29218  88533     73  3    0x100090  kqread        syslogd
 88533  317647      1      0  3    0x100082  netio         syslogd
 86144  163279      1     77  3    0x100090  poll          dhclient
 10979  169650      1      0  3        0x80  poll          dhclient
 63166  285434      0      0  3     0x14200  pgzero        zerothread
 22837  116142      0      0  3     0x14200  aiodoned      aiodoned
 24496   50017      0      0  3     0x14200  syncer        update
 29848  451514      0      0  3     0x14200  cleaner       cleaner
 33296   27751      0      0  3     0x14200  reaper        reaper
 25144  152077      0      0  3     0x14200  pgdaemon      pagedaemon
 82056  380704      0      0  3     0x14200  bored         crynlk
   761  520291      0      0  3     0x14200  bored         crypto
 76481   76250      0      0  3  0x40014200  acpi0         acpi0
 43279  478589      0      0  3  0x40014200                idle1
 96587  107872      0      0  3     0x14200  bored         softnet
 53106   92255      0      0  3     0x14200  bored         systqmp
 76870  210425      0      0  3     0x14200  bored         systq
 79587  381863      0      0  3  0x40014200  bored         softclock
 11708  198651      0      0  3  0x40014200                idle0
 64363  239130      0      0  3     0x14200  bored         smr
     1  296155      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{1}> show all locks
CPU 1:
exclusive mutex aobjpl r = 0 (0xffffffff82651ac8)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  mtx_enter_try+0x102
#2  mtx_enter+0x4b sys/kern/kern_lock.c:266
#3  pool_get+0xbf sys/kern/subr_pool.c:578
#4  uao_create+0x7c sys/uvm/uvm_aobj.c:728
#5  shmget_allocate_segment+0x352 sys/kern/sysv_shm.c:430
#6  sys_shmget+0x13f sys/kern/sysv_shm.c:468
#7  syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
#7  syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
#8  Xsyscall+0x128
Process 60170 (syz-executor5844) thread 0xffff800020ac1160 (514709)
shared rwlock vmmaplk r = 0 (0xfffffd807f008d08)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  uvm_vslock_device+0x11e sys/uvm/uvm_glue.c:174
#2  physio+0x26a sys/kern/kern_physio.c:139
#3  spec_write+0xd4 sys/kern/spec_vnops.c:309
#4  VOP_WRITE+0xc6 sys/kern/vfs_vops.c:269
#5  vn_write+0x199 sys/kern/vfs_vnops.c:414
#6  dofilewritev+0x1b7 sys/kern/sys_generic.c:364
#7  sys_pwritev+0xb8 sys/kern/vfs_syscalls.c:3260
#8  syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
#8  syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
#9  Xsyscall+0x128
Process 14252 (syz-executor5844) thread 0xffff800020ac18c8 (221866)
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff82651ec8)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  syscall+0x400 mi_syscall sys/sys/syscall_mi.h:83 [inline]
#1  syscall+0x400 sys/arch/amd64/amd64/trap.c:555
#2  Xsyscall+0x128
exclusive mutex aobjpl r = 0 (0xffffffff82651ac8)
#0  witness_lock+0x52e sys/kern/subr_witness.c:1163
#1  mtx_enter_try+0x102
#2  mtx_enter+0x4b sys/kern/kern_lock.c:266
#3  pool_get+0xbf sys/kern/subr_pool.c:578
#4  uao_create+0x7c sys/uvm/uvm_aobj.c:728
#5  shmget_allocate_segment+0x352 sys/kern/sysv_shm.c:430
#6  sys_shmget+0x13f sys/kern/sysv_shm.c:468
#7  syscall+0x4a4 mi_syscall sys/sys/syscall_mi.h:92 [inline]
#7  syscall+0x4a4 sys/arch/amd64/amd64/trap.c:555
#8  Xsyscall+0x128
ddb{1}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf  9461   6393K    6394K  78643K     10552        0
            pcb    13      8K       8K  78643K        13        0
         rtable    61      2K       2K  78643K       127        0
         ifaddr    29      8K       8K  78643K        30        0
       counters    39     33K      33K  78643K        39        0
       ioctlops     0      0K       4K  78643K      1467        0
          mount     1      1K       1K  78643K         1        0
         vnodes  1182     74K      74K  78643K      1192        0
      UFS quota     1     32K      32K  78643K         1        0
      UFS mount     5     36K      36K  78643K         5        0
            shm     2      1K       9K  78643K         6        0
         VM map     2      1K       1K  78643K         2        0
            sem     2      0K       0K  78643K         2        0
        dirhash    12      2K       2K  78643K        12        0
           ACPI  1809    196K     290K  78643K     12766        0
      file desc     1      0K       0K  78643K         1        0
           proc    59     63K      71K  78643K       358        0
    NFS srvsock     1      0K       0K  78643K         1        0
     NFS daemon     1     16K      16K  78643K         1        0
       in_multi    11      0K       0K  78643K        11        0
    ether_multi     1      0K       0K  78643K         1        0
    ISOFS mount     1     32K      32K  78643K         1        0
  MSDOSFS mount     1     16K      16K  78643K         1        0
           ttys    18     79K      79K  78643K        18        0
           exec     0      0K       1K  78643K       177        0
        pagedep     1      8K       8K  78643K         1        0
       inodedep     1     32K      32K  78643K         1        0
         newblk     1      0K       0K  78643K         1        0
        VM swap     7     26K      26K  78643K         7        0
       UVM amap    77      4K       4K  78643K       847        0
       UVM aobj     4      2K       2K  78643K        10        0
        memdesc     1      4K       4K  78643K         1        0
    crypto data     1      1K       1K  78643K         1        0
            NDP     4      0K       0K  78643K         4        0
           temp    27   3003K    3067K  78643K      1986        0
      SYN cache     2     16K      16K  78643K         2        0
ddb{1}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
arp         64        2    0        0     1     0     1     1     0     8    0
plcache    128       20    0        0     1     0     1     1     0     8    0
rtpcb       80       15    0       13     1     0     1     1     0     8    0
rtentry    112       23    0        1     1     0     1     1     0     8    0
unpcb      120       29    0       19     1     0     1     1     0     8    0
syncache   264        5    0        5     1     0     1     1     0     8    1
tcpcb      544       18    0       15     1     0     1     1     0     8    0
inpcb      280       39    0       33     1     0     1     1     0     8    0
pfosfp      40      846    0      423     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfstitem    24        9    0        0     1     0     1     1     0     8    0
pfstkey    112        9    0        0     1     0     1     1     0     8    0
pfstate    328        9    0        0     1     0     1     1     0     8    0
pfrule     1360      21    0       16     2     1     1     2     0     8    0
art_heap8  4096       1    0        0     1     0     1     1     0     8    0
art_heap4  256       97    0        0     7     0     7     7     0     8    0
art_table   32       98    0        0     1     0     1     1     0     8    0
art_node    16       22    0        2     1     0     1     1     0     8    0
shmpl      112        9    0        7     1     0     1     1     0     8    0
dirhash    1024      17    0        0     3     0     3     3     0     8    0
dino1pl    128     1421    0       31    45     0    45    45     0     8    0
ffsino     272     1421    0       31    93     0    93    93     0     8    0
nchpl      144     1603    0       52    58     0    58    58     0     8    0
uvmvnodes   72     1431    0        0    27     0    27    27     0     8    0
vnodes     208     1431    0        0    76     0    76    76     0     8    0
namei      1024    3826    0     3826     1     0     1     1     0     8    1
percpumem   16       30    0        0     1     0     1     1     0     8    0
scxspl     192     5115    0     5114     2     1     1     2     0     8    0
plimitpl   152       14    0        8     1     0     1     1     0     8    0
sigapl     432      212    0      196     2     0     2     2     0     8    0
futexpl     56       45    0       44     1     0     1     1     0     8    0
knotepl    112        5    0        0     1     0     1     1     0     8    0
kqueuepl   104        1    0        0     1     0     1     1     0     8    0
pipepl     160      128    0      121     1     0     1     1     0     8    0
fdescpl    488      213    0      196     3     0     3     3     0     8    0
filepl     152     1020    0      968     3     0     3     3     0     8    0
lockfpl    104        5    0        4     1     0     1     1     0     8    0
lockfspl    48        3    0        2     1     0     1     1     0     8    0
sessionpl  112       18    0        9     1     0     1     1     0     8    0
pgrppl      48       18    0        9     1     0     1     1     0     8    0
ucredpl     96       52    0       43     1     0     1     1     0     8    0
zombiepl   144      196    0      196     1     0     1     1     0     8    1
processpl  896      228    0      196     4     0     4     4     0     8    0
procpl     632      243    0      208     4     0     4     4     0     8    0
sockpl     384       83    0       63     3     0     3     3     0     8    0
mcl4k      4096       2    0        0     1     0     1     1     0     8    0
mcl2k      2048      56    0        0     7     0     7     7     0     8    0
mtagpl      80        1    0        0     1     0     1     1     0     8    0
mbufpl     256       80    0        0     5     0     5     5     0     8    0
bufpl      280     2228    0      283   139     0   139   139     0     8    0
anonpl      16    43710    0    34558    40     1    39    39     0   125    2
amapchunkpl 152     635    0      573     4     0     4     4     0   158    0
amappl16   192     1482    0     1003    25     0    25    25     0     8    1
amappl15   184       50    0       46     1     0     1     1     0     8    0
amappl14   176       15    0       14     1     0     1     1     0     8    0
amappl12   160        2    0        2     1     0     1     1     0     8    1
amappl11   152       53    0       38     1     0     1     1     0     8    0
amappl10   144        7    0        5     1     0     1     1     0     8    0
amappl9    136      425    0      424     1     0     1     1     0     8    0
amappl8    128       75    0       68     1     0     1     1     0     8    0
amappl7    120       74    0       66     1     0     1     1     0     8    0
amappl6    112       46    0       44     1     0     1     1     0     8    0
amappl5    104      131    0      117     1     0     1     1     0     8    0
amappl4     96      468    0      440     1     0     1     1     0     8    0
amappl3     88      115    0      104     1     0     1     1     0     8    0
amappl2     80      886    0      808     2     0     2     2     0     8    0
amappl1     72    13883    0    13415    15     5    10    15     0     8    0
amappl      80      436    0      407     1     0     1     1     0    84    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma256     256        6    0        6     1     1     0     1     0     8    0
dma128     128      253    0      253     1     1     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       18    0       17     1     0     1     1     0     8    0
aobjpl      64        9    0        6     1     0     1     1     0     8    0
pool(0xffffffff82651ab8:aobjpl): page inconsistency: page 0xfffffd807eca2000; 58 on list, 3 missing, 62 items per page
uaddrrnd    24      213    0      196     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24      213    0      196     1     0     1     1     0     8    0
vmmpekpl   168     5404    0     5381     2     0     2     2     0     8    0
vmmpepl    168    37884    0    36361    74     6    68    68     0   357    1
vmsppl     368      212    0      196     2     0     2     2     0     8    0
pdppl      4096     433    0      392     6     0     6     6     0     8    0
pvpl        32   147576    0   136524    93     0    93    93     0   265    3
pmappl     232      212    0      196     1     0     1     1     0     8    0
extentpl    40       46    0       29     1     0     1     1     0     8    0
phpool     112      131    0        3     4     0     4     4     0     8    0
ddb{1}> 

Crashes (3):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-openbsd-multicore 2019/12/05 23:36 openbsd ce2d25888d00 98b4ef2d .config log report syz C
ci-openbsd-main 2019/11/23 17:42 openbsd ee79b9a7c44b 598ca6c8 .config log report syz C
ci-openbsd-main 2019/11/23 16:06 openbsd ee79b9a7c44b 598ca6c8 .config log report
* Struck through repros no longer work on HEAD.