syzbot


panic: mutex ADDR not held in knote_dequeue

Status: fixed on 2021/06/11 16:00
Reported-by: syzbot+c2aba7645a218ce03027@syzkaller.appspotmail.com
Fix commit: baa47be05351 Remember to lock kqueue mutex in filt_timermodify().
First crash: 1043d, last: 1043d

Sample crash report:
login: panic: mutex 0xfffffd806e89a870 not held in knote_dequeue
Stopped at      db_enter+0x18:  addq    $0x8,%rsp
    TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
*155725   6868      0           0  0x4000000    0  syz-executor7930
 319424  40632      0     0x14000      0x200    1  zerothread
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:399
panic(ffffffff82460c20) at panic+0x177 sys/kern/subr_prf.c:202
knote_dequeue(fffffd806ecef070) at knote_dequeue+0x12b sys/kern/kern_event.c:1860
filt_timermodify(ffff800021268e30,fffffd806ecef070) at filt_timermodify+0x6f sys/kern/kern_event.c:479
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 filter_modify sys/kern/kern_event.c:662 [inline]
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 sys/kern/kern_event.c:1125
sys_kevent(ffff8000211b2a88,ffff800021269048,ffff800021269090) at sys_kevent+0x214 sys/kern/kern_event.c:887
syscall(ffff800021269110) at syscall+0x5bf mi_syscall sys/sys/syscall_mi.h:102 [inline]
syscall(ffff800021269110) at syscall+0x5bf sys/arch/amd64/amd64/trap.c:587
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x564680c1880, count: 7
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{0}> 
ddb{0}> set $lines = 0
ddb{0}> set $maxwidth = 0
ddb{0}> show panic
*cpu0: mutex 0xfffffd806e89a870 not held in knote_dequeue
ddb{0}> trace
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:399
panic(ffffffff82460c20) at panic+0x177 sys/kern/subr_prf.c:202
knote_dequeue(fffffd806ecef070) at knote_dequeue+0x12b sys/kern/kern_event.c:1860
filt_timermodify(ffff800021268e30,fffffd806ecef070) at filt_timermodify+0x6f sys/kern/kern_event.c:479
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 filter_modify sys/kern/kern_event.c:662 [inline]
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 sys/kern/kern_event.c:1125
sys_kevent(ffff8000211b2a88,ffff800021269048,ffff800021269090) at sys_kevent+0x214 sys/kern/kern_event.c:887
syscall(ffff800021269110) at syscall+0x5bf mi_syscall sys/sys/syscall_mi.h:102 [inline]
syscall(ffff800021269110) at syscall+0x5bf sys/arch/amd64/amd64/trap.c:587
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x564680c1880, count: -8
ddb{0}> show registers
rdi                                0
rsi                              0x1
rbp               0xffff800021268b80
rbx               0xffffffff8275dbf7    cpu_info_full_primary+0x2bf7
rdx                             0x8b
rcx                              0x2
rax                             0x3a
r8                0xffffffff816e1c04    kprintf+0x144
r9                               0x1
r10               0xdc56e1594e5dfa95
r11               0xf24ae5dbeab2f647
r12               0xffffffff8275d9f8    cpu_info_full_primary+0x29f8
r13                                0
r14                                0
r15                              0x1
rip               0xffffffff81910f78    db_enter+0x18
cs                               0x8
rflags                         0x246
rsp               0xffff800021268b70
ss                              0x10
db_enter+0x18:  addq    $0x8,%rsp
ddb{0}> show proc
PROC (syz-executor7930) pid=155725 stat=onproc
    flags process=0 proc=4000000<THREAD>
    pri=32, usrpri=52, nice=20
    forw=0xffffffffffffffff, list=0xffff8000211b22a8,0xffff8000211b3278
    process=0xffff800021224880 user=0xffff800021264000, vmspace=0xfffffd807f0098a0
    estcpu=36, cpticks=0, pctcpu=0.0
    user=0, sys=0, intr=0
ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
  6868  162702  23117      0  3        0x80  nanoslp       syz-executor7930
* 6868  155725  23117      0  7   0x4000000                syz-executor7930
  6868   36273  23117      0  2   0x4000000                syz-executor7930
 96415  241950  16533      0  2           0                syz-executor7930
 96415  341938  16533      0  3   0x4000080  fsleep        syz-executor7930
 96415  249725  16533      0  2   0x4000000                syz-executor7930
 16533  520432  41296      0  3        0x80  nanoslp       syz-executor7930
 23117  434765  41296      0  3        0x80  nanoslp       syz-executor7930
 41296  163560  21851      0  3        0x82  nanoslp       syz-executor7930
 21851  449004   5399      0  3    0x10008a  sigsusp       ksh
  5399    9273  74993      0  3        0x92  select        sshd
 44833   32472      1      0  3    0x100083  ttyin         getty
 74993  127584      1      0  3        0x80  select        sshd
 28113  487556  69736     74  3    0x100092  bpf           pflogd
 69736   21365      1      0  3        0x80  netio         pflogd
 30448  372728  94224     73  3    0x100090  kqread        syslogd
 94224  305917      1      0  3    0x100082  netio         syslogd
 93312  518488      1     77  3    0x100090  poll          dhclient
 15846  437395      1      0  3        0x80  poll          dhclient
 96061  135682      0      0  3     0x14200  bored         smr
 40632  319424      0      0  7     0x14200                zerothread
 94254  461080      0      0  3     0x14200  aiodoned      aiodoned
 44057    2900      0      0  3     0x14200  syncer        update
 36998  380132      0      0  3     0x14200  cleaner       cleaner
 84325  328959      0      0  3     0x14200  reaper        reaper
 35038  351601      0      0  3     0x14200  pgdaemon      pagedaemon
 51403  214450      0      0  3     0x14200  bored         crynlk
 50086  448154      0      0  3     0x14200  bored         crypto
 30097  464327      0      0  3     0x14200  bored         viomb
 60207  118260      0      0  3  0x40014200  acpi0         acpi0
 48566  353279      0      0  3  0x40014200                idle1
 62493  463075      0      0  3     0x14200  bored         softnet
 96662  496337      0      0  3     0x14200  bored         systqmp
 70361  236130      0      0  3     0x14200  bored         systq
 63745   67605      0      0  3  0x40014200  bored         softclock
 27601  255978      0      0  3  0x40014200                idle0
     1    8539      0      0  3        0x82  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{0}> show all locks
CPU 1:
exclusive mutex &uvm.fpageqlock r = 0 (0xffffffff8296da20)
#0  witness_lock+0x4b0 stacktrace_save sys/sys/stacktrace.h:36 [inline]
#0  witness_lock+0x4b0 sys/kern/subr_witness.c:1182
#1  mtx_enter_try+0x100
#2  mtx_enter+0x4b sys/kern/kern_lock.c:266
#3  uvm_pagezero_thread+0x191 sys/uvm/uvm_pmemrange.c:2185
#4  proc_trampoline+0x1c
Process 6868 (syz-executor7930) thread 0xffff8000211b2a88 (155725)
exclusive kernel_lock &kernel_lock r = 1 (0xffffffff827d4278)
#0  witness_lock+0x4b0 stacktrace_save sys/sys/stacktrace.h:36 [inline]
#0  witness_lock+0x4b0 sys/kern/subr_witness.c:1182
#1  syscall+0x439 mi_syscall sys/sys/syscall_mi.h:93 [inline]
#1  syscall+0x439 sys/arch/amd64/amd64/trap.c:587
#2  Xsyscall+0x128
ddb{0}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf 10112   6476K    6477K  78643K     11202        0
            pcb    13      8K       8K  78643K        13        0
         rtable    61      2K       2K  78643K       119        0
         ifaddr    29      8K       8K  78643K        30        0
       counters    40     33K      33K  78643K        40        0
       ioctlops     0      0K       4K  78643K      1467        0
          mount     1      1K       1K  78643K         1        0
            log     0      0K       0K  78643K         1        0
         vnodes  1183     74K      75K  78643K      1188        0
      UFS quota     1     32K      32K  78643K         1        0
      UFS mount     5     36K      36K  78643K         5        0
            shm     2      1K       1K  78643K         2        0
         VM map     2      1K       1K  78643K         2        0
            sem     2      0K       0K  78643K         2        0
        dirhash    12      2K       2K  78643K        12        0
           ACPI  1697    195K     286K  78643K     12598        0
      file desc     1      0K       0K  78643K         1        0
           proc    59     63K      71K  78643K       357        0
    NFS srvsock     1      0K       0K  78643K         1        0
     NFS daemon     1     16K      16K  78643K         1        0
       in_multi    11      0K       0K  78643K        11        0
    ether_multi     1      0K       0K  78643K         1        0
    ISOFS mount     1     32K      32K  78643K         1        0
  MSDOSFS mount     1     16K      16K  78643K         1        0
           ttys    19     95K      95K  78643K        19        0
           exec     0      0K       2K  78643K       298        0
        pagedep     1      8K       8K  78643K         1        0
       inodedep     1     32K      32K  78643K         1        0
         newblk     1      0K       0K  78643K         1        0
        VM swap     7     26K      26K  78643K         7        0
       UVM amap    98      4K       5K  78643K      1908        0
       UVM aobj     3      2K       2K  78643K         3        0
        memdesc     1      4K       4K  78643K         1        0
    crypto data     1      1K       1K  78643K         1        0
            NDP     4      0K       0K  78643K         4        0
           temp    23   3973K    4037K  78643K      1755        0
         kqueue     6      3K       5K  78643K       170        0
      SYN cache     2     16K      16K  78643K         2        0
ddb{0}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache    128       22    0        0     1     0     1     1     0     8    0
rtpcb      120       15    0       13     1     0     1     1     0     8    0
rtentry    112       23    0        1     1     0     1     1     0     8    0
unpcb      120       29    0       19     1     0     1     1     0     8    0
syncache   296        5    0        5     2     1     1     1     0     8    1
tcpcb      736        8    0        5     1     0     1     1     0     8    0
arp        120        2    0        0     1     0     1     1     0     8    0
inpcb      304       28    0       23     1     0     1     1     0     8    0
pfosfp      40     1428    0     1005     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfstitem    24        8    0        2     1     0     1     1     0     8    0
pfstkey    112        8    0        2     1     0     1     1     0     8    0
pfstate    320        8    0        2     1     0     1     1     0     8    0
pfrule     1360      21    0       16     2     1     1     2     0     8    0
art_heap8  4096       1    0        0     1     0     1     1     0     8    0
art_heap4  256       97    0        0     7     0     7     7     0     8    0
art_table   32       98    0        0     1     0     1     1     0     8    0
art_node    16       22    0        2     1     0     1     1     0     8    0
dirhash    1024      17    0        0     3     0     3     3     0     8    0
dino2pl    256     1446    0       57    87     0    87    87     0     8    0
ffsino     272     1446    0       57    93     0    93    93     0     8    0
nchpl      144     1665    0      117    58     0    58    58     0     8    0
uvmvnodes   72     1456    0        0    27     0    27    27     0     8    0
vnodes     224     1456    0        0    86     0    86    86     0     8    0
namei      1024    3949    0     3949     2     1     1     1     0     8    1
percpumem   16       32    0        0     1     0     1     1     0     8    0
scxspl     216     3851    0     3851    10     9     1     8     0     8    1
plimitpl   152       14    0        8     1     0     1     1     0     8    0
sigapl     424      265    0      232     4     0     4     4     0     8    0
futexpl     56      134    0      133     1     0     1     1     0     8    0
knotepl    112       21    0        0     1     0     1     1     0     8    0
kqueuepl   216       85    0       82     1     0     1     1     0     8    0
pipepl     336       64    0       60     2     1     1     1     0     8    0
fdescpl    496      249    0      232     3     0     3     3     0     8    0
filepl     152     1079    0     1030     2     0     2     2     0     8    0
lockfpl    104        5    0        4     1     0     1     1     0     8    0
lockfspl    48        3    0        2     1     0     1     1     0     8    0
sessionpl  144       18    0        9     1     0     1     1     0     8    0
pgrppl      48       18    0        9     1     0     1     1     0     8    0
ucredpl     96       62    0       53     1     0     1     1     0     8    0
zombiepl   144      232    0      232     2     1     1     1     0     8    1
processpl  1080     265    0      232     3     0     3     3     0     8    0
procpl     672      341    0      304     4     0     4     4     0     8    0
sockpl     480       72    0       55     4     1     3     3     0     8    0
mcl8k      8192       3    0        0     1     0     1     1     0     8    0
mcl4k      4096       3    0        0     1     0     1     1     0     8    0
mcl2k      2048      86    0        0    10     0    10    10     0     8    0
mtagpl      96        1    0        0     1     0     1     1     0     8    0
mbufpl     256      106    0        0     6     0     6     6     0     8    0
bufpl      280     1991    0       93   136     0   136   136     0     8    0
anonpl      24    36395    0    33899    19     3    16    16     0   186    0
amapchunkpl 152    3717    0     3535    10     2     8     8     0   158    1
amappl16   200       74    0       70     2     1     1     1     0     8    0
amappl15   192       51    0       46     1     0     1     1     0     8    0
amappl13   176       14    0       13     2     1     1     1     0     8    0
amappl12   168        5    0        5     1     1     0     1     0     8    0
amappl11   160       40    0       28     1     0     1     1     0     8    0
amappl10   152       31    0       24     1     0     1     1     0     8    0
amappl9    144      204    0      204     2     1     1     1     0     8    1
amappl8    136      230    0      223     1     0     1     1     0     8    0
amappl7    128       43    0       42     1     0     1     1     0     8    0
amappl6    120       84    0       74     1     0     1     1     0     8    0
amappl5    112      130    0      120     1     0     1     1     0     8    0
amappl4    104      524    0      498     1     0     1     1     0     8    0
amappl3     96      134    0      123     1     0     1     1     0     8    0
amappl2     88      292    0      259     1     0     1     1     0     8    0
amappl1     80     7179    0     6790     9     0     9     9     0     8    0
amappl      88     1704    0     1623     2     0     2     2     0    92    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma1024    1024       1    0        0     1     0     1     1     0     8    0
dma256     256        6    0        6     1     1     0     1     0     8    0
dma128     128      253    0      253     1     1     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       18    0       17     1     0     1     1     0     8    0
aobjpl      64        2    0        0     1     0     1     1     0     8    0
uaddrrnd    24      249    0      232     1     0     1     1     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24      249    0      232     1     0     1     1     0     8    0
vmmpekpl   168     5694    0     5677     1     0     1     1     0     8    0
vmmpepl    168    22428    0    21464    43     0    43    43     0   357    1
vmsppl     368      248    0      232     2     0     2     2     0     8    0
rwobjpl     56     6774    0     6243     9     1     8     8     0     8    0
pdppl      4096     505    0      464    59    18    41    41     0     8    0
pvpl        32   108737    0   104309    40     3    37    37     0   265    1
pmappl     232      248    0      232     1     0     1     1     0     8    0
extentpl    40       58    0       40     1     0     1     1     0     8    0
phpool     112      280    0       23     8     0     8     8     0     8    0
ddb{0}> machine ddbcpu 0
Invalid cpu 0
ddb{0}> trace
db_enter() at db_enter+0x18 sys/arch/amd64/amd64/db_interface.c:399
panic(ffffffff82460c20) at panic+0x177 sys/kern/subr_prf.c:202
knote_dequeue(fffffd806ecef070) at knote_dequeue+0x12b sys/kern/kern_event.c:1860
filt_timermodify(ffff800021268e30,fffffd806ecef070) at filt_timermodify+0x6f sys/kern/kern_event.c:479
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 filter_modify sys/kern/kern_event.c:662 [inline]
kqueue_register(fffffd806e89a870,ffff800021268e30,ffff8000211b2a88) at kqueue_register+0xa89 sys/kern/kern_event.c:1125
sys_kevent(ffff8000211b2a88,ffff800021269048,ffff800021269090) at sys_kevent+0x214 sys/kern/kern_event.c:887
syscall(ffff800021269110) at syscall+0x5bf mi_syscall sys/sys/syscall_mi.h:102 [inline]
syscall(ffff800021269110) at syscall+0x5bf sys/arch/amd64/amd64/trap.c:587
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x564680c1880, count: -8
ddb{0}> machine ddbcpu 1
Stopped at      x86_ipi_db+0x1a:        addq    $0x8,%rsp
x86_ipi_db(ffff800020d58ff0) at x86_ipi_db+0x1a sys/arch/amd64/amd64/db_interface.c:353
x86_ipi_handler() at x86_ipi_handler+0xb7 sys/arch/amd64/amd64/ipi.c:106
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x23
__sanitizer_cov_trace_pc() at __sanitizer_cov_trace_pc+0x4f sys/dev/kcov.c:149
uvm_pagezero_thread(ffff8000211b2548) at uvm_pagezero_thread+0xb0 sys/uvm/uvm_pmemrange.c:2171
end trace frame: 0x0, count: 10
ddb{1}> trace
x86_ipi_db(ffff800020d58ff0) at x86_ipi_db+0x1a sys/arch/amd64/amd64/db_interface.c:353
x86_ipi_handler() at x86_ipi_handler+0xb7 sys/arch/amd64/amd64/ipi.c:106
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x23
__sanitizer_cov_trace_pc() at __sanitizer_cov_trace_pc+0x4f sys/dev/kcov.c:149
uvm_pagezero_thread(ffff8000211b2548) at uvm_pagezero_thread+0xb0 sys/uvm/uvm_pmemrange.c:2171
end trace frame: 0x0, count: -5
ddb{1}> 

Crashes (24):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/06/11 03:31 openbsd daca582a2cbb 1ba81399 .config console log report syz C ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 03:15 openbsd daca582a2cbb 1ba81399 .config console log report syz C ci-openbsd-main panic: mutex ADDR not held in knote_dequeue
2021/06/11 03:02 openbsd daca582a2cbb 1ba81399 .config console log report syz C ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 13:54 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 13:53 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 13:31 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 12:14 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 11:16 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 08:40 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 07:22 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 07:02 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 05:19 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-main panic: mutex ADDR not held in knote_dequeue
2021/06/11 04:07 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:57 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-main panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:54 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-main panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:48 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:47 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:47 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:44 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:44 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:43 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-setuid panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:40 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:36 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
2021/06/11 02:34 openbsd daca582a2cbb 1ba81399 .config console log report ci-openbsd-multicore panic: mutex ADDR not held in knote_dequeue
* Struck through repros no longer work on HEAD.