syzbot


protection_fault: softclock

Status: upstream: reported on 2024/01/09 18:26
Reported-by: syzbot+507519d94c88c40557d7@syzkaller.appspotmail.com
First crash: 111d, last: 65d

Sample crash report:
kernel: protection fault trap, code=0
Stopped at      softclock+0xc7: movq    %rcx,0x8(%rax)
ddb{0}> 
ddb{0}> set $lines = 0
ddb{0}> set $maxwidth = 0
ddb{0}> show panic
the kernel did not panic
ddb{0}> trace
softclock(0) at softclock+0xc7 sys/kern/kern_timeout.c:750
softintr_dispatch(0) at softintr_dispatch+0xfb sys/arch/amd64/amd64/softintr.c:90
Xsoftclock() at Xsoftclock+0x27
acpicpu_idle() at acpicpu_idle+0x312 sys/dev/acpi/acpicpu.c:1206
sched_idle(ffffffff82d1bff0) at sched_idle+0x41e sys/kern/kern_sched.c:183
end trace frame: 0x0, count: -5
ddb{0}> show registers
rdi                                0
rsi                              0x1
rbp               0xffff80002a101ef0
rbx                              0x1
rdx                                0
rcx               0xffffffff82e2fb20    timeout_todo
rax               0x3737303130333231
r8                              0xd8
r9                              0xd8
r10               0x58bea71c3636c7d9
r11               0x7da11b5f01b70d28
r12                       0xffffffff
r13               0xffffffff82df1ab0    logsoftc+0x30
r14                                0
r15               0xfffffd8067c9ef00
rip               0xffffffff81e99ec7    softclock+0xc7
cs                               0x8
rflags                       0x10246    __ALIGN_SIZE+0xf246
rsp               0xffff80002a101eb0
ss                              0x10
softclock+0xc7: movq    %rcx,0x8(%rax)
ddb{0}> show proc
PROC (idle0) tid=78310 pid=45323 tcnt=1 stat=onproc
    flags process=14000<NOZOMBIE,SYSTEM> proc=40000200<SYSTEM,CPUPEG>
    runpri=0, usrpri=50, slppri=0, nice=20
    wchan=0x0, wmesg=, ps_single=0x0
    forw=0x9eb9a17ad8836598, list=0xffff80002a0fc7e8,0xffff80002a0fbd58
    process=0xffff8000ffffe7f0 user=0xffff80002a0fd000, vmspace=0xffffffff82debee8
    estcpu=0, cpticks=70945, pctcpu=0.0, user=0, sys=0, intr=0
ddb{0}> ps
   PID     TID   PPID    UID  S       FLAGS  WAIT          COMMAND
 51941  305717  25102      0  2       0x480                syz-executor.1
 51941  418642  25102      0  3   0x4000080  ttyout        syz-executor.1
 51941   67369  25102      0  3   0x4000080  fsleep        syz-executor.1
 51941  177296  25102      0  3   0x4000080  ttyout        syz-executor.1
 26292  169140  50176      0  3        0x80  nanoslp       syz-executor.5
 26292  190919  50176      0  3   0x4000080  netcon2       syz-executor.5
 26292  413665  50176      0  3   0x4000080  fsleep        syz-executor.5
 26292  480591  50176      0  3   0x4000080  fsleep        syz-executor.5
 32170  123470  80804      0  3        0x82  piperd        syz-executor.0
 23334  235558  80804      0  3        0x82  piperd        syz-executor.7
 49651  171097      1      0  3    0x100083  ttyin         getty
 93296  108697  80804      0  3        0x82  piperd        syz-executor.2
 77930  151706      0      0  3     0x14280  nfsidl        nfsio
 38921   16539      0      0  3     0x14280  nfsidl        nfsio
  8398  448461      0      0  3     0x14280  nfsidl        nfsio
 54234  289320      0      0  3     0x14280  nfsidl        nfsio
  7776  450730      0      0  3     0x14280  nfsidl        nfsio
 16593   38551      0      0  3     0x14280  nfsidl        nfsio
  4924   31569      0      0  3     0x14280  nfsidl        nfsio
 93802  327806      0      0  3     0x14280  nfsidl        nfsio
 84820   48413      0      0  3     0x14280  nfsidl        nfsio
 37320   57886      0      0  3     0x14280  nfsidl        nfsio
 58525  155047      0      0  3     0x14280  nfsidl        nfsio
 49795  517868      0      0  3     0x14280  nfsidl        nfsio
 96127  296911      0      0  3     0x14280  nfsidl        nfsio
  8958  202602      0      0  3     0x14280  nfsidl        nfsio
 63941  431054      0      0  3     0x14280  nfsidl        nfsio
 67450  192951      0      0  3     0x14280  nfsidl        nfsio
 36029  305508      0      0  3     0x14280  nfsidl        nfsio
 45459  434329      0      0  3     0x14280  nfsidl        nfsio
 51657  188263      0      0  3     0x14280  nfsidl        nfsio
 46004  514298      0      0  3     0x14280  nfsidl        nfsio
 50176   19130  80804      0  3        0x82  nanoslp       syz-executor.5
 25102  122335  80804      0  7         0x2                syz-executor.1
 68100   62557  80804      0  3        0x82  piperd        syz-executor.4
 28850   36188      0      0  3     0x14200  acct          acct
 50974  163773  80804      0  3        0x82  piperd        syz-executor.3
 89159  505354  80804      0  3        0x82  piperd        syz-executor.6
 57575  114479      0      0  3     0x14200  bored         sosplice
 80804  131111  63653      0  3   0x2000082  wait          syz-fuzzer
 80804  346429  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804  481797  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804  341623  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804  409681  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  505696  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  299522  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  326866  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804  120875  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  157337  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804   19762  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 80804   69281  63653      0  3   0x6000082  wait          syz-fuzzer
 80804   87917  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  321857  63653      0  3   0x6000082  kqread        syz-fuzzer
 80804  290732  63653      0  3   0x6000082  wait          syz-fuzzer
 80804  154780  63653      0  3   0x6000082  thrsleep      syz-fuzzer
 63653  269493  12641      0  3    0x10008a  sigsusp       ksh
 12641  147839  88471      0  3        0x9a  kqread        sshd
 88471  176361      1      0  3        0x88  kqread        sshd
 82660  150302  83698     74  3   0x1100092  bpf           pflogd
 83698   31968      1      0  3        0x80  netio         pflogd
 37625    5853  37548     73  3   0x1100090  kqread        syslogd
 37548   50945      1      0  3    0x100082  netio         syslogd
 95952    4041      1      0  3    0x100080  kqread        resolvd
 95655  260420      0      0  3     0x14200  bored         smr
 73129   28269      0      0  3     0x14200  pgzero        zerothread
 61477  365364      0      0  3     0x14200  aiodoned      aiodoned
  8888  240008      0      0  3     0x14200  syncer        update
 97806  519624      0      0  3     0x14200  cleaner       cleaner
 43744  240971      0      0  3     0x14200  reaper        reaper
 54119  256425      0      0  3     0x14200  pgdaemon      pagedaemon
 28024  290059      0      0  3     0x14200  bored         viomb
 16015   81528      0      0  3  0x40014200  acpi0         acpi0
 33983  434344      0      0  3  0x40014200                idle1
 40656  449039      0      0  3     0x14200  bored         softnet3
 99722  141447      0      0  3     0x14200  bored         softnet2
 45944   88368      0      0  3     0x14200  bored         softnet1
 30641  250749      0      0  3     0x14200  bored         softnet0
 65238  115387      0      0  3     0x14200  bored         systqmp
 95777  474237      0      0  3     0x14200  bored         systq
 68957  375236      0      0  3     0x14200  tmoslp        softclockmp
 28479  169403      0      0  3  0x40014200  tmoslp        softclock
*45323   78310      0      0  7  0x40014200                idle0
     1   87284      0      0  3     0x80082  wait          init
     0       0     -1      0  3     0x10200  scheduler     swapper
ddb{0}> show all locks
CPU 0:
exclusive mutex /syzkaller/managers/multicore/kernel/sys/kern/kern_timeout.c:57 r = 0 (0xffffffff82d1eae0)
#0  witness_lock+0x447
#1  mtx_enter_try+0x104
#2  mtx_enter+0x4f sys/kern/kern_lock.c:266
#3  softclock_process_tick_timeout+0x1b2 sys/kern/kern_timeout.c:722
#4  softclock+0x130 sys/kern/kern_timeout.c:748
#5  softintr_dispatch+0xfb sys/arch/amd64/amd64/softintr.c:90
#6  Xsoftclock+0x27
#7  acpicpu_idle+0x312 sys/dev/acpi/acpicpu.c:1206
#8  sched_idle+0x41e sys/kern/kern_sched.c:183
#9  proc_trampoline+0x10
Process 45323 (idle0) thread 0xffff80002a0fbff0 (78310)
exclusive kernel_lock &kernel_lock r = 0 (0xffffffff82da9108)
#0  witness_lock+0x447
#1  softintr_dispatch+0x52 sys/arch/amd64/amd64/softintr.c:88
#2  Xsoftclock+0x27
#3  acpicpu_idle+0x312 sys/dev/acpi/acpicpu.c:1206
#4  sched_idle+0x41e sys/kern/kern_sched.c:183
#5  proc_trampoline+0x10
exclusive mutex /syzkaller/managers/multicore/kernel/sys/kern/kern_timeout.c:57 r = 0 (0xffffffff82d1eae0)
#0  witness_lock+0x447
#1  mtx_enter_try+0x104
#2  mtx_enter+0x4f sys/kern/kern_lock.c:266
#3  softclock_process_tick_timeout+0x1b2 sys/kern/kern_timeout.c:722
#4  softclock+0x130 sys/kern/kern_timeout.c:748
#5  softintr_dispatch+0xfb sys/arch/amd64/amd64/softintr.c:90
#6  Xsoftclock+0x27
#7  acpicpu_idle+0x312 sys/dev/acpi/acpicpu.c:1206
#8  sched_idle+0x41e sys/kern/kern_sched.c:183
#9  proc_trampoline+0x10
ddb{0}> show malloc
           Type InUse  MemUse  HighUse   Limit  Requests Type Lim
         devbuf 10230   6510K    7369K 166960K     30411        0
            pcb    15     24K      29K 166960K      2016        0
         rtable   199     14K      16K 166960K      5351        0
             pf    37     10K      11K 166960K       492        0
         ifaddr    41     16K      18K 166960K       541        0
        ifgroup    64      2K       3K 166960K       876        0
         sysctl     4      1K       2K 166960K        18        0
       counters    68     36K      37K 166960K       510        0
       ioctlops     0      0K       4K 166960K      3421        0
            iov     0      0K      34K 166960K      3509        0
          mount     1      1K       1K 166960K         1        0
            log     0      0K       0K 166960K         4        0
         vnodes  1538     96K      96K 166960K     10791        0
      UFS quota     1     32K      32K 166960K         1        0
      UFS mount     5     36K      36K 166960K         5        0
            shm     2      1K       9K 166960K       285        0
         VM map     2      1K       1K 166960K         2        0
            sem    12      0K       0K 166960K      3266        0
        dirhash    12      2K       3K 166960K       183        0
           ACPI  1697    195K     286K 166960K     12548        0
      file desc    12     41K      85K 166960K     27779        0
          sigio     0      0K       0K 166960K      1449        0
           proc    75     68K     128K 166960K      4135        0
        subproc   104      6K       6K 166960K      1216        0
    NFS srvsock     1      0K       0K 166960K         1        0
     NFS daemon     1     16K      16K 166960K         1        0
    ip_moptions     0      0K       0K 166960K      2346        0
       in_multi    77      5K       7K 166960K      1347        0
    ether_multi     1      0K       0K 166960K        29        0
            mrt     1      0K       0K 166960K         6        0
    ISOFS mount     1     32K      32K 166960K         1        0
  MSDOSFS mount     1     16K      16K 166960K         1        0
           ttys   271   1208K    1208K 166960K       271        0
           exec     0      0K       1K 166960K      4225        0
     pfkey data     0      0K       0K 166960K        14        0
            tdb     3      0K       0K 166960K         3        0
        VM swap     8     62K      64K 166960K        10        0
       UVM amap   654   1503K    1508K 166960K    271569        0
       UVM aobj   131      4K       4K 166960K       131        0
        memdesc     1      4K       4K 166960K         1        0
    crypto data     1      1K       1K 166960K         1        0
    ip6_options     0      0K       0K 166960K      1109        0
            NDP    14      0K       1K 166960K       422        0
           temp    74   6772K    6904K 166960K    251735        0
         kqueue     6     10K      30K 166960K      2118        0
      SYN cache     2     36K      44K 166960K         3        0
ddb{0}> show all pools
Name      Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
plcache    128       24    0        0     1     0     1     1     0     8    0
rtpcb      120     1632    0     1631    20    19     1     4     0     8    0
rtentry    112     1579    0     1491     7     4     3     4     0     8    0
unpcb      144    21054    0    21046   206   200     6    10     0     8    5
syncache   320      246    0      246    42    41     1     1     0     8    1
sackhl      24        6    0        6     3     3     0     1     0     8    0
tcpqe       32      887    0      887    36    35     1     2     0     8    1
tcpcb      808     8038    0     8022   178   174     4    12     0     8    0
arp        120      342    0      328     1     0     1     1     0     8    0
inpcb      376    20866    0    20849   303   294     9    17     0     8    4
nd6        136      289    0      271     1     0     1     1     0     8    0
pkpcb       40      218    0      218    20    20     0     1     0     8    0
kcovpl      48       93    0       85     1     0     1     1     0     8    0
ppxss      1168      38    0       38     8     8     0     1     0     8    0
pffrag     232      276    0      275     3     2     1     1     0   482    0
pffrnode    88      276    0      275     3     2     1     1     0     8    0
pffrent     40      714    0      713     3     2     1     1     0     8    0
pfosfp      40     1428    0     1005     5     0     5     5     0     8    0
pfosfpen   112     1428    0      714    21     0    21    21     0     8    0
pfstitem    24      921    0      895     1     0     1     1     0     8    0
pfstkey    128      923    0      897     7     5     2     3     0     8    0
pfstate    376      923    0      897    23    20     3     9     0     8    0
pfrule     1344      21    0       16     2     1     1     2     0     8    0
rttmr      136        1    0        1     1     1     0     1     0     8    0
art_heap8  4096      21    0       20     8     7     1     3     0     8    0
art_heap4  256     5057    0     4687    81    55    26    30     0     8    0
art_table   32     5078    0     4707     4     0     4     4     0     8    0
art_node    16     1510    0     1430     1     0     1     1     0     8    0
sysvmsgpl   40        9    0        9     2     2     0     1     0     8    0
semapl     112     3264    0     3254     1     0     1     1     0     8    0
shmpl      112      128    0        0     4     0     4     4     0     8    0
dirhash    1024     131    0      114     3     0     3     3     0     8    0
dino2pl    256    36956    0    35387    99     0    99    99     0     8    0
ffsino     272    36956    0    35387   105     0   105   105     0     8    0
nchpl      144    74245    0    72602    64     0    64    64     0     8    0
uvmvnodes   80     7848    0        0   161     0   161   161     0     8    0
vnodes     216     7848    0        0   436     0   436   436     0     8    0
namei      1024  246705    0   246705    10     9     1     2     0     8    1
percpumem   16      269    0      221     1     0     1     1     0     8    0
vcpupl     2048     236    0        3    30     0    30    30     0     8    0
vmpool     696      308    0       75    23     1    22    22     0     8    0
kstatmem   264      470    0      442     3     0     3     3     0     8    0
scxspl     216   218876    0   218876    43    41     2     8     1     8    2
plimitpl   152     3032    0     3016     1     0     1     1     0     8    0
sigapl     424    27978    0    27916    17     9     8     9     0     8    0
futexpl     64   211286    0   211283     3     2     1     1     0     8    0
knotepl    120      905    0        0    17     1    16    17     0     8    0
kqueuepl   216     5454    0     5449   100    96     4     8     0     8    3
pipepl     320     4399    0     4371   122   119     3    11     0     8    0
fdescpl    496    27913    0    27891     5     0     5     5     0     8    0
filepl     152   157440    0   157214   252   238    14    23     0     8    2
lockfpl    104     6172    0     6171     8     7     1     2     0     8    0
lockfspl    48     2017    0     2016     1     0     1     1     0     8    0
sessionpl  144      121    0      105     1     0     1     1     0     8    0
pgrppl      48      486    0      470     1     0     1     1     0     8    0
ucredpl    104    21395    0    21375     1     0     1     1     0     8    0
zombiepl   144    27916    0    27916     3     2     1     1     0     8    1
processpl  1136   27978    0    27916     6     1     5     6     0     8    0
procpl     680    76367    0    76284    16     7     9    10     0     8    1
srpgc       96      125    0      125    35    34     1     1     0     8    1
sosppl     168      325    0      323    30    29     1     1     0     8    0
sockpl     584    43798    0    43772   388   376    12    21     0     8    6
mcl64k     65536     49    0        0     5     2     3     3     0     8    0
mcl16k     16384     47    0        0     5     2     3     3     0     8    0
mcl12k     12288     33    0        0     2     0     2     2     0     8    0
mcl9k      9216      19    0        0     2     0     2     2     0     8    0
mcl8k      8192      41    0        0     3     0     3     3     0     8    0
mcl4k      4096      74    0        0     4     1     3     3     0     8    0
mcl2k2     2112      24    0        0     2     0     2     2     0     8    0
mcl2k      2048     736    0        0    39    24    15    38     0     8    1
mtagpl      96      583    0        0     9     3     6     9     0     8    0
mbufpl     256     1938    0        0    86     1    85    85     0     8    0
bufpl      280    45171    0    37323   561     0   561   561     0     8    0
anonpl      24  2746513    0  2732733   349   237   112   167     0   186    0
amapchunkpl 152  853060    0   852229   169   122    47    67     0   158    5
amappl16   200    57077    0    56616   256   229    27    44     0     8    0
amappl15   192       21    0       21     7     7     0     1     0     8    0
amappl14   184      449    0      429     5     3     2     2     0     8    0
amappl13   176       28    0       28     6     6     0     1     0     8    0
amappl12   168    29909    0    29881     3     1     2     2     0     8    0
amappl11   160       79    0       70     1     0     1     1     0     8    0
amappl10   152      182    0      159     2     0     2     2     0     8    0
amappl9    144      438    0      436     1     0     1     1     0     8    0
amappl8    136     1479    0     1241     9     0     9     9     0     8    0
amappl7    128      480    0      456     3     1     2     3     0     8    0
amappl6    120     1807    0     1785     1     0     1     1     0     8    0
amappl5    112      610    0      602     1     0     1     1     0     8    0
amappl4    104     1435    0     1400    14    12     2     2     0     8    0
amappl3     96   166612    0   166544     3     0     3     3     0     8    0
amappl2     88    29612    0    29539     9     7     2     3     0     8    0
amappl1     80   112594    0   112131    23    11    12    23     0     8    0
amappl      88   269616    0   269302     9     0     9     9     0    92    0
dma4096    4096       1    0        1     1     1     0     1     0     8    0
dma1024    1024       1    0        0     1     0     1     1     0     8    0
dma256     256        6    0        6     1     1     0     1     0     8    0
dma128     128      253    0      253     1     1     0     1     0     8    0
dma64       64        6    0        6     1     1     0     1     0     8    0
dma32       32        7    0        7     1     1     0     1     0     8    0
dma16       16       18    0       17     1     0     1     1     0     8    0
aobjpl      72      130    0        0     3     0     3     3     0     8    0
uaddrrnd    24    28221    0    27966     2     0     2     2     0     8    0
uaddrbest   32        2    0        0     1     0     1     1     0     8    0
uaddr       24    28221    0    27966     2     0     2     2     0     8    0
vmmpekpl   168   199857    0   199749     5     0     5     5     0     8    0
vmmpepl    168  1667155    0  1664727   508   362   146   164     0   357    0
vmsppl     448    28220    0    27966    33     4    29    29     0     8    0
rwobjpl     56   401609    0   391904   163    24   139   142     0     8    0
pdppl      4096   56449    0    56165  1085   789   296   296     0     8   12
pvpl        32    42757    0        0   345     0   345   345     0   265    0
pmappl     248    28220    0    27966    18     1    17    17     0     8    0
extentpl    40       56    0       38     1     0     1     1     0     8    0
phpool     112     2189    0     1461    21     0    21    21     0     8    0
ddb{0}> machine ddbcpu 0
Invalid cpu 0
ddb{0}> trace
softclock(0) at softclock+0xc7 sys/kern/kern_timeout.c:750
softintr_dispatch(0) at softintr_dispatch+0xfb sys/arch/amd64/amd64/softintr.c:90
Xsoftclock() at Xsoftclock+0x27
acpicpu_idle() at acpicpu_idle+0x312 sys/dev/acpi/acpicpu.c:1206
sched_idle(ffffffff82d1bff0) at sched_idle+0x41e sys/kern/kern_sched.c:183
end trace frame: 0x0, count: -5
ddb{0}> machine ddbcpu 1
Stopped at      x86_ipi_db+0x1e:        addq    $0x8,%rsp
ddb{1}> trace
x86_ipi_db(ffff800029cebff0) at x86_ipi_db+0x1e sys/arch/amd64/amd64/db_interface.c:393
x86_ipi_handler() at x86_ipi_handler+0xb7 sys/arch/amd64/amd64/ipi.c:106
Xresume_lapic_ipi() at Xresume_lapic_ipi+0x27
__mp_lock(ffffffff82da8f00) at __mp_lock+0x122 __mp_lock_spin sys/kern/kern_lock.c:116 [inline]
__mp_lock(ffffffff82da8f00) at __mp_lock+0x122 sys/kern/kern_lock.c:147
syscall(ffff80002bc961b0) at syscall+0x51a mi_syscall sys/sys/syscall_mi.h:183 [inline]
syscall(ffff80002bc961b0) at syscall+0x51a sys/arch/amd64/amd64/trap.c:577
Xsyscall() at Xsyscall+0x128
end of kernel
end trace frame: 0x7d0ad51447c0, count: -6

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/02/22 11:03 openbsd 921b38ae7cad 345111b5 .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore protection_fault: softclock
2024/01/07 14:38 openbsd 172a5cb9e97b 28c42cff .config console log report [disk image] [bsd.gdb] [kernel image] ci-openbsd-multicore protection_fault: softclock
* Struck through repros no longer work on HEAD.