syzbot


INFO: rcu detected stall in smp_call_function_many

Status: closed as invalid on 2018/12/31 07:57
Subsystems: perf
[Documentation on labels]
Reported-by: syzbot+e0f63c28c647aad79bf7@syzkaller.appspotmail.com
First crash: 2221d, last: 2065d

Sample crash report:
VMExit: intr_info=00000000 errcode=00000000 ilen=00000000
        reason=80000021 qualification=0000000000000000
IDTVectoring: info=00000000 errcode=00000000
TSC Offset = 0xfffffec69836bc82
EPT pointer = 0x00000001d7a7901e
INFO: rcu_sched detected stalls on CPUs/tasks:
	(detected by 1, t=125002 jiffies, g=94253, c=94252, q=107)
All QSes seen, last rcu_sched kthread activity 124849 (4295369659-4295244810), jiffies_till_next_fqs=3, root ->qsmask 0x0
kworker/1:0     R  running task    23448    19      2 0x80000008
Workqueue: events perf_sched_delayed
Call Trace:
 <IRQ>
 sched_show_task.cold.85+0x27a/0x301 kernel/sched/core.c:5321
 print_other_cpu_stall.cold.77+0x92f/0x9d2 kernel/rcu/tree.c:1441
 check_cpu_stall.isra.60+0x706/0xf50 kernel/rcu/tree.c:1559
 __rcu_pending kernel/rcu/tree.c:3244 [inline]
 rcu_pending kernel/rcu/tree.c:3291 [inline]
 rcu_check_callbacks+0x237/0xcb0 kernel/rcu/tree.c:2646
 update_process_times+0x2d/0x70 kernel/time/timer.c:1636
 tick_sched_handle+0x9f/0x180 kernel/time/tick-sched.c:164
 tick_sched_timer+0x45/0x130 kernel/time/tick-sched.c:1274
 __run_hrtimer kernel/time/hrtimer.c:1398 [inline]
 __hrtimer_run_queues+0x3e3/0x10a0 kernel/time/hrtimer.c:1460
 hrtimer_interrupt+0x2f3/0x750 kernel/time/hrtimer.c:1518
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1025 [inline]
 smp_apic_timer_interrupt+0x15d/0x710 arch/x86/kernel/apic/apic.c:1050
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863
 </IRQ>
RIP: 0010:rep_nop arch/x86/include/asm/processor.h:658 [inline]
RIP: 0010:cpu_relax arch/x86/include/asm/processor.h:663 [inline]
RIP: 0010:csd_lock_wait kernel/smp.c:108 [inline]
RIP: 0010:smp_call_function_single+0x3e0/0x5b0 kernel/smp.c:302
Code: e8 03 4c 01 e8 c6 00 f8 48 89 44 24 28 e8 f8 06 0c 00 45 85 e4 74 6a 48 8b 44 24 28 48 83 c3 18 49 89 c4 e8 d2 05 0c 00 f3 90 <48> 89 d8 41 c6 04 24 04 48 c1 e8 03 42 0f b6 04 28 84 c0 74 08 3c 
RSP: 0018:ffff8801d9b57160 EFLAGS: 00000293 ORIG_RAX: ffffffffffffff13
RAX: ffff8801d9b48500 RBX: ffff8801d9b57258 RCX: ffffffff816d5092
RDX: 0000000000000000 RSI: ffffffff816d503e RDI: 0000000000000005
RBP: ffff8801d9b572b0 R08: ffff8801d9b48500 R09: ffffed003b5c5b18
R10: ffffed003b5c5b18 R11: ffff8801dae2d8c7 R12: ffffed003b36ae40
R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000001
 smp_call_function_many+0x870/0xa20 kernel/smp.c:434
 smp_call_function kernel/smp.c:492 [inline]
 on_each_cpu+0x46/0x1d0 kernel/smp.c:602
 text_poke_bp+0xb9/0x158 arch/x86/kernel/alternative.c:796
 __jump_label_transform.isra.0+0x43c/0x890 arch/x86/kernel/jump_label.c:102
 arch_jump_label_transform+0x2f/0x40 arch/x86/kernel/jump_label.c:110
 __jump_label_update+0x16e/0x1a0 kernel/jump_label.c:375
 jump_label_update+0x151/0x2e0 kernel/jump_label.c:760
 static_key_disable_cpuslocked+0x278/0x2f0 kernel/jump_label.c:171
 static_key_disable+0x1a/0x30 kernel/jump_label.c:179
 perf_sched_delayed+0x65/0x70 kernel/events/core.c:4322
 process_one_work+0xc1e/0x1b50 kernel/workqueue.c:2145
 worker_thread+0x1cc/0x1440 kernel/workqueue.c:2279
 kthread+0x345/0x410 kernel/kthread.c:240
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412
rcu_sched kthread starved for 124849 jiffies! g94253 c94252 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=0
RCU grace-period kthread stack dump:
rcu_sched       R  running task    22392    10      2 0x80000000
Call Trace:
 context_switch kernel/sched/core.c:2856 [inline]
 __schedule+0x801/0x1e30 kernel/sched/core.c:3498
 schedule+0xef/0x430 kernel/sched/core.c:3542
 schedule_timeout+0x138/0x240 kernel/time/timer.c:1801
 rcu_gp_kthread+0x700/0x1c80 kernel/rcu/tree.c:2179
 kthread+0x345/0x410 kernel/kthread.c:240
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412

Crashes (138):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2018/06/06 04:32 upstream 5037be168f0e 863a24bb .config console log report ci-upstream-kasan-gce-root
2018/03/25 02:32 upstream bcfc1f455466 2e9d9054 .config console log report ci-upstream-kasan-gce-386
2018/08/28 11:25 https://github.com/google/kmsan.git master 72cde92031e1 7ef1de9e .config console log report ci-upstream-kmsan-gce
2018/08/27 11:36 https://github.com/google/kmsan.git master 0cc51dc9a291 758cd203 .config console log report ci-upstream-kmsan-gce
2018/08/27 06:20 https://github.com/google/kmsan.git master 0cc51dc9a291 758cd203 .config console log report ci-upstream-kmsan-gce
2018/08/26 15:48 https://github.com/google/kmsan.git master 0cc51dc9a291 758cd203 .config console log report ci-upstream-kmsan-gce
2018/08/26 12:29 https://github.com/google/kmsan.git master 0cc51dc9a291 758cd203 .config console log report ci-upstream-kmsan-gce
2018/08/26 00:26 https://github.com/google/kmsan.git master 0cc51dc9a291 76e7c3df .config console log report ci-upstream-kmsan-gce
2018/08/25 13:52 https://github.com/google/kmsan.git master 0cc51dc9a291 9be5aa1d .config console log report ci-upstream-kmsan-gce
2018/08/24 07:11 https://github.com/google/kmsan.git master 0cc51dc9a291 95b5c82b .config console log report ci-upstream-kmsan-gce
2018/08/22 18:25 https://github.com/google/kmsan.git master 0cc51dc9a291 95b5c82b .config console log report ci-upstream-kmsan-gce
2018/08/22 02:03 https://github.com/google/kmsan.git master 0cc51dc9a291 95b5c82b .config console log report ci-upstream-kmsan-gce
2018/08/21 08:10 https://github.com/google/kmsan.git master 0cc51dc9a291 95b5c82b .config console log report ci-upstream-kmsan-gce
2018/08/19 10:05 https://github.com/google/kmsan.git master 0cc51dc9a291 2dc4378f .config console log report ci-upstream-kmsan-gce
2018/08/18 19:25 https://github.com/google/kmsan.git master 0cc51dc9a291 db1858f6 .config console log report ci-upstream-kmsan-gce
2018/08/18 03:48 https://github.com/google/kmsan.git master 0cc51dc9a291 db1858f6 .config console log report ci-upstream-kmsan-gce
2018/08/17 10:01 https://github.com/google/kmsan.git master 0cc51dc9a291 9ccc1d45 .config console log report ci-upstream-kmsan-gce
2018/08/15 05:04 https://github.com/google/kmsan.git master 0cc51dc9a291 0e6dcb88 .config console log report ci-upstream-kmsan-gce
2018/08/14 15:44 https://github.com/google/kmsan.git master 0cc51dc9a291 7a88b141 .config console log report ci-upstream-kmsan-gce
2018/08/12 21:30 https://github.com/google/kmsan.git master 0cc51dc9a291 7a88b141 .config console log report ci-upstream-kmsan-gce
2018/07/23 17:40 https://github.com/google/kmsan.git master d1c2a46a46f6 f69c5fcd .config console log report ci-upstream-kmsan-gce
2018/07/22 20:53 https://github.com/google/kmsan.git master d1c2a46a46f6 8cc079c3 .config console log report ci-upstream-kmsan-gce
2018/07/22 00:03 https://github.com/google/kmsan.git master d1c2a46a46f6 8cc079c3 .config console log report ci-upstream-kmsan-gce
2018/07/21 20:20 https://github.com/google/kmsan.git master d1c2a46a46f6 8cc079c3 .config console log report ci-upstream-kmsan-gce
2018/07/20 13:48 https://github.com/google/kmsan.git master cf8cd3cd03e2 49f35839 .config console log report ci-upstream-kmsan-gce
2018/07/20 09:04 https://github.com/google/kmsan.git master cf8cd3cd03e2 49f35839 .config console log report ci-upstream-kmsan-gce
2018/07/20 05:07 https://github.com/google/kmsan.git master cf8cd3cd03e2 49f35839 .config console log report ci-upstream-kmsan-gce
2018/07/19 15:45 https://github.com/google/kmsan.git master cf8cd3cd03e2 49f35839 .config console log report ci-upstream-kmsan-gce
2018/07/18 10:41 https://github.com/google/kmsan.git master 80ecacc456c1 809256c3 .config console log report ci-upstream-kmsan-gce
2018/07/17 18:47 https://github.com/google/kmsan.git master 80ecacc456c1 6d5bd5b5 .config console log report ci-upstream-kmsan-gce
2018/07/16 15:13 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/16 09:03 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/16 05:40 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/16 03:01 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/15 19:32 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/15 16:38 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/15 06:08 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/14 23:23 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/14 15:22 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/14 07:06 https://github.com/google/kmsan.git master 80ecacc456c1 92a49505 .config console log report ci-upstream-kmsan-gce
2018/07/12 11:41 https://github.com/google/kmsan.git master b64f7ec04e12 06c33b3a .config console log report ci-upstream-kmsan-gce
2018/07/12 09:48 https://github.com/google/kmsan.git master b64f7ec04e12 2e0e3130 .config console log report ci-upstream-kmsan-gce
2018/07/11 11:25 https://github.com/google/kmsan.git master b64f7ec04e12 2e0e3130 .config console log report ci-upstream-kmsan-gce
2018/07/11 06:14 https://github.com/google/kmsan.git master b64f7ec04e12 2e0e3130 .config console log report ci-upstream-kmsan-gce
2018/07/06 11:01 https://github.com/google/kmsan.git master 9c9df9f275f0 18403e65 .config console log report ci-upstream-kmsan-gce
2018/07/05 15:22 https://github.com/google/kmsan.git master c1adbc427188 f525fd72 .config console log report ci-upstream-kmsan-gce
2018/07/01 19:22 https://github.com/google/kmsan.git master 123906095e30 dba0b50e .config console log report ci-upstream-kmsan-gce
2018/06/30 20:47 https://github.com/google/kmsan.git master 123906095e30 dba0b50e .config console log report ci-upstream-kmsan-gce
* Struck through repros no longer work on HEAD.