syzbot


INFO: rcu detected stall in __run_timers (6)

Status: auto-closed as invalid on 2022/06/02 21:46
Reported-by: syzbot+@syzkaller.appspotmail.com
First crash: 289d, last: 273d
similar bugs (5):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in __run_timers (4) 4 1031d 1045d 0/24 auto-closed as invalid on 2020/05/06 02:55
upstream INFO: rcu detected stall in __run_timers 2 1545d 1547d 0/24 closed as invalid on 2018/09/11 10:51
upstream INFO: rcu detected stall in __run_timers (2) 58 1417d 1541d 0/24 closed as dup on 2019/01/02 16:40
upstream INFO: rcu detected stall in __run_timers (3) 4 1157d 1179d 0/24 auto-closed as invalid on 2019/12/31 08:04
upstream INFO: rcu detected stall in __run_timers (5) C error error 63 305d 818d 0/24 closed as invalid on 2022/02/08 10:39

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-....: (10475 ticks this GP) idle=8c3/1/0x4000000000000000 softirq=242131/242131 fqs=2483 
	(t=10500 jiffies g=345101 q=921)
NMI backtrace for cpu 1
CPU: 1 PID: 13159 Comm: kworker/u4:8 Not tainted 5.17.0-rc4-syzkaller-00051-gc5d9ae265b10 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_dump_cpu_stacks+0x25e/0x3f0 kernel/rcu/tree_stall.h:343
 print_cpu_stall kernel/rcu/tree_stall.h:604 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:688 [inline]
 rcu_pending kernel/rcu/tree.c:3919 [inline]
 rcu_sched_clock_irq.cold+0x402/0x755 kernel/rcu/tree.c:2617
 update_process_times+0x16d/0x200 kernel/time/timer.c:1785
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1428
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1097
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:160 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x25/0x40 kernel/locking/spinlock.c:202
Code: 0f 1f 44 00 00 55 48 8b 74 24 08 48 89 fd 48 83 c7 18 e8 4e f7 0c f8 48 89 ef e8 a6 75 0d f8 e8 31 72 2e f8 fb bf 01 00 00 00 <e8> c6 0a 00 f8 65 8b 05 0f c4 b1 76 85 c0 74 02 5d c3 e8 7b 99 af
RSP: 0018:ffffc90000dc0e08 EFLAGS: 00000202
RAX: 00000000033b181a RBX: ffffffff874dbfa0 RCX: 1ffffffff1ff914e
RDX: 0000000000000000 RSI: 0000000000000102 RDI: 0000000000000001
RBP: ffff8880b9d28480 R08: 0000000000000001 R09: ffffffff8ffc1957
R10: 0000000000000001 R11: 0000000000000000 R12: ffffc90000dc0e98
R13: 0000000000000000 R14: dffffc0000000000 R15: ffff8880b9d28480
 expire_timers kernel/time/timer.c:1465 [inline]
 __run_timers.part.0+0x66c/0xa30 kernel/time/timer.c:1734
 __run_timers kernel/time/timer.c:1715 [inline]
 run_timer_softirq+0xb3/0x1d0 kernel/time/timer.c:1747
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x123/0x180 kernel/softirq.c:637
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649
 common_interrupt+0xa9/0xc0 arch/x86/kernel/irq.c:240
 </IRQ>
 <TASK>
 asm_common_interrupt+0x1e/0x40 arch/x86/include/asm/idtentry.h:629
RIP: 0010:spin_unlock include/linux/spinlock.h:389 [inline]
RIP: 0010:__text_poke+0x5cf/0x8c0 arch/x86/kernel/alternative.c:1059
Code: 85 ba 01 00 00 48 83 7c 24 08 00 0f 85 26 01 00 00 9c 58 f6 c4 02 0f 85 a5 01 00 00 48 83 7c 24 08 00 74 01 fb 48 8b 7c 24 60 <e8> 8c 7f 25 08 48 8b 4c 24 18 48 b8 00 00 00 00 00 fc ff df 48 c7
RSP: 0018:ffffc90000ca79f8 EFLAGS: 00000206
RAX: 0000000000000006 RBX: 0000000000000007 RCX: 1ffffffff1ff914e
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff888010db5120
RBP: ffff8880118d9558 R08: 0000000000000001 R09: ffffffff8ffc1957
R10: 0000000000000001 R11: 000000000000003f R12: ffffffff81c01d6d
R13: 0000000000000001 R14: 0000000000000d6e R15: ffffffff8bcb43e0
 text_poke arch/x86/kernel/alternative.c:1083 [inline]
 text_poke_bp_batch+0x134/0x6f0 arch/x86/kernel/alternative.c:1297
 text_poke_flush arch/x86/kernel/alternative.c:1470 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:1467 [inline]
 text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1477
 arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x32f/0x410 kernel/jump_label.c:830
 static_key_disable_cpuslocked+0x152/0x1b0 kernel/jump_label.c:207
 static_key_disable+0x16/0x20 kernel/jump_label.c:215
 toggle_allocation_gate mm/kfence/core.c:749 [inline]
 toggle_allocation_gate+0x183/0x390 mm/kfence/core.c:727
 process_one_work+0x9ac/0x1650 kernel/workqueue.c:2307
 worker_thread+0x657/0x1110 kernel/workqueue.c:2454
 kthread+0x2e9/0x3a0 kernel/kthread.c:377
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
----------------
Code disassembly (best guess):
   0:	0f 1f 44 00 00       	nopl   0x0(%rax,%rax,1)
   5:	55                   	push   %rbp
   6:	48 8b 74 24 08       	mov    0x8(%rsp),%rsi
   b:	48 89 fd             	mov    %rdi,%rbp
   e:	48 83 c7 18          	add    $0x18,%rdi
  12:	e8 4e f7 0c f8       	callq  0xf80cf765
  17:	48 89 ef             	mov    %rbp,%rdi
  1a:	e8 a6 75 0d f8       	callq  0xf80d75c5
  1f:	e8 31 72 2e f8       	callq  0xf82e7255
  24:	fb                   	sti
  25:	bf 01 00 00 00       	mov    $0x1,%edi
* 2a:	e8 c6 0a 00 f8       	callq  0xf8000af5 <-- trapping instruction
  2f:	65 8b 05 0f c4 b1 76 	mov    %gs:0x76b1c40f(%rip),%eax        # 0x76b1c445
  36:	85 c0                	test   %eax,%eax
  38:	74 02                	je     0x3c
  3a:	5d                   	pop    %rbp
  3b:	c3                   	retq
  3c:	e8                   	.byte 0xe8
  3d:	7b 99                	jnp    0xffffffd8
  3f:	af                   	scas   %es:(%rdi),%eax

Crashes (2):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-kasan-gce 2022/02/16 11:35 upstream c5d9ae265b10 8b9ca619 .config log report info INFO: rcu detected stall in __run_timers
ci-upstream-net-kasan-gce 2022/03/04 21:37 net-next f9f52c347428 45a13a73 .config log report info INFO: rcu detected stall in __run_timers
* Struck through repros no longer work on HEAD.