syzbot

 sign-in | mailing list | source | docs
🐞 Open [717] 🐞 Fixed [181] 🐞 Invalid [640] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in call_timer_fn

Status: auto-closed as invalid on 2020/08/29 09:07
Reported-by: syzbot+a9853ca066b9c66386a9@syzkaller.appspotmail.com
First crash: 1573d, last: 1454d
▶ ▼ Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: rcu detected stall in call_timer_fn (2) 16 1267d 1541d 0/1 auto-closed as invalid on 2021/03/04 12:30
linux-4.14 INFO: rcu detected stall in call_timer_fn (2) 1 1329d 1329d 0/1 auto-closed as invalid on 2020/12/31 22:00
upstream INFO: rcu detected stall in call_timer_fn (2) kernel C done error 47 826d 1683d 0/26 closed as invalid on 2022/02/08 10:46
upstream INFO: rcu detected stall in call_timer_fn bpf net 2 2020d 2033d 0/26 closed as dup on 2019/01/02 16:26
linux-4.19 INFO: rcu detected stall in call_timer_fn 1 1681d 1681d 0/1 auto-closed as invalid on 2020/01/15 18:43
upstream INFO: rcu detected stall in call_timer_fn (4) mm 1 84d 84d 0/26 auto-obsoleted due to no activity on 2024/03/31 10:44
upstream INFO: rcu detected stall in call_timer_fn (3) kernel 3 566d 581d 0/26 auto-obsoleted due to no activity on 2023/01/12 18:45

Sample crash report:
INFO: rcu_preempt self-detected stall on CPU
	1-...: (1 GPs behind) idle=6f6/140000000000002/0 softirq=106116/106117 fqs=11 
	 (t=10500 jiffies g=31394 c=31393 q=123)
rcu_preempt kthread starved for 10228 jiffies! g31394 c31393 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=0
rcu_preempt     R  running task    29728     8      2 0x80000000
Call Trace:
 schedule+0x8d/0x1b0 kernel/sched/core.c:3428
 schedule_timeout+0x43c/0xe40 kernel/time/timer.c:1746
 rcu_gp_kthread+0xb8e/0x1ce0 kernel/rcu/tree.c:2255
 kthread+0x30d/0x420 kernel/kthread.c:232
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
NMI backtrace for cpu 1
CPU: 1 PID: 10721 Comm: syz-executor.0 Not tainted 4.14.177-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x13e/0x194 lib/dump_stack.c:58
 nmi_cpu_backtrace.cold+0x57/0x93 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x139/0x17e lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
 rcu_dump_cpu_stacks+0x16d/0x1b8 kernel/rcu/tree.c:1396
 print_cpu_stall kernel/rcu/tree.c:1542 [inline]
 check_cpu_stall kernel/rcu/tree.c:1610 [inline]
 __rcu_pending kernel/rcu/tree.c:3390 [inline]
 rcu_pending kernel/rcu/tree.c:3452 [inline]
 rcu_check_callbacks.cold+0x490/0xdaa kernel/rcu/tree.c:2792
 update_process_times+0x29/0x60 kernel/time/timer.c:1590
 tick_sched_handle+0x7d/0x150 kernel/time/tick-sched.c:165
 tick_sched_timer+0x3e/0x120 kernel/time/tick-sched.c:1223
 __run_hrtimer kernel/time/hrtimer.c:1223 [inline]
 __hrtimer_run_queues+0x25c/0xbb0 kernel/time/hrtimer.c:1287
 hrtimer_interrupt+0x1e6/0x5e0 kernel/time/hrtimer.c:1321
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x117/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x8f/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:779 [inline]
RIP: 0010:lock_acquire+0x1ec/0x3f0 kernel/locking/lockdep.c:4001
RSP: 0018:ffff8880aeb07d20 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fa2cd1 RBX: ffff888209ac0580 RCX: 00000000aa568565
RDX: dffffc0000000000 RSI: ffff888209ac0e30 RDI: 0000000000000286
RBP: ffff8880aeb07db0 R08: 0000000000007c5f R09: ffffffff8a09db38
R10: ffff888209ac0e30 R11: ffff888209ac0580 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000
 call_timer_fn+0x102/0x650 kernel/time/timer.c:1276
 expire_timers kernel/time/timer.c:1318 [inline]
 __run_timers kernel/time/timer.c:1636 [inline]
 __run_timers kernel/time/timer.c:1604 [inline]
 run_timer_softirq+0x52a/0x1390 kernel/time/timer.c:1649
 __do_softirq+0x254/0x9bf kernel/softirq.c:288
 invoke_softirq kernel/softirq.c:368 [inline]
 irq_exit+0x15b/0x1a0 kernel/softirq.c:409
 exiting_irq arch/x86/include/asm/apic.h:648 [inline]
 smp_apic_timer_interrupt+0x141/0x5e0 arch/x86/kernel/apic/apic.c:1102
 apic_timer_interrupt+0x8f/0xa0 arch/x86/entry/entry_64.S:792
 </IRQ>
RIP: 0010:get_current arch/x86/include/asm/current.h:15 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x50 kernel/kcov.c:60
RSP: 0018:ffff88820d117758 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 8000000088288007 RBX: 8000000088288007 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 00007f390d50c000 RDI: 8000000088288007
RBP: 1ffff11041a22eec R08: 0000000000000001 R09: 0000000000000001
R10: ffff888209ac0e30 R11: ffff888209ac0580 R12: ffff888096b3d108
R13: 00007f390d50c000 R14: 0000000000000001 R15: 00007f390d50c000
 __pte_needs_invert arch/x86/include/asm/pgtable-invert.h:18 [inline]
 protnone_mask arch/x86/include/asm/pgtable-invert.h:24 [inline]
 pte_pfn arch/x86/include/asm/pgtable.h:195 [inline]
 _vm_normal_page+0xc2/0x390 mm/memory.c:833
 zap_pte_range mm/memory.c:1315 [inline]
 zap_pmd_range mm/memory.c:1444 [inline]
 zap_pud_range mm/memory.c:1473 [inline]
 zap_p4d_range mm/memory.c:1494 [inline]
 unmap_page_range+0x7f6/0x1930 mm/memory.c:1515
 unmap_single_vma+0x147/0x2b0 mm/memory.c:1560
 unmap_vmas+0x9d/0x160 mm/memory.c:1590
 exit_mmap+0x26d/0x4b0 mm/mmap.c:3056
 __mmput kernel/fork.c:930 [inline]
 mmput+0x103/0x420 kernel/fork.c:951
 exit_mm kernel/exit.c:545 [inline]
 do_exit+0x933/0x2b00 kernel/exit.c:845
 do_group_exit+0x100/0x310 kernel/exit.c:955
 get_signal+0x385/0x1ca0 kernel/signal.c:2423
 do_signal+0x7c/0x1690 arch/x86/kernel/signal.c:814
 exit_to_usermode_loop+0x159/0x220 arch/x86/entry/common.c:160
 prepare_exit_to_usermode arch/x86/entry/common.c:199 [inline]
 syscall_return_slowpath arch/x86/entry/common.c:270 [inline]
 do_syscall_64+0x4a3/0x640 arch/x86/entry/common.c:297
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45f66a
RSP: 002b:00007f390cf09c58 EFLAGS: 00000246 ORIG_RAX: 00000000000000e4
RAX: 0000000000000000 RBX: 000000000078bf08 RCX: 000000000045f66a
RDX: 00000000000f4240 RSI: 00007f390cf09c60 RDI: 0000000000000001
RBP: 000000000078bf00 R08: 0000000000000009 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 000000000078bf0c
R13: 000000000000006a R14: 00000000004c30b4 R15: 000000000078bf0c
INFO: rcu_sched detected stalls on CPUs/tasks:
	1-...: (1 GPs behind) idle=6f6/140000000000002/0 softirq=106108/106117 fqs=8 
	(detected by 0, t=10551 jiffies, g=16958, c=16957, q=5)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 10721 Comm: syz-executor.0 Not tainted 4.14.177-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff888209ac0580 task.stack: ffff88820d110000
RIP: 0010:lock_acquire+0x46/0x3f0 kernel/locking/lockdep.c:3990
RSP: 0018:ffff8880aeb073c0 EFLAGS: 00000807
RAX: dffffc0000000000 RBX: ffff888209ac0580 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff888209ac0e04
RBP: ffffffff8a70e5a0 R08: 0000000000000001 R09: 0000000000000000
R10: ffff88821fff7050 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000
FS:  00007f390cf0a700(0000) GS:ffff8880aeb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055aa75511000 CR3: 0000000007c6a000 CR4: 00000000001426e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x8c/0xbf kernel/locking/spinlock.c:160
 debug_object_deactivate lib/debugobjects.c:502 [inline]
 debug_object_deactivate+0xe7/0x350 lib/debugobjects.c:491
 debug_hrtimer_deactivate kernel/time/hrtimer.c:406 [inline]
 debug_deactivate kernel/time/hrtimer.c:453 [inline]
 __run_hrtimer kernel/time/hrtimer.c:1193 [inline]
 __hrtimer_run_queues+0x15b/0xbb0 kernel/time/hrtimer.c:1287
 hrtimer_interrupt+0x1e6/0x5e0 kernel/time/hrtimer.c:1321
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x117/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x8f/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:on_stack arch/x86/include/asm/stacktrace.h:41 [inline]
RIP: 0010:stack_access_ok+0x5c/0x130 arch/x86/kernel/unwind_orc.c:279
RSP: 0018:ffff8880aeb07750 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: dffffc0000000000 RBX: 1ffff11015d60eef RCX: 1ffff11015d60f13
RDX: 0000000000000008 RSI: ffff88820d1178f0 RDI: ffff8880aeb078a0
RBP: ffff88820d1178f0 R08: 0000000000000001 R09: ffff88820d1177d8
R10: ffff8880aeb078c5 R11: 0000000000058071 R12: ffff8880aeb07890
R13: dffffc0000000000 R14: ffff88820d117920 R15: 0000000000000001
 deref_stack_reg+0x56/0xc0 arch/x86/kernel/unwind_orc.c:289
 unwind_next_frame+0xeff/0x17a0 arch/x86/kernel/unwind_orc.c:474
 __save_stack_trace+0x6b/0xd0 arch/x86/kernel/stacktrace.c:44
 save_stack+0x32/0xa0 mm/kasan/kasan.c:447
 set_track mm/kasan/kasan.c:459 [inline]
 kasan_kmalloc mm/kasan/kasan.c:551 [inline]
 kasan_kmalloc+0xbf/0xe0 mm/kasan/kasan.c:529
 kmem_cache_alloc_node+0x148/0x7a0 mm/slab.c:3642
 __alloc_skb+0x9a/0x4c0 net/core/skbuff.c:193
 alloc_skb include/linux/skbuff.h:980 [inline]
 mrp_pdu_init net/802/mrp.c:303 [inline]
 mrp_pdu_append_vecattr_event+0x6f1/0xfa0 net/802/mrp.c:402
 mrp_attr_event+0x151/0x1e0 net/802/mrp.c:497
 mrp_mad_event+0x47/0x60 net/802/mrp.c:577
 mrp_join_timer+0x2f/0x80 net/802/mrp.c:594
 call_timer_fn+0x14a/0x650 kernel/time/timer.c:1279
 expire_timers kernel/time/timer.c:1318 [inline]
 __run_timers kernel/time/timer.c:1636 [inline]
 __run_timers kernel/time/timer.c:1604 [inline]
 run_timer_softirq+0x52a/0x1390 kernel/time/timer.c:1649
 __do_softirq+0x254/0x9bf kernel/softirq.c:288
 invoke_softirq kernel/softirq.c:368 [inline]
 irq_exit+0x15b/0x1a0 kernel/softirq.c:409
 exiting_irq arch/x86/include/asm/apic.h:648 [inline]
 smp_apic_timer_interrupt+0x141/0x5e0 arch/x86/kernel/apic/apic.c:1102
 apic_timer_interrupt+0x8f/0xa0 arch/x86/entry/entry_64.S:792
 </IRQ>
RIP: 0010:get_current arch/x86/include/asm/current.h:15 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x50 kernel/kcov.c:60
RSP: 0018:ffff88820d117758 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 8000000088288007 RBX: 8000000088288007 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 00007f390d50c000 RDI: 8000000088288007
RBP: 1ffff11041a22eec R08: 0000000000000001 R09: 0000000000000001
R10: ffff888209ac0e30 R11: ffff888209ac0580 R12: ffff888096b3d108
R13: 00007f390d50c000 R14: 0000000000000001 R15: 00007f390d50c000
 __pte_needs_invert arch/x86/include/asm/pgtable-invert.h:18 [inline]
 protnone_mask arch/x86/include/asm/pgtable-invert.h:24 [inline]
 pte_pfn arch/x86/include/asm/pgtable.h:195 [inline]
 _vm_normal_page+0xc2/0x390 mm/memory.c:833
 zap_pte_range mm/memory.c:1315 [inline]
 zap_pmd_range mm/memory.c:1444 [inline]
 zap_pud_range mm/memory.c:1473 [inline]
 zap_p4d_range mm/memory.c:1494 [inline]
 unmap_page_range+0x7f6/0x1930 mm/memory.c:1515
 unmap_single_vma+0x147/0x2b0 mm/memory.c:1560
 unmap_vmas+0x9d/0x160 mm/memory.c:1590
 exit_mmap+0x26d/0x4b0 mm/mmap.c:3056
 __mmput kernel/fork.c:930 [inline]
 mmput+0x103/0x420 kernel/fork.c:951
 exit_mm kernel/exit.c:545 [inline]
 do_exit+0x933/0x2b00 kernel/exit.c:845
 do_group_exit+0x100/0x310 kernel/exit.c:955
 get_signal+0x385/0x1ca0 kernel/signal.c:2423
 do_signal+0x7c/0x1690 arch/x86/kernel/signal.c:814
 exit_to_usermode_loop+0x159/0x220 arch/x86/entry/common.c:160
 prepare_exit_to_usermode arch/x86/entry/common.c:199 [inline]
 syscall_return_slowpath arch/x86/entry/common.c:270 [inline]
 do_syscall_64+0x4a3/0x640 arch/x86/entry/common.c:297
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45f66a
RSP: 002b:00007f390cf09c58 EFLAGS: 00000246 ORIG_RAX: 00000000000000e4
RAX: 0000000000000000 RBX: 000000000078bf08 RCX: 000000000045f66a
RDX: 00000000000f4240 RSI: 00007f390cf09c60 RDI: 0000000000000001
RBP: 000000000078bf00 R08: 0000000000000009 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 000000000078bf0c
R13: 000000000000006a R14: 00000000004c30b4 R15: 000000000078bf0c
Code: 41 89 f4 55 48 89 fd 53 65 48 8b 1c 25 40 ee 01 00 48 8d bb 84 08 00 00 48 83 ec 18 48 89 fa 4c 89 0c 24 48 c1 ea 03 0f b6 14 02 <48> 89 f8 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85 dd 02 00 00

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/05/01 09:06 linux-4.14.y 050272a0423e 143a10e9 .config console log report ci2-linux-4-14
2020/04/28 11:24 linux-4.14.y 050272a0423e e3ecea2e .config console log report ci2-linux-4-14
2020/03/08 11:24 linux-4.14.y 78d697fc93f9 2e9971bb .config console log report ci2-linux-4-14
2020/02/15 18:49 linux-4.14.y 98db2bf27b9e 5d7b90f1 .config console log report ci2-linux-4-14
2020/01/03 00:06 linux-4.14.y 4c5bf01e16a7 25a0186e .config console log report ci2-linux-4-14
* Struck through repros no longer work on HEAD.