syzbot


INFO: rcu detected stall in sync_hw_clock

Status: upstream: reported on 2025/01/10 04:32
Reported-by: syzbot+75a48db91652b630f27d@syzkaller.appspotmail.com
First crash: 25d, last: 22d
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in sync_hw_clock origin:upstream C 2 26d 397d 0/3 upstream: reported C repro on 2024/01/04 12:38
upstream INFO: rcu detected stall in sync_hw_clock kernel 3 752d 801d 0/28 auto-obsoleted due to no activity on 2023/04/21 23:58
upstream BUG: soft lockup in sync_hw_clock efi C 1 293d 289d 0/28 auto-obsoleted due to no activity on 2024/07/26 14:56

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=c204/1/0x4000000000000000 softirq=26712/26712 fqs=2
	(detected by 1, t=10503 jiffies, g=35909, q=179 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 125 Comm: kworker/0:2 Not tainted 6.1.124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: events_freezable_power_ sync_hw_clock
RIP: 0010:__lock_acquire+0xf20/0x1f80 kernel/locking/lockdep.c:5044
Code: 48 c1 e5 20 48 c7 c0 78 40 9b 8e 48 c1 e8 03 0f b6 04 38 84 c0 0f 85 6d 0b 00 00 48 09 dd 83 3d 4d 89 30 0d 00 4c 8b 7c 24 30 <45> 89 ee 4c 8b 6c 24 48 0f 85 1a 03 00 00 48 83 7c 24 40 00 0f 84
RSP: 0018:ffffc90000007a80 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 00000000db16ba9e RCX: 000000000018fa7d
RDX: 00000000faf9e11e RSI: 0000000029bd22a8 RDI: dffffc0000000000
RBP: 4b21b045db16ba9e R08: dffffc0000000000 R09: fffffbfff2249c4e
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff88801cb9a898
R13: 0000000000000001 R14: ffff88801cb9a910 R15: 1ffff11003973522
FS:  0000000000000000(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020001000 CR3: 0000000059d51000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
 __raw_spin_lock_irq include/linux/spinlock_api_smp.h:119 [inline]
 _raw_spin_lock_irq+0xcf/0x110 kernel/locking/spinlock.c:170
 __run_hrtimer kernel/time/hrtimer.c:1693 [inline]
 __hrtimer_run_queues+0x6d3/0xe50 kernel/time/hrtimer.c:1753
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1815
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1107 [inline]
 __sysvec_apic_timer_interrupt+0x158/0x5b0 arch/x86/kernel/apic/apic.c:1124
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1118 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1118
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:691
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xd4/0x130 kernel/locking/spinlock.c:194
Code: 9c 8f 44 24 20 42 80 3c 23 00 74 08 4c 89 f7 e8 92 22 25 f7 f6 44 24 21 02 75 4e 41 f7 c7 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> f7 7f a1 f6 65 8b 05 98 5f 45 75 85 c0 74 3f 48 c7 04 24 0e 36
RSP: 0018:ffffc90002d27aa0 EFLAGS: 00000206
RAX: 132c0818b1c16c00 RBX: 1ffff920005a4f58 RCX: ffffffff816b126a
RDX: dffffc0000000000 RSI: ffffffff8b0c0340 RDI: 0000000000000001
RBP: ffffc90002d27b30 R08: dffffc0000000000 R09: fffffbfff2249c86
R10: 0000000000000000 R11: dffffc0000000001 R12: dffffc0000000000
R13: 1ffff920005a4f54 R14: ffffc90002d27ac0 R15: 0000000000000246
 hrtimer_start include/linux/hrtimer.h:420 [inline]
 sched_sync_hw_clock kernel/time/ntp.c:519 [inline]
 sync_hw_clock+0x418/0x520 kernel/time/ntp.c:660
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: rcu_preempt kthread starved for 10495 jiffies! g35909 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:26712 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x143f/0x4570 kernel/sched/core.c:6558
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1965
 rcu_gp_fqs_loop+0x2d2/0x1150 kernel/rcu/tree.c:1706
 rcu_gp_kthread+0xa3/0x3b0 kernel/rcu/tree.c:1905
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 7925 Comm: syz.1.1161 Not tainted 6.1.124-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:csd_lock_wait kernel/smp.c:424 [inline]
RIP: 0010:smp_call_function_many_cond+0x1fae/0x3460 kernel/smp.c:998
Code: 2f 44 89 ee 83 e6 01 31 ff e8 0e 44 0b 00 41 83 e5 01 49 bd 00 00 00 00 00 fc ff df 75 0a e8 99 40 0b 00 e9 1b ff ff ff f3 90 <42> 0f b6 04 2b 84 c0 75 14 41 f7 07 01 00 00 00 0f 84 fe fe ff ff
RSP: 0018:ffffc9000c53f100 EFLAGS: 00000293
RAX: ffffffff817f5b89 RBX: 1ffff110171c8595 RCX: ffff88802ca25940
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc9000c53f4e0 R08: ffffffff817f5b52 R09: ffffed10171e7753
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000800000000
R13: dffffc0000000000 R14: 0000000000000000 R15: ffff8880b8e42ca8
FS:  0000000000000000(0000) GS:ffff8880b8f00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555575b91808 CR3: 000000005ff28000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3b/0x80 kernel/smp.c:1166
 __flush_tlb_multi arch/x86/include/asm/paravirt.h:87 [inline]
 flush_tlb_multi arch/x86/mm/tlb.c:924 [inline]
 flush_tlb_mm_range+0x353/0x590 arch/x86/mm/tlb.c:1010
 tlb_flush arch/x86/include/asm/tlb.h:20 [inline]
 tlb_flush_mmu_tlbonly+0x1ab/0x410 include/asm-generic/tlb.h:430
 tlb_flush_mmu+0x28/0x210 mm/mmu_gather.c:260
 tlb_finish_mmu+0xce/0x1f0 mm/mmu_gather.c:361
 exit_mmap+0x3b3/0x980 mm/mmap.c:3250
 __mmput+0x115/0x3c0 kernel/fork.c:1200
 exit_mm+0x226/0x300 kernel/exit.c:565
 do_exit+0x9f1/0x26a0 kernel/exit.c:858
 do_group_exit+0x202/0x2b0 kernel/exit.c:1021
 get_signal+0x16f7/0x17d0 kernel/signal.c:2871
 arch_do_signal_or_restart+0xb0/0x1a10 arch/x86/kernel/signal.c:871
 exit_to_user_mode_loop+0x6a/0x100 kernel/entry/common.c:174
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:210
 __syscall_exit_to_user_mode_work kernel/entry/common.c:292 [inline]
 syscall_exit_to_user_mode+0x60/0x270 kernel/entry/common.c:303
 do_syscall_64+0x47/0xb0 arch/x86/entry/common.c:87
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f706f985d29
Code: Unable to access opcode bytes at 0x7f706f985cff.
RSP: 002b:00007f70708140e8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: 0000000000000001 RBX: 00007f706fb75fa8 RCX: 00007f706f985d29
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00007f706fb75fac
RBP: 00007f706fb75fa0 R08: 0000000000000010 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 00007f706fb75fac
R13: 0000000000000000 R14: 00007ffdf78e5360 R15: 00007ffdf78e5448
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/01/13 12:57 linux-6.1.y c63962be84ef 249ceea9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in sync_hw_clock
2025/01/10 04:31 linux-6.1.y c63962be84ef 67d7ec0a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in sync_hw_clock
* Struck through repros no longer work on HEAD.