syzbot


INFO: rcu detected stall in sys_clock_nanosleep (3)

Status: closed as invalid on 2022/02/08 10:00
Subsystems: kernel
[Documentation on labels]
First crash: 883d, last: 873d
Similar bugs (8)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in sys_clock_nanosleep 1 227d 227d 0/3 auto-obsoleted due to no activity on 2023/12/20 18:13
linux-6.1 INFO: rcu detected stall in sys_clock_nanosleep (2) 2 41d 106d 0/3 upstream: reported on 2024/01/10 22:07
upstream INFO: rcu detected stall in sys_clock_nanosleep (5) kernel C error done 25 5d02h 253d 0/26 upstream: reported C repro on 2023/08/17 09:25
upstream INFO: rcu detected stall in sys_clock_nanosleep (4) kernel 6 576d 696d 0/26 auto-obsoleted due to no activity on 2022/12/31 21:43
upstream INFO: rcu detected stall in sys_clock_nanosleep kernel 1 1128d 1128d 0/26 auto-closed as invalid on 2021/06/22 18:23
linux-6.1 INFO: rcu detected stall in sys_clock_nanosleep 1 403d 403d 0/3 auto-obsoleted due to no activity on 2023/07/17 12:13
upstream INFO: rcu detected stall in sys_clock_nanosleep (2) kernel 1 1006d 1006d 0/26 auto-closed as invalid on 2021/10/22 22:53
linux-5.15 INFO: rcu detected stall in sys_clock_nanosleep (2) 2 30d 103d 0/3 upstream: reported on 2024/01/13 21:31

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=297/1/0x4000000000000000 softirq=52612/52613 fqs=39 
	(detected by 1, t=10502 jiffies, g=81881, q=312)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 24893 Comm: syz-executor.5 Not tainted 5.16.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:memory_is_poisoned_n mm/kasan/generic.c:135 [inline]
RIP: 0010:memory_is_poisoned mm/kasan/generic.c:159 [inline]
RIP: 0010:check_region_inline mm/kasan/generic.c:180 [inline]
RIP: 0010:kasan_check_range+0x11c/0x180 mm/kasan/generic.c:189
Code: eb 0c 48 83 c2 01 4c 89 d8 4c 39 da 74 0e 80 3a 00 74 ef 49 8d 04 2c 48 85 d2 75 0b 48 89 da 48 29 c2 e9 55 ff ff ff 49 39 d2 <75> 17 49 0f be 02 41 83 e1 07 49 39 c1 7d 0a 5b b8 01 00 00 00 5d
RSP: 0018:ffffc90000007d88 EFLAGS: 00000046
RAX: 0000000000000001 RBX: fffff52000000fbb RCX: ffffffff815d3180
RDX: fffff52000000fba RSI: 0000000000000004 RDI: ffffc90000007dd0
RBP: fffff52000000fba R08: 0000000000000001 R09: ffffc90000007dd3
R10: fffff52000000fba R11: 0000000000000000 R12: ffff8880b9c2a408
R13: ffff8880b9c2a410 R14: ffff8880b9c2a400 R15: 0000000000000001
FS:  0000555556036400(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbaa2a7d603 CR3: 0000000075641000 CR4: 0000000000350ef0
Call Trace:
 <IRQ>
 instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
 atomic_try_cmpxchg_acquire include/linux/atomic/atomic-instrumented.h:512 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:82 [inline]
 do_raw_spin_lock+0x120/0x2b0 kernel/locking/spinlock_debug.c:115
 __run_hrtimer kernel/time/hrtimer.c:1689 [inline]
 __hrtimer_run_queues+0x243/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:finish_task_switch.isra.0+0x323/0xb80 kernel/sched/core.c:4852
Code: f3 85 f3 07 85 c0 75 02 0f 0b 48 85 db 0f 85 25 01 00 00 4c 89 f7 e8 fc f7 ff ff e8 c7 46 2d 00 fb 65 48 8b 1c 25 40 70 02 00 <48> 8d bb 80 15 00 00 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1
RSP: 0018:ffffc9000bf67ae8 EFLAGS: 00000206
RAX: 0000000000033459 RBX: ffff888041a60000 RCX: 1ffffffff1b22051
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc9000bf67b30 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff817def48 R11: 0000000000000001 R12: ffff8880b9c3a7d8
R13: ffff88807ef01d00 R14: ffff8880b9c39a80 R15: ffff8880b9c3a4b8
 context_switch kernel/sched/core.c:4975 [inline]
 __schedule+0xaa2/0x4940 kernel/sched/core.c:6253
 schedule+0xd2/0x260 kernel/sched/core.c:6326
 freezable_schedule include/linux/freezer.h:172 [inline]
 do_nanosleep+0x24e/0x690 kernel/time/hrtimer.c:2044
 hrtimer_nanosleep+0x1f9/0x4a0 kernel/time/hrtimer.c:2097
 common_nsleep+0xa2/0xc0 kernel/time/posix-timers.c:1227
 __do_sys_clock_nanosleep kernel/time/posix-timers.c:1267 [inline]
 __se_sys_clock_nanosleep kernel/time/posix-timers.c:1245 [inline]
 __x64_sys_clock_nanosleep+0x2f4/0x430 kernel/time/posix-timers.c:1245
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f4a434d2811
Code: 24 0c 89 3c 24 48 89 4c 24 18 e8 aa e7 ff ff 4c 8b 54 24 18 48 8b 54 24 10 41 89 c0 8b 74 24 0c 8b 3c 24 b8 e6 00 00 00 0f 05 <44> 89 c7 48 89 04 24 e8 e3 e7 ff ff 48 8b 04 24 eb 97 66 2e 0f 1f
RSP: 002b:00007fff8cbe47d0 EFLAGS: 00000293 ORIG_RAX: 00000000000000e6
RAX: ffffffffffffffda RBX: 000000000000000f RCX: 00007f4a434d2811
RDX: 00007fff8cbe4810 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00007fff8cbe489c R08: 0000000000000000 R09: 00007fff8cbf2080
R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000032
R13: 000000000008497e R14: 0000000000000004 R15: 00007fff8cbe4900
 </TASK>
rcu: rcu_preempt kthread starved for 10424 jiffies! g81881 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28688 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4972 [inline]
 __schedule+0xa9a/0x4940 kernel/sched/core.c:6253
 schedule+0xd2/0x260 kernel/sched/core.c:6326
 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
 rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1955
 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2128
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 25326 Comm: syz-executor.5 Not tainted 5.16.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_check_gp_kthread_starvation.cold+0x1fb/0x200 kernel/rcu/tree_stall.h:481
 print_other_cpu_stall kernel/rcu/tree_stall.h:586 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:729 [inline]
 rcu_pending kernel/rcu/tree.c:3878 [inline]
 rcu_sched_clock_irq+0x2125/0x2200 kernel/rcu/tree.c:2597
 update_process_times+0x16d/0x200 kernel/time/timer.c:1785
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1421
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_many_cond+0x452/0xc20 kernel/smp.c:969
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 f4 4c 89 f5 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 80 75 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 33 06 00 00 8b 43 08 31
RSP: 0018:ffffc9000416f9d8 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9c41d40 RCX: 0000000000000000
RDX: ffff8880793f5700 RSI: ffffffff816c3540 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff816c3566 R11: 0000000000000000 R12: ffffed10173883a9
R13: 0000000000000000 R14: ffff8880b9c41d48 R15: 0000000000000001
 on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1135
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:1112 [inline]
 text_poke_bp_batch+0x1b3/0x560 arch/x86/kernel/alternative.c:1297
 text_poke_flush arch/x86/kernel/alternative.c:1451 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:1448 [inline]
 text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1458
 arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x1d5/0x430 kernel/jump_label.c:830
 static_key_disable_cpuslocked+0x152/0x1b0 kernel/jump_label.c:207
 static_key_disable+0x16/0x20 kernel/jump_label.c:215
 tracepoint_remove_func kernel/tracepoint.c:417 [inline]
 tracepoint_probe_unregister+0x632/0xc30 kernel/tracepoint.c:551
 trace_event_reg+0x181/0x350 kernel/trace/trace_events.c:517
 perf_trace_event_unreg.isra.0+0xac/0x2b0 kernel/trace/trace_event_perf.c:162
 perf_trace_destroy+0xb5/0xf0 kernel/trace/trace_event_perf.c:243
 _free_event+0x2ee/0x1390 kernel/events/core.c:4982
 put_event kernel/events/core.c:5076 [inline]
 perf_event_release_kernel+0x6f2/0x940 kernel/events/core.c:5191
 perf_release+0x33/0x40 kernel/events/core.c:5201
 __fput+0x286/0x9f0 fs/file_table.c:280
 task_work_run+0xdd/0x1a0 kernel/task_work.c:164
 tracehook_notify_resume include/linux/tracehook.h:189 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:175 [inline]
 exit_to_user_mode_prepare+0x27e/0x290 kernel/entry/common.c:207
 __syscall_exit_to_user_mode_work kernel/entry/common.c:289 [inline]
 syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:300
 do_syscall_64+0x42/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f4a4346072b
Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8 63 fc ff ff 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 35 44 89 c7 89 44 24 0c e8 a1 fc ff ff 8b 44
RSP: 002b:00007fff8cbe4650 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00007f4a4346072b
RDX: 0000001b2f820000 RSI: 00007f4a43425000 RDI: 0000000000000011
RBP: 0000000000000012 R08: 0000000000000000 R09: 00007fff8cbf2080
R10: 00000000000008d4 R11: 0000000000000293 R12: 00007f4a435c2860
R13: 00007f4a435c2860 R14: 0000000000000000 R15: 000000000008494f
 </TASK>
----------------
Code disassembly (best guess):
   0:	eb 0c                	jmp    0xe
   2:	48 83 c2 01          	add    $0x1,%rdx
   6:	4c 89 d8             	mov    %r11,%rax
   9:	4c 39 da             	cmp    %r11,%rdx
   c:	74 0e                	je     0x1c
   e:	80 3a 00             	cmpb   $0x0,(%rdx)
  11:	74 ef                	je     0x2
  13:	49 8d 04 2c          	lea    (%r12,%rbp,1),%rax
  17:	48 85 d2             	test   %rdx,%rdx
  1a:	75 0b                	jne    0x27
  1c:	48 89 da             	mov    %rbx,%rdx
  1f:	48 29 c2             	sub    %rax,%rdx
  22:	e9 55 ff ff ff       	jmpq   0xffffff7c
  27:	49 39 d2             	cmp    %rdx,%r10
* 2a:	75 17                	jne    0x43 <-- trapping instruction
  2c:	49 0f be 02          	movsbq (%r10),%rax
  30:	41 83 e1 07          	and    $0x7,%r9d
  34:	49 39 c1             	cmp    %rax,%r9
  37:	7d 0a                	jge    0x43
  39:	5b                   	pop    %rbx
  3a:	b8 01 00 00 00       	mov    $0x1,%eax
  3f:	5d                   	pop    %rbp

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/12/04 18:13 upstream bbef3c7a63d2 a617004c .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_clock_nanosleep
2021/11/28 11:56 upstream 3498e7f2bb41 63eeac02 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_clock_nanosleep
2021/11/24 23:50 upstream 5f53fa508db0 545ab074 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_clock_nanosleep
* Struck through repros no longer work on HEAD.