syzbot

 sign-in | mailing list | source | docs
🐞 Open [869] ≡ Subsystems 🐞 Fixed [4877] 🐞 Invalid [11654] ⬇ Missing Backports [69] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in sys_clock_adjtime

Status: upstream: reported C repro on 2020/07/07 15:13
Subsystems: fs
[Documentation on labels]
Reported-by: syzbot+25b7addb06e92c482190@syzkaller.appspotmail.com
First crash: 1245d, last: 9d09h
Cause bisection: introduced by (bisect log) [release commit]:
commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date: Mon Oct 22 06:37:37 2018 +0000

  Linux 4.19

Crash: kernel panic: panic_on_warn set (log)
Repro: C syz .config
  
Fix bisection the fix commit could be any of (bisect log):
  7cc2a8ea1048 Merge tag 'block-5.8-2020-07-01' of git://git.kernel.dk/linux-block
  303392fd5c16 Merge tag 'leds-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds
   
▶ ▼ Discussions (3)
Title Replies (including bot) Last reply
[syzbot] Monthly fs report (Aug 2023) 0 (1) 2023/08/08 12:55
[syzbot] Monthly fs report (May 2023) 0 (1) 2023/05/06 14:02
INFO: rcu detected stall in sys_clock_adjtime 0 (1) 2020/07/07 15:13
▶ ▼ Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in sys_clock_adjtime 15 5d20h 255d 0/3 upstream: reported on 2023/03/24 02:32
linux-5.15 INFO: rcu detected stall in sys_clock_adjtime origin:upstream C 22 24d 242d 0/3 upstream: reported C repro on 2023/04/06 18:02
▶ ▼ Last patch testing requests (3)
Created Duration User Patch Repo Result
2023/09/13 00:53 15m retest repro upstream report log
2023/09/13 00:53 20m retest repro upstream OK log
2022/10/03 01:30 13m retest repro upstream report log
▶ ▼ Fix bisection attempts (10)
Created Duration User Patch Repo Result
2021/07/04 01:21 31m bisect fix upstream job log (2)
2021/06/04 00:45 26m bisect fix upstream job log (0) log
2021/05/04 12:27 30m bisect fix upstream job log (0) log
2021/04/04 09:29 31m bisect fix upstream job log (0) log
2021/03/04 23:42 29m bisect fix upstream job log (0) log
2021/01/30 18:05 25m bisect fix upstream job log (0) log
2020/12/31 17:38 25m bisect fix upstream job log (0) log
2020/12/01 17:14 24m bisect fix upstream job log (0) log
2020/11/01 16:44 29m bisect fix upstream job log (0) log
2020/09/17 03:07 25m bisect fix upstream job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=e3fc/1/0x4000000000000000 softirq=7998/8007 fqs=1
rcu: 	(detected by 1, t=10502 jiffies, g=5481, q=110 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5024 Comm: syz-executor164 Not tainted 6.5.0-rc6-syzkaller-00036-g4853c74bd7ab #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
RIP: 0010:taprio_set_budgets+0x19d/0x360 net/sched/sch_taprio.c:680
Code: ff 7f e9 88 00 00 00 e8 51 bc ae f8 48 8b 6c 24 18 48 8b 1c 24 48 89 e8 48 c1 e8 03 48 b9 00 00 00 00 00 fc ff df 80 3c 08 00 <74> 08 48 89 ef e8 39 8e 07 f9 48 69 6d 00 e8 03 00 00 48 89 df be
RSP: 0018:ffffc90000007c30 EFLAGS: 00000046
RAX: 1ffff110288f2780 RBX: ffff88801ab532e0 RCX: dffffc0000000000
RDX: 0000000080010002 RSI: 0000000000010000 RDI: 0000000000000000
RBP: ffff888144793c00 R08: ffffffff88dcfbc6 R09: fffff52000000f80
R10: dffffc0000000000 R11: fffff52000000f80 R12: 0000000000000000
R13: ffff888144793c00 R14: ffff888144793c00 R15: 0000000000000000
FS:  00005555574c7380(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 0000000072d43000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 advance_sched+0x993/0xc80 net/sched/sch_taprio.c:988
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x59f/0xd10 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x396/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1098 [inline]
 __sysvec_apic_timer_interrupt+0x13f/0x480 arch/x86/kernel/apic/apic.c:1115
 sysvec_apic_timer_interrupt+0x90/0xb0 arch/x86/kernel/apic/apic.c:1109
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xd8/0x140 kernel/locking/spinlock.c:194
Code: 9c 8f 44 24 20 42 80 3c 23 00 74 08 4c 89 f7 e8 ee b5 1d f7 f6 44 24 21 02 75 4e 41 f7 c7 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> b3 09 98 f6 65 8b 05 d4 e0 3c 75 85 c0 74 3f 48 c7 04 24 0e 36
RSP: 0018:ffffc90003b7f900 EFLAGS: 00000206
RAX: d9a652a42c06e200 RBX: 1ffff9200076ff24 RCX: ffffffff816bdd8a
RDX: dffffc0000000000 RSI: ffffffff8b0a97e0 RDI: 0000000000000001
RBP: ffffc90003b7f990 R08: ffffffff907b4307 R09: 1ffffffff20f6860
R10: dffffc0000000000 R11: fffffbfff20f6861 R12: dffffc0000000000
R13: 1ffff9200076ff20 R14: ffffc90003b7f920 R15: 0000000000000246
 clock_was_set+0x6a1/0x830 kernel/time/hrtimer.c:972
 timekeeping_inject_offset+0x663/0x710 kernel/time/timekeeping.c:1395
 do_adjtimex+0x515/0xab0 kernel/time/timekeeping.c:2444
 do_clock_adjtime kernel/time/posix-timers.c:1159 [inline]
 __do_sys_clock_adjtime kernel/time/posix-timers.c:1171 [inline]
 __se_sys_clock_adjtime kernel/time/posix-timers.c:1162 [inline]
 __x64_sys_clock_adjtime+0x1e0/0x290 kernel/time/posix-timers.c:1162
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7fc1cb60fee9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 d1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd6c04a108 EFLAGS: 00000246 ORIG_RAX: 0000000000000131
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fc1cb60fee9
RDX: 00007fc1cb60fee9 RSI: 0000000020000040 RDI: 0000000000000000
RBP: 00000000000f4240 R08: 0000000000000000 R09: 0000000100000000
R10: 0000000100000000 R11: 0000000000000246 R12: 00007ffd6c04a160
R13: 0000000000000001 R14: 00007ffd6c04a160 R15: 0000000000000003
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.575 msecs
rcu: rcu_preempt kthread starved for 10500 jiffies! g5481 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:26952 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5381 [inline]
 __schedule+0x1873/0x48f0 kernel/sched/core.c:6710
 schedule+0xc3/0x180 kernel/sched/core.c:6786
 schedule_timeout+0x1bd/0x310 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x2c6/0x1010 kernel/rcu/tree.c:1609
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1808
 kthread+0x2b8/0x350 kernel/kthread.c:389
 ret_from_fork+0x2e/0x60 arch/x86/kernel/process.c:145
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 47 Comm: kworker/u4:3 Not tainted 6.5.0-rc6-syzkaller-00036-g4853c74bd7ab #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:__sanitizer_cov_trace_pc+0x5c/0x60 kernel/kcov.c:225
Code: e0 15 00 00 83 fa 02 75 21 48 8b 91 e8 15 00 00 48 8b 32 48 8d 7e 01 8b 89 e4 15 00 00 48 39 cf 73 08 48 89 3a 48 89 44 f2 08 <c3> 0f 1f 00 f3 0f 1e fa 4c 8b 04 24 65 48 8b 15 a0 fc 77 7e 65 8b
RSP: 0018:ffffc90000b87778 EFLAGS: 00000293
RAX: ffffffff8180a437 RBX: 1ffff1101730897d RCX: ffff888015ee5940
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc90000b87980 R08: ffffffff8180a402 R09: 1ffffffff20f6860
R10: dffffc0000000000 R11: fffffbfff20f6861 R12: ffff8880b9844be8
R13: ffff8880b993d0c8 R14: ffff8880b993d0c0 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005555574c7650 CR3: 000000000d130000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 csd_lock_wait kernel/smp.c:296 [inline]
 smp_call_function_many_cond+0x1877/0x27d0 kernel/smp.c:835
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1003
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2001 [inline]
 text_poke_bp_batch+0x2d7/0x960 arch/x86/kernel/alternative.c:2211
 text_poke_flush arch/x86/kernel/alternative.c:2402 [inline]
 text_poke_finish+0x1a/0x30 arch/x86/kernel/alternative.c:2409
 arch_jump_label_transform_apply+0x17/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x132/0x250 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:831
 process_one_work+0x92c/0x12c0 kernel/workqueue.c:2600
 worker_thread+0xa63/0x1210 kernel/workqueue.c:2751
 kthread+0x2b8/0x350 kernel/kthread.c:389
 ret_from_fork+0x2e/0x60 arch/x86/kernel/process.c:145
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>

Crashes (43):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/08/17 12:31 upstream 4853c74bd7ab 74b106b6 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2020/07/07 05:48 upstream 7cc2a8ea1048 51095195 .config console log report syz C ci-upstream-kasan-gce-smack-root
2023/11/25 09:37 upstream 0f5cc96c367f 5b429f39 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/11/22 17:23 upstream c2d5304e6c64 03e12510 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/11/13 20:51 upstream b85ea95d0864 cb976f63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/27 17:26 upstream 750b95887e56 bf285f0c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/27 03:58 upstream 3a568e3a961b bf285f0c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/27 01:45 upstream 3a568e3a961b bf285f0c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/21 19:53 upstream 9c5d00cb7b6b 361b23dc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/02 23:28 upstream 8f1b4600373f 50b20e75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/10/02 05:23 upstream 8a749fd1a872 8e26a358 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/30 12:49 upstream 9f3ebbef746f 8e26a358 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/27 22:16 upstream 633b47cb009d 2895a507 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/27 17:02 upstream 0e945134b680 2895a507 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/26 22:19 upstream 50768a425b46 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/25 11:59 upstream 6465e260f487 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/23 02:20 upstream 8018e02a8703 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/21 23:53 upstream b5cbe7c00aa0 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/21 09:23 upstream 42dc814987c1 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/09/17 22:10 upstream e789286468a9 0b6a67ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/08/28 09:25 upstream 2dde18cd1d8f 03d9c195 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/08/25 12:22 upstream 4f9e7fabf864 03d9c195 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/08/22 21:00 upstream 53663f4103ff b81ca3f6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/08/20 08:14 upstream 9e6c269de404 d216d8a0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/07/30 01:07 upstream 12214540ad87 92476829 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/07/24 11:19 upstream 6eaae1980760 06460670 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/07/04 09:55 upstream 24be4d0b46bb 6e553898 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/06/28 12:55 upstream 6f612579be9d 8064cb02 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/05/22 10:06 upstream 44c026a73be8 4bce1a3e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/05/18 10:25 upstream 1b66c114d161 3bb7af1d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/05/05 04:57 upstream 3c4aa4434377 518a39a6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/03/14 14:20 upstream fc89d7fb499b 0d5c4377 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/01/18 11:14 upstream c1649ec55708 42660d9e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2023/01/18 00:02 upstream 6e50979a9c87 aedf5331 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2022/10/14 09:56 upstream 55be6084c8e0 4954e4b2 .config console log report info [disk image] [vmlinux] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2021/10/01 07:13 upstream 4de593fb965f 1d849ab4 .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_clock_adjtime
2020/10/02 14:35 upstream 472e5b056f00 9602ddf4 .config console log report info ci-upstream-kasan-gce-smack-root
2020/09/22 06:18 upstream 98477740630f 9e1fa68e .config console log report info ci-upstream-kasan-gce-smack-root
2020/08/18 03:07 upstream 06a4ec1d9dc6 424dd8e7 .config console log report ci-upstream-kasan-gce-smack-root
2020/08/07 08:34 upstream d6efb3ac3e6c cb436c69 .config console log report ci-upstream-kasan-gce-smack-root
2020/08/04 15:13 upstream c0842fbc1b18 80a06902 .config console log report ci-upstream-kasan-gce-smack-root
2020/07/27 13:08 upstream 92ed30191993 cb93dc6a .config console log report ci-upstream-kasan-gce-smack-root
2023/06/11 08:42 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci d8b213732169 7086cdb9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in sys_clock_adjtime
* Struck through repros no longer work on HEAD.