syzbot

 sign-in | mailing list | source | docs
🐞 Open [552] 🐞 Fixed [32] 🐞 Invalid [297] ⬇ Missing Backports [45] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in gc_worker

Status: upstream: reported C repro on 2023/08/06 21:00
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+c6ab59ecefe93f5cab03@syzkaller.appspotmail.com
First crash: 265d, last: 76d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Bug presence (1)
Date Name Commit Repro Result
2023/08/07 upstream (ToT) 52a93d39b17d C [report] INFO: rcu detected stall in corrupted
▶ ▼ Similar bugs (5)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in gc_worker (2) netfilter C unreliable 4 833d 871d 0/26 closed as invalid on 2022/02/08 10:33
upstream INFO: rcu detected stall in gc_worker (3) netfilter C done done 47 90d 769d 0/26 upstream: reported C repro on 2022/03/20 12:02
upstream INFO: rcu detected stall in gc_worker netfilter 8 1837d 1922d 0/26 auto-closed as invalid on 2019/10/14 16:34
linux-6.1 INFO: rcu detected stall in gc_worker origin:upstream C 7 4d00h 300d 0/3 upstream: reported C repro on 2023/07/03 02:12
linux-4.19 INFO: rcu detected stall in gc_worker syz error 1 736d 736d 0/1 upstream: reported syz repro on 2022/04/22 15:43
▶ ▼ Fix bisection attempts (4)
Created Duration User Patch Repo Result
2024/03/16 14:35 18m bisect fix linux-5.15.y error job log (0)
2024/01/19 00:07 2h02m bisect fix linux-5.15.y job log (0) log
2023/11/27 09:09 1h53m bisect fix linux-5.15.y job log (0) log
2023/10/04 20:45 1h54m bisect fix linux-5.15.y job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 GPs behind) idle=0c7/1/0x4000000000000000 softirq=4794/4795 fqs=2100 
	(detected by 1, t=10505 jiffies, g=3825, q=500)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 3534 Comm: kworker/0:1 Not tainted 5.15.148-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Workqueue: events_power_efficient gc_worker
RIP: 0010:native_irq_disable arch/x86/include/asm/irqflags.h:40 [inline]
RIP: 0010:arch_local_irq_disable arch/x86/include/asm/irqflags.h:75 [inline]
RIP: 0010:arch_local_irq_save arch/x86/include/asm/irqflags.h:107 [inline]
RIP: 0010:lock_acquire+0x193/0x4f0 kernel/locking/lockdep.c:5619
Code: 00 00 00 00 00 00 9c 8f 84 24 80 00 00 00 42 80 3c 2b 00 74 08 4c 89 ff e8 fa 61 67 00 48 8d 5c 24 60 4c 8b bc 24 80 00 00 00 <fa> 48 c7 c7 80 1d 8b 8a e8 50 1b ba 08 65 ff 05 c9 f6 9f 7e 45 31
RSP: 0018:ffffc90000007a00 EFLAGS: 00000046
RAX: 0000000000000000 RBX: ffffc90000007a60 RCX: ffffffff8162815c
RDX: 0000000000000000 RSI: ffffffff8ad87e20 RDI: ffffffff8ad87de0
RBP: ffffc90000007b48 R08: dffffc0000000000 R09: fffffbfff1bc73d6
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000000f48
R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000046
FS:  0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 000000000c68e000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
 debug_object_activate+0x166/0x860 lib/debugobjects.c:693
 debug_hrtimer_activate kernel/time/hrtimer.c:420 [inline]
 debug_activate kernel/time/hrtimer.c:475 [inline]
 enqueue_hrtimer+0x30/0x310 kernel/time/hrtimer.c:1084
 __run_hrtimer kernel/time/hrtimer.c:1702 [inline]
 __hrtimer_run_queues+0x6b6/0xcf0 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x8c/0xb0 arch/x86/kernel/apic/apic.c:1096
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:lock_acquire+0x252/0x4f0 kernel/locking/lockdep.c:5627
Code: 2b 00 74 08 4c 89 f7 e8 4c 61 67 00 f6 44 24 61 02 0f 85 84 01 00 00 41 f7 c7 00 02 00 00 74 01 fb 48 c7 44 24 40 0e 36 e0 45 <4b> c7 44 25 00 00 00 00 00 43 c7 44 25 09 00 00 00 00 43 c7 44 25
RSP: 0018:ffffc900025a7980 EFLAGS: 00000206
RAX: 0000000000000001 RBX: 1ffff920004b4f3c RCX: 1ffff920004b4edc
RDX: dffffc0000000000 RSI: ffffffff8a8b2220 RDI: ffffffff8ad87e40
RBP: ffffc900025a7ae0 R08: dffffc0000000000 R09: fffffbfff1f7a019
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff920004b4f38
R13: dffffc0000000000 R14: ffffc900025a79e0 R15: 0000000000000246
 rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:269
 rcu_read_lock include/linux/rcupdate.h:696 [inline]
 gc_worker+0x289/0x14b0 net/netfilter/nf_conntrack_core.c:1470
 process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
 worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.759 msecs

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/02/11 11:46 linux-5.15.y 6139f2a02fe0 77b23aa1 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in gc_worker
2023/08/06 21:00 linux-5.15.y 38d4ca22a528 4ffcc9ef .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in gc_worker
2023/12/18 20:16 linux-5.15.y d0fc081c6241 924661f4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in gc_worker
2023/12/12 15:43 linux-5.15.y 8a1d809b0545 ebcad15c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in gc_worker
2023/10/26 16:58 linux-5.15.y 12952a23a5da 23afc60f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in gc_worker
2024/01/21 21:46 linux-5.15.y ddcaf4999061 9bd8dcda .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 INFO: rcu detected stall in gc_worker
* Struck through repros no longer work on HEAD.