syzbot

 sign-in | mailing list | source | docs
🐞 Open [567] 🐞 Fixed [66] 🐞 Invalid [207] ⬇ Missing Backports [39] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in batadv_nc_worker

Status: upstream: reported C repro on 2023/07/06 06:16
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+13306b743f5eee80aab3@syzkaller.appspotmail.com
First crash: 300d, last: 23d
Fix commit to backport (bisect log) :
tree: upstream
commit 8c21ab1bae945686c602c5bfa4e3f3352c2452c5
Author: Eric Dumazet <edumazet@google.com>
Date: Tue Aug 29 12:35:41 2023 +0000

  net/sched: fq_pie: avoid stalls in fq_pie_timer()

[report pending]
  
Fix bisection: the issue occurs on the latest tested release (bisect log)
Crash: INFO: rcu detected stall in corrupted (log)
Repro: C syz .config
  
▶ ▼ Bug presence (2)
Date Name Commit Repro Result
2023/09/18 linux-6.1.y (ToT) 09045dae0d90 C [report] INFO: rcu detected stall in corrupted
2023/09/18 upstream (ToT) 2cf0f7156238 C Didn't crash
▶ ▼ Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in batadv_nc_worker cgroups mm 7 1610d 1612d 0/26 closed as invalid on 2019/12/04 14:04
upstream INFO: rcu detected stall in batadv_nc_worker (2) kernel 4 1575d 1575d 0/26 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in batadv_nc_worker (3) batman C done inconclusive 208 3d11h 1308d 0/26 upstream: reported C repro on 2020/10/01 10:35
linux-5.15 INFO: rcu detected stall in batadv_nc_worker 18 12d 323d 0/3 upstream: reported on 2023/06/13 20:27
▶ ▼ Fix bisection attempts (2)
Created Duration User Patch Repo Result
2023/11/06 15:29 7h44m fix candidate upstream job log (1)
2023/10/25 07:11 2h37m bisect fix linux-6.1.y job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
	(detected by 0, t=10502 jiffies, g=8329, q=1477 ncpus=2)
rcu: All QSes seen, last rcu_preempt kthread activity 10482 (4294985187-4294974705), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_preempt kthread starved for 10483 jiffies! g8329 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25496 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x132c/0x4330 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1935
 rcu_gp_fqs_loop+0x2c2/0x1010 kernel/rcu/tree.c:1661
 rcu_gp_kthread+0xa3/0x3a0 kernel/rcu/tree.c:1860
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9 Comm: kworker/u4:0 Not tainted 6.1.53-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/04/2023
Workqueue: bat_events batadv_nc_worker
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:write_comp_data kernel/kcov.c:236 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp8+0x28/0x80 kernel/kcov.c:311
Code: 1f 00 4c 8b 04 24 65 48 8b 0d 24 57 78 7e 65 8b 05 25 57 78 7e a9 00 01 ff 00 74 10 a9 00 01 00 00 74 57 83 b9 1c 16 00 00 00 <74> 4e 8b 81 f8 15 00 00 83 f8 03 75 43 48 8b 91 00 16 00 00 44 8b
RSP: 0018:ffffc900001e0a68 EFLAGS: 00000246
RAX: 0000000000000101 RBX: 00004189374bc6a7 RCX: ffff88813fec0000
RDX: ffff88813fec0000 RSI: 0000000000000000 RDI: 00004189374bc6a7
RBP: 0019999999999998 R08: ffffffff88b0ae11 R09: 0000000000000003
R10: ffffffffffffffff R11: dffffc0000000001 R12: 0000000225c17d04
R13: 0000000000000000 R14: 00000015798ee228 R15: 1ffff1100d049d63
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000080 CR3: 000000000ce8e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 pie_calculate_probability+0x241/0x820 net/sched/sch_pie.c:345
 fq_pie_timer+0x1ff/0x410 net/sched/sch_fq_pie.c:387
 call_timer_fn+0x19e/0x6b0 kernel/time/timer.c:1474
 expire_timers kernel/time/timer.c:1519 [inline]
 __run_timers+0x67c/0x890 kernel/time/timer.c:1790
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1803
 __do_softirq+0x2e9/0xa4c kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:650
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:649
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:35 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_local_irq_save arch/x86/include/asm/irqflags.h:106 [inline]
RIP: 0010:lock_acquire+0x199/0x5a0 kernel/locking/lockdep.c:5657
Code: eb 03 42 80 3c 2b 00 74 08 4c 89 ff e8 d0 bf 76 00 48 c7 84 24 80 00 00 00 00 00 00 00 9c 8f 84 24 80 00 00 00 42 80 3c 2b 00 <74> 08 4c 89 ff e8 2d bf 76 00 48 8d 5c 24 60 4c 8b bc 24 80 00 00
RSP: 0018:ffffc900000e7a80 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 1ffff9200001cf60 RCX: ffffffff8169f64b
RDX: 0000000000000000 RSI: ffffffff8b3ceb60 RDI: ffffffff8b3ceb20
RBP: ffffc900000e7bd8 R08: dffffc0000000000 R09: fffffbfff1ce766e
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff9200001cf58
R13: dffffc0000000000 R14: 0000000000000000 R15: ffffc900000e7b00
 rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:306
 rcu_read_lock include/linux/rcupdate.h:747 [inline]
 batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:408 [inline]
 batadv_nc_worker+0xc1/0x5b0 net/batman-adv/network-coding.c:719
 process_one_work+0x8aa/0x11f0 kernel/workqueue.c:2292
 worker_thread+0xa5f/0x1210 kernel/workqueue.c:2439
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>

Crashes (14):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/09/18 20:52 linux-6.1.y 09045dae0d90 0b6a67ac .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/04/08 15:39 linux-6.1.y 347385861c50 53df08b6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/03/29 17:45 linux-6.1.y e5cd595e23c1 c52bcb23 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/01/21 21:43 linux-6.1.y 8fd7f4462453 9bd8dcda .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/12/07 17:59 linux-6.1.y c6114c845984 0a02ce36 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/12/03 16:17 linux-6.1.y c6114c845984 f819d6f7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/11/21 20:40 linux-6.1.y 69e434a1cb21 cb976f63 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/11/10 02:41 linux-6.1.y fb2635ac69ab 56230772 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/09/08 22:28 linux-6.1.y 59b13c2b647e 6654cf89 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in batadv_nc_worker
2023/08/24 10:19 linux-6.1.y 802aacbbffe2 4d7ae7ab .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/08/22 12:29 linux-6.1.y 6c44e13dc284 6b415825 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/07/30 02:51 linux-6.1.y d2a6dc4eaf6d 92476829 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/07/20 08:39 linux-6.1.y a456e1743881 4547cdf9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/07/06 06:15 linux-6.1.y 0f4ac6b4c5f0 ba5dba36 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
* Struck through repros no longer work on HEAD.