syzbot


INFO: rcu detected stall in batadv_nc_worker

Status: upstream: reported C repro on 2023/07/06 06:16
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+13306b743f5eee80aab3@syzkaller.appspotmail.com
First crash: 469d, last: 21h22m
Fix commit to backport (bisect log) :
tree: upstream
commit 8c21ab1bae945686c602c5bfa4e3f3352c2452c5
Author: Eric Dumazet <edumazet@google.com>
Date: Tue Aug 29 12:35:41 2023 +0000

  net/sched: fq_pie: avoid stalls in fq_pie_timer()

[report pending]
  
Fix bisection: the issue occurs on the latest tested release (bisect log)
Crash: INFO: rcu detected stall in corrupted (log)
Repro: C syz .config
  
Bug presence (2)
Date Name Commit Repro Result
2023/09/18 linux-6.1.y (ToT) 09045dae0d90 C [report] INFO: rcu detected stall in corrupted
2023/09/18 upstream (ToT) 2cf0f7156238 C Didn't crash
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in batadv_nc_worker cgroups mm 7 1779d 1781d 0/28 closed as invalid on 2019/12/04 14:04
upstream INFO: rcu detected stall in batadv_nc_worker (2) kernel 4 1744d 1744d 0/28 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in batadv_nc_worker (3) batman C done inconclusive 511 11h56m 1477d 0/28 upstream: reported C repro on 2020/10/01 10:35
linux-5.15 INFO: rcu detected stall in batadv_nc_worker 127 4d11h 492d 0/3 upstream: reported on 2023/06/13 20:27
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2023/11/06 15:29 7h44m fix candidate upstream OK (1) job log
2023/10/25 07:11 2h37m bisect fix linux-6.1.y OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
	(detected by 0, t=10502 jiffies, g=8329, q=1477 ncpus=2)
rcu: All QSes seen, last rcu_preempt kthread activity 10482 (4294985187-4294974705), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_preempt kthread starved for 10483 jiffies! g8329 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25496 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x132c/0x4330 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1935
 rcu_gp_fqs_loop+0x2c2/0x1010 kernel/rcu/tree.c:1661
 rcu_gp_kthread+0xa3/0x3a0 kernel/rcu/tree.c:1860
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9 Comm: kworker/u4:0 Not tainted 6.1.53-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/04/2023
Workqueue: bat_events batadv_nc_worker
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:write_comp_data kernel/kcov.c:236 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp8+0x28/0x80 kernel/kcov.c:311
Code: 1f 00 4c 8b 04 24 65 48 8b 0d 24 57 78 7e 65 8b 05 25 57 78 7e a9 00 01 ff 00 74 10 a9 00 01 00 00 74 57 83 b9 1c 16 00 00 00 <74> 4e 8b 81 f8 15 00 00 83 f8 03 75 43 48 8b 91 00 16 00 00 44 8b
RSP: 0018:ffffc900001e0a68 EFLAGS: 00000246
RAX: 0000000000000101 RBX: 00004189374bc6a7 RCX: ffff88813fec0000
RDX: ffff88813fec0000 RSI: 0000000000000000 RDI: 00004189374bc6a7
RBP: 0019999999999998 R08: ffffffff88b0ae11 R09: 0000000000000003
R10: ffffffffffffffff R11: dffffc0000000001 R12: 0000000225c17d04
R13: 0000000000000000 R14: 00000015798ee228 R15: 1ffff1100d049d63
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000080 CR3: 000000000ce8e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 pie_calculate_probability+0x241/0x820 net/sched/sch_pie.c:345
 fq_pie_timer+0x1ff/0x410 net/sched/sch_fq_pie.c:387
 call_timer_fn+0x19e/0x6b0 kernel/time/timer.c:1474
 expire_timers kernel/time/timer.c:1519 [inline]
 __run_timers+0x67c/0x890 kernel/time/timer.c:1790
 run_timer_softirq+0x63/0xf0 kernel/time/timer.c:1803
 __do_softirq+0x2e9/0xa4c kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:650
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:649
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:35 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_local_irq_save arch/x86/include/asm/irqflags.h:106 [inline]
RIP: 0010:lock_acquire+0x199/0x5a0 kernel/locking/lockdep.c:5657
Code: eb 03 42 80 3c 2b 00 74 08 4c 89 ff e8 d0 bf 76 00 48 c7 84 24 80 00 00 00 00 00 00 00 9c 8f 84 24 80 00 00 00 42 80 3c 2b 00 <74> 08 4c 89 ff e8 2d bf 76 00 48 8d 5c 24 60 4c 8b bc 24 80 00 00
RSP: 0018:ffffc900000e7a80 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 1ffff9200001cf60 RCX: ffffffff8169f64b
RDX: 0000000000000000 RSI: ffffffff8b3ceb60 RDI: ffffffff8b3ceb20
RBP: ffffc900000e7bd8 R08: dffffc0000000000 R09: fffffbfff1ce766e
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff9200001cf58
R13: dffffc0000000000 R14: 0000000000000000 R15: ffffc900000e7b00
 rcu_lock_acquire+0x2a/0x30 include/linux/rcupdate.h:306
 rcu_read_lock include/linux/rcupdate.h:747 [inline]
 batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:408 [inline]
 batadv_nc_worker+0xc1/0x5b0 net/batman-adv/network-coding.c:719
 process_one_work+0x8aa/0x11f0 kernel/workqueue.c:2292
 worker_thread+0xa5f/0x1210 kernel/workqueue.c:2439
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>

Crashes (98):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/09/18 20:52 linux-6.1.y 09045dae0d90 0b6a67ac .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/10/17 05:45 linux-6.1.y aa4cd140bba5 666f77ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/10/03 00:05 linux-6.1.y aa4cd140bba5 02f9582a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/09/28 07:34 linux-6.1.y e526b12bf916 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/09/19 09:10 linux-6.1.y e526b12bf916 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/09/17 16:55 linux-6.1.y 5f55cad62cc9 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/09/15 11:07 linux-6.1.y 5f55cad62cc9 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/09/09 08:12 linux-6.1.y 5ca5b389fddf 9750182a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/31 12:58 linux-6.1.y 311d8503ef9f 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/17 17:41 linux-6.1.y 117ac406ba90 dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/17 13:08 linux-6.1.y 117ac406ba90 dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/16 16:37 linux-6.1.y 117ac406ba90 e1c76ab2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/14 00:33 linux-6.1.y 36790ef5e00b f21a18ca .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/12 20:27 linux-6.1.y 36790ef5e00b 842184b3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/12 19:01 linux-6.1.y 36790ef5e00b 842184b3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/10 00:40 linux-6.1.y 48d525b0e463 a83d9288 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/08 00:55 linux-6.1.y 48d525b0e463 109d2082 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/02 18:21 linux-6.1.y c1cec4dad96b 1e9c4cf3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/29 22:37 linux-6.1.y c1cec4dad96b 5187fc86 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/27 23:13 linux-6.1.y c1cec4dad96b 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/27 20:26 linux-6.1.y c1cec4dad96b 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/26 13:30 linux-6.1.y c18e82d3ee44 3f86dfed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/26 12:30 linux-6.1.y c18e82d3ee44 3f86dfed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/21 11:12 linux-6.1.y 9b3f9a5b12dc b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: rcu detected stall in batadv_nc_worker
2024/07/20 20:49 linux-6.1.y 9b3f9a5b12dc b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/18 23:57 linux-6.1.y 9b3f9a5b12dc 71884c12 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/18 09:12 linux-6.1.y cac15753b8ce 0f902625 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/17 11:29 linux-6.1.y cac15753b8ce 215bec2d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/16 10:48 linux-6.1.y cac15753b8ce e8709b21 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/13 11:17 linux-6.1.y 266ee8e06d5b eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/12 12:36 linux-6.1.y 266ee8e06d5b eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/10 11:02 linux-6.1.y 7753af06eebf bc144f9a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/09 03:21 linux-6.1.y 7753af06eebf bc23a442 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/09 00:51 linux-6.1.y 7753af06eebf bc23a442 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/08 15:03 linux-6.1.y 7753af06eebf cde64f7d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/07 14:45 linux-6.1.y 7753af06eebf 2a40360c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/05 00:56 linux-6.1.y 99e6a620de00 3f2748a3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/04 23:37 linux-6.1.y 99e6a620de00 3f2748a3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/07/02 10:00 linux-6.1.y 99e6a620de00 07f0a0a0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/29 05:39 linux-6.1.y 99e6a620de00 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/23 20:42 linux-6.1.y eb44d83053d6 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/22 02:18 linux-6.1.y eb44d83053d6 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/22 01:09 linux-6.1.y eb44d83053d6 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/21 02:38 linux-6.1.y eb44d83053d6 dac2aa43 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/19 08:55 linux-6.1.y eb44d83053d6 41b7e219 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/18 19:33 linux-6.1.y eb44d83053d6 639d6cdf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/06/18 08:24 linux-6.1.y eb44d83053d6 ce6011bc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2024/08/05 15:30 linux-6.1.y 48d525b0e463 e35c337f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: rcu detected stall in batadv_nc_worker
2024/07/20 22:03 linux-6.1.y 9b3f9a5b12dc b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: rcu detected stall in batadv_nc_worker
2024/07/04 20:15 linux-6.1.y 99e6a620de00 3f2748a3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: rcu detected stall in batadv_nc_worker
2023/07/30 02:51 linux-6.1.y d2a6dc4eaf6d 92476829 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/07/20 08:39 linux-6.1.y a456e1743881 4547cdf9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
2023/07/06 06:15 linux-6.1.y 0f4ac6b4c5f0 ba5dba36 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: rcu detected stall in batadv_nc_worker
* Struck through repros no longer work on HEAD.