syzbot


INFO: rcu detected stall in nsim_fib_event_work

Status: upstream: reported syz repro on 2024/09/01 09:18
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+e5beaea4288bdc45b1fe@syzkaller.appspotmail.com
First crash: 81d, last: 63d
Bug presence (3)
Date Name Commit Repro Result
2024/10/16 linux-5.15.y (ToT) 3a5928702e71 C [report] INFO: rcu detected stall in corrupted
2024/09/01 upstream (ToT) 431c1646e1f8 C [report] BUG: workqueue lockup
2024/10/16 upstream (ToT) c964ced77262 C Didn't crash
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in nsim_fib_event_work 1 163d 163d 0/3 auto-obsoleted due to no activity on 2024/09/19 04:15
linux-6.1 INFO: rcu detected stall in nsim_fib_event_work (2) 1 46d 46d 0/3 upstream: reported on 2024/10/06 20:10
Last patch testing requests (2)
Created Duration User Patch Repo Result
2024/10/14 02:22 13m retest repro linux-5.15.y report log
2024/10/14 02:22 14m retest repro linux-5.15.y report log
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/10/26 09:57 5m fix candidate upstream error job log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P4030/1:b..l
	(detected by 0, t=10502 jiffies, g=5937, q=203)
task:kworker/0:3     state:R  running task     stack:    0 pid: 4030 ppid:     2 flags:0x00000008
Workqueue: events nsim_fib_event_work
Call trace:
 __switch_to+0x308/0x5e8 arch/arm64/kernel/process.c:518
 context_switch kernel/sched/core.c:5027 [inline]
 __schedule+0xf10/0x1e48 kernel/sched/core.c:6373
 preempt_schedule_irq+0x90/0x274 kernel/sched/core.c:6777
 arm64_preempt_schedule_irq+0x174/0x2c4 arch/arm64/kernel/entry-common.c:260
 el1_interrupt+0x40/0x58 arch/arm64/kernel/entry-common.c:463
 el1h_64_irq_handler+0x18/0x24 arch/arm64/kernel/entry-common.c:470
 el1h_64_irq+0x78/0x7c arch/arm64/kernel/entry.S:522
 arch_local_irq_enable arch/arm64/include/asm/irqflags.h:35 [inline]
 __local_bh_enable_ip+0x238/0x470 kernel/softirq.c:388
 local_bh_enable+0x28/0x34 include/linux/bottom_half.h:32
 rht_assign_unlock+0xa0/0xc0 include/linux/rhashtable.h:404
 __rhashtable_insert_fast include/linux/rhashtable.h:788 [inline]
 rhashtable_insert_fast+0x57c/0xb94 include/linux/rhashtable.h:825
 nsim_fib4_rt_add drivers/net/netdevsim/fib.c:360 [inline]
 nsim_fib4_rt_insert drivers/net/netdevsim/fib.c:430 [inline]
 nsim_fib4_event drivers/net/netdevsim/fib.c:462 [inline]
 nsim_fib_event drivers/net/netdevsim/fib.c:882 [inline]
 nsim_fib_event_work+0x1464/0x33c4 drivers/net/netdevsim/fib.c:1481
 process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
 process_scheduled_works kernel/workqueue.c:2373 [inline]
 worker_thread+0xb88/0x1034 kernel/workqueue.c:2459
 kthread+0x37c/0x45c kernel/kthread.c:334
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870
rcu: rcu_preempt kthread starved for 10496 jiffies! g5937 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:    0 pid:   15 ppid:     2 flags:0x00000008
Call trace:
 __switch_to+0x308/0x5e8 arch/arm64/kernel/process.c:518
 context_switch kernel/sched/core.c:5027 [inline]
 __schedule+0xf10/0x1e48 kernel/sched/core.c:6373
 schedule+0x11c/0x1c8 kernel/sched/core.c:6456
 schedule_timeout+0x1d8/0x344 kernel/time/timer.c:1914
 rcu_gp_fqs_loop+0x26c/0x13d4 kernel/rcu/tree.c:1972
 rcu_gp_kthread+0xc4/0x2c4 kernel/rcu/tree.c:2145
 kthread+0x37c/0x45c kernel/kthread.c:334
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870
rcu: Stack dump where RCU GP kthread last ran:
Task dump for CPU 0:
task:syz.0.28        state:R  running task     stack:    0 pid: 4209 ppid:  4120 flags:0x00000001
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 sched_show_task+0x43c/0x574 kernel/sched/core.c:8637
 dump_cpu_task+0x84/0xa8 kernel/sched/core.c:10924
 rcu_check_gp_kthread_starvation+0x1f0/0x254 kernel/rcu/tree_stall.h:488
 print_other_cpu_stall+0x202c/0x2290 kernel/rcu/tree_stall.h:592
 check_cpu_stall kernel/rcu/tree_stall.h:745 [inline]
 rcu_pending kernel/rcu/tree.c:3932 [inline]
 rcu_sched_clock_irq+0x1894/0x1b8c kernel/rcu/tree.c:2619
 update_process_times+0x1d4/0x270 kernel/time/timer.c:1818
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x340/0x4f8 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x458/0xca4 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x2c0/0xb64 kernel/time/hrtimer.c:1814
 timer_handler drivers/clocksource/arm_arch_timer.c:659 [inline]
 arch_timer_handler_virt+0x74/0x88 drivers/clocksource/arm_arch_timer.c:670
 handle_percpu_devid_irq+0x29c/0x7fc kernel/irq/chip.c:930
 generic_handle_irq_desc include/linux/irqdesc.h:158 [inline]
 handle_irq_desc kernel/irq/irqdesc.c:652 [inline]
 handle_domain_irq+0xec/0x178 kernel/irq/irqdesc.c:707
 gic_handle_irq+0x78/0x1c8 drivers/irqchip/irq-gic-v3.c:758
 call_on_irq_stack+0x24/0x4c arch/arm64/kernel/entry.S:899
 do_interrupt_handler+0x74/0x94 arch/arm64/kernel/entry-common.c:267
 el1_interrupt+0x30/0x58 arch/arm64/kernel/entry-common.c:454
 el1h_64_irq_handler+0x18/0x24 arch/arm64/kernel/entry-common.c:470
 el1h_64_irq+0x78/0x7c arch/arm64/kernel/entry.S:522
 arch_local_irq_restore+0x8/0x10 arch/arm64/include/asm/irqflags.h:122
 lock_is_held include/linux/lockdep.h:287 [inline]
 ___might_sleep+0x98/0x4dc kernel/sched/core.c:9602
 __might_sleep+0x98/0xf0 kernel/sched/core.c:9590
 __might_fault+0x80/0x128 mm/memory.c:5324
 restore_sigframe arch/arm64/kernel/signal.c:503 [inline]
 __arm64_sys_rt_sigreturn+0x3e8/0x22ec arch/arm64/kernel/signal.c:555
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/19 00:13 linux-5.15.y 3a5928702e71 c673ca06 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 INFO: rcu detected stall in nsim_fib_event_work
2024/09/01 09:17 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 INFO: rcu detected stall in nsim_fib_event_work
* Struck through repros no longer work on HEAD.