INFO: rcu detected stall in nsim_dev_trap_report_work

Status: fixed on 2023/10/12 12:48
Subsystems: net
[Documentation on labels]
Fix commit: 8c21ab1bae94 net/sched: fq_pie: avoid stalls in fq_pie_timer()
First crash: 136d, last: 93d
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in nsim_dev_trap_report_work (2) net usb syz done 1 41d 38d 0/25 upstream: reported syz repro on 2023/11/01 08:50

Sample crash report:
rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P5115 0-.... } 2677 jiffies s: 1565 root: 0x1/T
rcu: blocking rcu_node structures (internal RCU debug):
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5155 Comm: kworker/0:8 Not tainted 6.5.0-syzkaller-11938-g65d6e954e378 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: events nsim_dev_trap_report_work
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:27 [inline]
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x4/0x70 kernel/kcov.c:207
Code: 66 0f 1f 44 00 00 f3 0f 1e fa 48 8b be b0 01 00 00 e8 b0 ff ff ff 31 c0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa <65> 8b 05 bd 18 7c 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b
RSP: 0018:ffffc90000007bb8 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 0000000000abcc77 RCX: ffffffff8861e34d
RDX: ffff88801e980000 RSI: 0000000000000100 RDI: 0000000000000005
RBP: 00000010c6f7a0b5 R08: 0000000000000005 R09: 00000000000f4240
R10: 00000000000f4240 R11: 00000000b379c802 R12: 00000000000f4240
R13: 0000000000000001 R14: 0000000000112e0b R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555555d30788 CR3: 000000000c976000 CR4: 0000000000350ef0
Call Trace:
 pie_calculate_probability+0x266/0x850 net/sched/sch_pie.c:347
 fq_pie_timer+0x1da/0x4f0 net/sched/sch_fq_pie.c:387
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers kernel/time/timer.c:1751 [inline]
 __run_timers+0x764/0xb10 kernel/time/timer.c:2022
 run_timer_softirq+0x58/0xd0 kernel/time/timer.c:2035
 __do_softirq+0x218/0x965 kernel/softirq.c:553
 invoke_softirq kernel/softirq.c:427 [inline]
 __irq_exit_rcu kernel/softirq.c:632 [inline]
 irq_exit_rcu+0xb7/0x120 kernel/softirq.c:644
 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1074
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:orc_ip arch/x86/kernel/unwind_orc.c:80 [inline]
RIP: 0010:__orc_find+0xa1/0x130 arch/x86/kernel/unwind_orc.c:102
Code: 94 4d 00 48 89 e8 48 29 d8 48 89 c2 48 c1 e8 3f 48 c1 fa 02 48 01 d0 48 d1 f8 4c 8d 3c 83 4c 89 fa 48 c1 ea 03 42 0f b6 0c 32 <4c> 89 fa 83 e2 07 83 c2 03 38 ca 7c 04 84 c9 75 6f 49 63 17 4c 89
RSP: 0018:ffffc90004ca76c0 EFLAGS: 00000213
RAX: 000000000000000a RBX: ffffffff8edc31dc RCX: 0000000000000000
RDX: 1ffffffff1db8640 RSI: ffffffff813a0f2a RDI: 0000000000000005
RBP: ffffffff8edc322c R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000015 R11: 89618b4f0d26dd47 R12: ffffffff81304654
R13: ffffffff8f53f712 R14: dffffc0000000000 R15: ffffffff8edc3204
 orc_find arch/x86/kernel/unwind_orc.c:227 [inline]
 unwind_next_frame+0x329/0x2390 arch/x86/kernel/unwind_orc.c:494
 arch_stack_walk+0xfa/0x170 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x96/0xd0 kernel/stacktrace.c:122
 kasan_save_stack+0x33/0x50 mm/kasan/common.c:45
 kasan_set_track+0x25/0x30 mm/kasan/common.c:52
 kasan_save_free_info+0x2b/0x40 mm/kasan/generic.c:522
 ____kasan_slab_free mm/kasan/common.c:236 [inline]
 ____kasan_slab_free+0x15b/0x1b0 mm/kasan/common.c:200
 kasan_slab_free include/linux/kasan.h:162 [inline]
 slab_free_hook mm/slub.c:1800 [inline]
 slab_free_freelist_hook+0x114/0x1e0 mm/slub.c:1826
 slab_free mm/slub.c:3809 [inline]
 kmem_cache_free+0xf0/0x480 mm/slub.c:3831
 kfree_skbmem+0xef/0x1b0 net/core/skbuff.c:1004
 __kfree_skb net/core/skbuff.c:1062 [inline]
 consume_skb net/core/skbuff.c:1277 [inline]
 consume_skb+0xdf/0x170 net/core/skbuff.c:1271
 nsim_dev_trap_report drivers/net/netdevsim/dev.c:821 [inline]
 nsim_dev_trap_report_work+0x872/0xc70 drivers/net/netdevsim/dev.c:850
 process_one_work+0x887/0x15d0 kernel/workqueue.c:2630
 process_scheduled_works kernel/workqueue.c:2703 [inline]
 worker_thread+0x8bb/0x1290 kernel/workqueue.c:2784
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304

Crashes (4):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/09/07 00:57 upstream 65d6e954e378 72324844 .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in nsim_dev_trap_report_work
2023/09/01 18:24 upstream 99d99825fc07 696ea0d2 .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in nsim_dev_trap_report_work
2023/07/29 23:16 upstream ffabf7c73176 92476829 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: rcu detected stall in nsim_dev_trap_report_work
2023/07/26 01:54 net-next ec87f05402f5 6756545c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in nsim_dev_trap_report_work
* Struck through repros no longer work on HEAD.