INFO: rcu detected stall in net_rx_action (5)

Status: auto-obsoleted due to no activity on 2024/07/25 01:10
Subsystems: virt wireguard wireless
[Documentation on labels]
First crash: 876d, last: 839d
▶ ▼ Similar bugs (12)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in net_rx_action (3) kvm 1 67 1570d 1570d 0/29 auto-closed as invalid on 2022/06/05 10:21
upstream INFO: rcu detected stall in net_rx_action net virt 1 3 2030d 2038d 0/29 auto-closed as invalid on 2021/04/21 12:41
linux-5.15 INFO: rcu detected stall in net_rx_action 1 1 1058d 1058d 0/3 auto-obsoleted due to no activity on 2023/12/28 19:24
upstream INFO: rcu detected stall in net_rx_action (2) batman 1 1 1723d 1723d 0/29 closed as invalid on 2022/02/08 10:10
upstream INFO: rcu detected stall in net_rx_action (4) kernel 1 1 1286d 1286d 0/29 auto-obsoleted due to no activity on 2023/05/05 00:12
android-5-15 BUG: soft lockup in net_rx_action 1 1 848d 848d 0/2 auto-obsoleted due to no activity on 2024/07/16 07:14
android-5-10 BUG: soft lockup in net_rx_action 1 2 722d 734d 0/2 auto-obsoleted due to no activity on 2024/11/18 17:50
android-6-1 BUG: soft lockup in net_rx_action 1 2 740d 784d 0/2 auto-obsoleted due to no activity on 2024/10/31 15:08
android-54 BUG: soft lockup in net_rx_action 1 1 788d 788d 0/2 auto-obsoleted due to no activity on 2024/09/14 14:52
linux-4.19 BUG: soft lockup in net_rx_action (2) 1 1 1406d 1406d 0/1 auto-obsoleted due to no activity on 2023/02/03 20:26
android-5-15 BUG: soft lockup in net_rx_action (2) 1 1 729d 729d 0/2 auto-obsoleted due to no activity on 2024/11/11 18:48
linux-4.19 BUG: soft lockup in net_rx_action 1 1 1670d 1670d 0/1 auto-closed as invalid on 2022/05/16 12:13

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=75ac/1/0x4000000000000000 softirq=43880/43881 fqs=22
rcu: 	(detected by 0, t=10502 jiffies, g=63365, q=569 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 11275 Comm: syz-executor.4 Not tainted 6.9.0-rc4-syzkaller-00173-g3cdb45594619 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
RIP: 0010:lockdep_recursion_finish kernel/locking/lockdep.c:467 [inline]
RIP: 0010:lock_release+0x5c9/0x9f0 kernel/locking/lockdep.c:5776
Code: 00 fc ff df 4c 8b 64 24 08 48 8b 5c 24 28 49 89 dd 4c 8d b4 24 90 00 00 00 48 c7 c7 20 ca ca 8b e8 5c 67 04 0a b8 ff ff ff ff <65> 0f c1 05 3f e4 8f 7e 83 f8 01 0f 85 9a 00 00 00 4c 89 f3 48 c1
RSP: 0018:ffffc90000a07d00 EFLAGS: 00000086
RAX: 00000000ffffffff RBX: 0000000000000046 RCX: ffffc90000a07d03
RDX: 0000000000000002 RSI: ffffffff8bcaca20 RDI: ffffffff8c1eb160
RBP: ffffc90000a07e30 R08: ffffffff8fa7b4af R09: 1ffffffff1f4f695
R10: dffffc0000000000 R11: fffffbfff1f4f696 R12: 1ffff92000140fac
R13: 0000000000000046 R14: ffffc90000a07d90 R15: dffffc0000000000
FS:  00005555843b4480(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2ce22000 CR3: 000000005b3f0000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_unlock include/linux/spinlock_api_smp.h:141 [inline]
 _raw_spin_unlock+0x16/0x50 kernel/locking/spinlock.c:186
 spin_unlock include/linux/spinlock.h:391 [inline]
 advance_sched+0x9b4/0xca0 net/sched/sch_taprio.c:981
 __run_hrtimer kernel/time/hrtimer.c:1692 [inline]
 __hrtimer_run_queues+0x5a5/0xd50 kernel/time/hrtimer.c:1756
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1818
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline]
 __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1049
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0x52/0xc0 arch/x86/kernel/apic/apic.c:1043
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:orc_ip arch/x86/kernel/unwind_orc.c:80 [inline]
RIP: 0010:__orc_find arch/x86/kernel/unwind_orc.c:102 [inline]
RIP: 0010:orc_find arch/x86/kernel/unwind_orc.c:227 [inline]
RIP: 0010:unwind_next_frame+0x502/0x2a00 arch/x86/kernel/unwind_orc.c:494
Code: e8 3f 48 01 c8 48 83 e0 fe 49 8d 5c 45 00 48 89 d8 48 c1 e8 03 48 b9 00 00 00 00 00 fc ff df 0f b6 04 08 84 c0 75 34 48 63 2b <48> 01 dd 48 89 ef 4c 89 f6 e8 50 be 54 00 48 8d 43 04 4c 39 f5 4c
RSP: 0018:ffffc90000a08288 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff901fd7c0 RCX: dffffc0000000000
RDX: 0000000080000101 RSI: ffffffff8b76dae5 RDI: ffffffff8b76dad4
RBP: fffffffffb57033d R08: ffffffff81414480 R09: ffffc90000a08450
R10: 0000000000000003 R11: ffffffff8181dfb0 R12: ffffffff901fd7c4
R13: ffffffff901fd7bc R14: ffffffff8b76dae5 R15: ffffffff901fd7b8
 arch_stack_walk+0x151/0x1b0 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:312 [inline]
 __kasan_mempool_unpoison_object+0x64/0x100 mm/kasan/common.c:535
 kasan_mempool_unpoison_object include/linux/kasan.h:339 [inline]
 napi_skb_cache_get+0x11e/0x230 net/core/skbuff.c:357
 __napi_build_skb net/core/skbuff.c:527 [inline]
 __napi_alloc_skb+0x217/0x540 net/core/skbuff.c:846
 napi_alloc_skb include/linux/skbuff.h:3358 [inline]
 page_to_skb+0x275/0x9b0 drivers/net/virtio_net.c:619
 receive_mergeable drivers/net/virtio_net.c:1699 [inline]
 receive_buf+0x3b3/0x3890 drivers/net/virtio_net.c:1820
 virtnet_receive drivers/net/virtio_net.c:2126 [inline]
 virtnet_poll+0x850/0x1ad0 drivers/net/virtio_net.c:2219
 __napi_poll+0xcb/0x490 net/core/dev.c:6638
 napi_poll net/core/dev.c:6707 [inline]
 net_rx_action+0x7bb/0x1090 net/core/dev.c:6822
 __do_softirq+0x2c6/0x980 kernel/softirq.c:554
 invoke_softirq kernel/softirq.c:428 [inline]
 __irq_exit_rcu+0xf2/0x1c0 kernel/softirq.c:633
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:645
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:is_bpf_text_address+0xcf/0x2b0
Code: ff c6 05 9d a7 ed 0d 01 48 c7 c7 a0 7b d0 8b be 0f 03 00 00 48 c7 c2 e0 7b d0 8b e8 6b 53 ce ff 49 bd 00 00 00 00 00 fc ff df <8b> 2d 9b 6b 5c 0c 48 89 2c 24 83 e5 01 48 8d 1c ed b0 f3 00 8e 48
RSP: 0018:ffffc90003127680 EFLAGS: 00000293
RAX: ffffffff81a4877d RBX: 0000000000000001 RCX: ffff88802c4e9e00
RDX: 0000000000000000 RSI: ffffffff8c1eb140 RDI: ffffffff8c1eb100
RBP: ffffffff81a48736 R08: ffffffff81a48766 R09: 1ffffffff25df6a0
R10: dffffc0000000000 R11: fffffbfff25df6a1 R12: ffff88802c4e9e00
R13: dffffc0000000000 R14: 0000000000000001 R15: 00007fd46a47cd9a
 kernel_text_address+0xa7/0xe0 kernel/extable.c:125
 __kernel_text_address+0xd/0x40 kernel/extable.c:79
 unwind_get_return_address+0x5d/0xc0 arch/x86/kernel/unwind_orc.c:369
 arch_stack_walk+0x125/0x1b0 arch/x86/kernel/stacktrace.c:26
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack+0x3f/0x60 mm/kasan/common.c:47
 __kasan_record_aux_stack+0xac/0xc0 mm/kasan/generic.c:541
 __call_rcu_common kernel/rcu/tree.c:2734 [inline]
 call_rcu+0x167/0xa70 kernel/rcu/tree.c:2838
 __destroy_inode+0x2d9/0x650 fs/inode.c:286
 destroy_inode fs/inode.c:309 [inline]
 evict+0x521/0x630 fs/inode.c:682
 __dentry_kill+0x20d/0x630 fs/dcache.c:603
 dput+0x19f/0x2b0 fs/dcache.c:845
 find_next_child fs/libfs.c:551 [inline]
 simple_recursive_removal+0x2bb/0x830 fs/libfs.c:566
 debugfs_remove+0x49/0x70 fs/debugfs/inode.c:827
 kvm_destroy_vm_debugfs arch/x86/kvm/../../../virt/kvm/kvm_main.c:1081 [inline]
 kvm_destroy_vm arch/x86/kvm/../../../virt/kvm/kvm_main.c:1346 [inline]
 kvm_put_kvm+0x12f/0x1300 arch/x86/kvm/../../../virt/kvm/kvm_main.c:1420
 kvm_vm_release+0x46/0x50 arch/x86/kvm/../../../virt/kvm/kvm_main.c:1443
 __fput+0x429/0x8a0 fs/file_table.c:422
 __do_sys_close fs/open.c:1556 [inline]
 __se_sys_close fs/open.c:1541 [inline]
 __x64_sys_close+0x7f/0x110 fs/open.c:1541
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fd46a47cd9a
Code: 48 3d 00 f0 ff ff 77 48 c3 0f 1f 80 00 00 00 00 48 83 ec 18 89 7c 24 0c e8 03 7f 02 00 8b 7c 24 0c 89 c2 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 36 89 d7 89 44 24 0c e8 63 7f 02 00 8b 44 24
RSP: 002b:00007ffd999d8770 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007fd46a47cd9a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004
RBP: 00007fd46a5ad980 R08: 0000001b2d220000 R09: 0000000000000541
R10: 000000008136a0aa R11: 0000000000000293 R12: 00000000000671ea
R13: 0000000000066ef9 R14: 00007ffd999d8930 R15: 00007fd46a434cb0
 </TASK>
rcu: rcu_preempt kthread starved for 10457 jiffies! g63365 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25880 pid:16    tgid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0x1796/0x4a00 kernel/sched/core.c:6746
 __schedule_loop kernel/sched/core.c:6823 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6838
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2582
 rcu_gp_fqs_loop+0x2df/0x1370 kernel/rcu/tree.c:1663
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1862
 kthread+0x2f0/0x390 kernel/kthread.c:388
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 11271 Comm: syz-executor.0 Not tainted 6.9.0-rc4-syzkaller-00173-g3cdb45594619 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline]
RIP: 0010:smp_call_function_many_cond+0x1894/0x2a00 kernel/smp.c:855
Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 15 db 0b 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 c0 d6 0b 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 a4 d6
RSP: 0018:ffffc900045ef960 EFLAGS: 00000293
RAX: ffffffff818a29cc RBX: 1ffff110172a8861 RCX: ffff888051f20000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc900045efb60 R08: ffffffff818a299b R09: 1ffffffff25df6a0
R10: dffffc0000000000 R11: fffffbfff25df6a1 R12: dffffc0000000000
R13: ffff8880b9544308 R14: ffff8880b943f780 R15: 0000000000000001
FS:  000055555bfbe480(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2d02a000 CR3: 000000005bd66000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1023
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2086 [inline]
 text_poke_bp_batch+0x352/0xb30 arch/x86/kernel/alternative.c:2296
 text_poke_flush arch/x86/kernel/alternative.c:2487 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2494
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_disable_cpuslocked+0xce/0x1c0 kernel/jump_label.c:235
 static_key_disable+0x1a/0x20 kernel/jump_label.c:243
 tracepoint_remove_func kernel/tracepoint.c:417 [inline]
 tracepoint_probe_unregister+0x6f2/0x990 kernel/tracepoint.c:551
 bpf_raw_tp_link_release+0x63/0x90 kernel/bpf/syscall.c:3511
 bpf_link_free+0x10e/0x2d0 kernel/bpf/syscall.c:3052
 bpf_link_put_direct kernel/bpf/syscall.c:3093 [inline]
 bpf_link_release+0x7b/0x90 kernel/bpf/syscall.c:3100
 __fput+0x429/0x8a0 fs/file_table.c:422
 __do_sys_close fs/open.c:1556 [inline]
 __se_sys_close fs/open.c:1541 [inline]
 __x64_sys_close+0x7f/0x110 fs/open.c:1541
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fc39a67cd9a
Code: 48 3d 00 f0 ff ff 77 48 c3 0f 1f 80 00 00 00 00 48 83 ec 18 89 7c 24 0c e8 03 7f 02 00 8b 7c 24 0c 89 c2 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 36 89 d7 89 44 24 0c e8 63 7f 02 00 8b 44 24
RSP: 002b:00007ffd382f43f0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fc39a67cd9a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006
RBP: 00007fc39a7ad980 R08: 0000001b2d420000 R09: 000000000000029b
R10: 00000000819d1d7d R11: 0000000000000293 R12: 000000000006711a
R13: 0000000000066f2b R14: 00007ffd382f45b0 R15: 00007fc39a634cb0
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/04/20 08:00 upstream 3cdb45594619 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in net_rx_action
2024/03/19 18:41 upstream 480e035fc4c7 baa80228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in net_rx_action
2024/04/26 01:07 https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing a160e1202ca3 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-usb BUG: soft lockup in net_rx_action
* Struck through repros no longer work on HEAD.