BUG: soft lockup in net_tx_action

Status: upstream: reported C repro on 2023/06/23 13:57
Bug presence: origin:upstream
[Documentation on labels]
First crash: 247d, last: 5d14h
Bug presence (1)
Date Name Commit Repro Result
2023/06/24 upstream (ToT) a92b7d26c743 C [report] BUG: soft lockup in net_tx_action
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in net_tx_action net C unreliable done 35 68d 969d 0/26 upstream: reported C repro on 2021/07/01 15:50
linux-4.19 BUG: soft lockup in net_tx_action 1 898d 898d 0/1 auto-closed as invalid on 2022/01/08 10:24
android-5-15 BUG: soft lockup in net_tx_action 1 187d 187d 0/2 auto-obsoleted due to no activity on 2023/11/20 11:28
Fix bisection attempts (6)
Created Duration User Patch Repo Result
2024/02/20 08:26 1h59m bisect fix linux-5.15.y job log (0) log
2024/01/10 06:15 2h01m bisect fix linux-5.15.y job log (0) log
2023/12/11 01:38 2h21m bisect fix linux-5.15.y job log (0) log
2023/11/10 17:51 2h08m bisect fix linux-5.15.y job log (0) log
2023/10/07 22:37 2h13m bisect fix linux-5.15.y job log (0) log
2023/08/18 03:56 3h13m bisect fix linux-5.15.y job log (0) log

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 143s! [swapper/0:0]
Modules linked in:
irq event stamp: 256677
hardirqs last  enabled at (256676): [<ffffffff8137d332>] kvm_wait+0x1a2/0x200
hardirqs last disabled at (256677): [<ffffffff8a1afc0a>] sysvec_apic_timer_interrupt+0xa/0xb0 arch/x86/kernel/apic/apic.c:1096
softirqs last  enabled at (255866): [<ffffffff814d4d05>] invoke_softirq kernel/softirq.c:432 [inline]
softirqs last  enabled at (255866): [<ffffffff814d4d05>] __irq_exit_rcu+0x155/0x240 kernel/softirq.c:636
softirqs last disabled at (255879): [<ffffffff814d4d05>] invoke_softirq kernel/softirq.c:432 [inline]
softirqs last disabled at (255879): [<ffffffff814d4d05>] __irq_exit_rcu+0x155/0x240 kernel/softirq.c:636
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.118-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/27/2023
RIP: 0010:native_safe_halt arch/x86/include/asm/irqflags.h:51 [inline]
RIP: 0010:arch_safe_halt arch/x86/include/asm/irqflags.h:89 [inline]
RIP: 0010:kvm_wait+0x1b4/0x200 arch/x86/kernel/kvm.c:918
Code: e0 48 c1 e8 03 42 0f b6 04 28 84 c0 75 42 45 0f b6 34 24 e8 fe 97 4e 00 44 3a 74 24 1c 75 10 66 90 0f 00 2d 3e 84 50 09 fb f4 <e9> c8 fe ff ff fb e9 c2 fe ff ff 44 89 e1 80 e1 07 38 c1 0f 8c 54
RSP: 0018:ffffc90000007a60 EFLAGS: 00000246
RAX: a5010cc06e0c9600 RBX: 1ffff92000000f50 RCX: ffffffff8162db08
RDX: dffffc0000000000 RSI: ffffffff8a8afc60 RDI: ffffffff8ad86100
RBP: ffffc90000007b30 R08: dffffc0000000000 R09: fffffbfff1f79639
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff88807675e0f0
R13: dffffc0000000000 R14: 0000000000000003 R15: ffffc90000007aa0
FS:  0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005636613a6030 CR3: 000000007d21d000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 pv_wait arch/x86/include/asm/paravirt.h:597 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline]
 __pv_queued_spin_lock_slowpath+0x6bc/0xc40 kernel/locking/qspinlock.c:508
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:585 [inline]
 queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
 queued_spin_lock include/asm-generic/qspinlock.h:85 [inline]
 do_raw_spin_lock+0x269/0x370 kernel/locking/spinlock_debug.c:115
 spin_lock include/linux/spinlock.h:363 [inline]
 net_tx_action+0x6c5/0x8e0 net/core/dev.c:5049
 __do_softirq+0x3b3/0x93a kernel/softirq.c:558
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:636
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:648
 sysvec_apic_timer_interrupt+0x91/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:native_save_fl arch/x86/include/asm/irqflags.h:22 [inline]
RIP: 0010:arch_local_save_flags arch/x86/include/asm/irqflags.h:70 [inline]
RIP: 0010:arch_irqs_disabled arch/x86/include/asm/irqflags.h:132 [inline]
RIP: 0010:acpi_safe_halt drivers/acpi/processor_idle.c:110 [inline]
RIP: 0010:acpi_idle_do_entry+0x10f/0x340 drivers/acpi/processor_idle.c:570
Code: 9e 5b f7 48 83 e3 08 0f 85 0a 01 00 00 4c 8d 74 24 20 e8 94 0b 62 f7 0f 1f 44 00 00 e8 aa 9a 5b f7 0f 00 2d 13 00 be 00 fb f4 <4c> 89 f3 48 c1 eb 03 42 80 3c 3b 00 74 08 4c 89 f7 e8 0b f7 a4 f7
RSP: 0018:ffffffff8c607b80 EFLAGS: 000002d3
RAX: ffffffff8a245fa6 RBX: 0000000000000000 RCX: ffffffff8c6bb5c0
RDX: 0000000000000000 RSI: ffffffff8a8afc60 RDI: ffffffff8ad86100
RBP: ffffffff8c607c10 R08: ffffffff81866b60 R09: fffffbfff18d76b9
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffffffff18c0f70
R13: ffff8880129df804 R14: ffffffff8c607ba0 R15: dffffc0000000000
 acpi_idle_enter+0x352/0x4f0 drivers/acpi/processor_idle.c:705
 cpuidle_enter_state+0x521/0xef0 drivers/cpuidle/cpuidle.c:237
 cpuidle_enter+0x59/0x90 drivers/cpuidle/cpuidle.c:351
 call_cpuidle kernel/sched/idle.c:158 [inline]
 cpuidle_idle_call kernel/sched/idle.c:239 [inline]
 do_idle+0x3e4/0x670 kernel/sched/idle.c:306
 cpu_startup_entry+0x14/0x20 kernel/sched/idle.c:403
 start_kernel+0x491/0x53a init/main.c:1144

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/06/23 13:57 linux-5.15.y f67653019430 79782afc .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: soft lockup in net_tx_action
* Struck through repros no longer work on HEAD.