syzbot

 sign-in | mailing list | source | docs
🐞 Open [985] ≡ Subsystems 🐞 Fixed [5216] 🐞 Invalid [12474] ⬇ Missing Backports [82] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in tcp_write_timer (3)

Status: auto-closed as invalid on 2021/09/13 13:17
Subsystems: net
[Documentation on labels]
First crash: 1039d, last: 1039d
▶ ▼ Similar bugs (9)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in tcp_write_timer (2) bpf 2 1172d 1227d 0/26 auto-closed as invalid on 2021/05/03 11:59
upstream INFO: rcu detected stall in tcp_write_timer net 3 1721d 1716d 0/26 auto-closed as invalid on 2019/10/25 14:11
linux-4.14 INFO: rcu detected stall in tcp_write_timer 4 1323d 1566d 0/1 auto-closed as invalid on 2021/01/02 05:45
linux-4.19 INFO: rcu detected stall in tcp_write_timer 2 1452d 1506d 0/1 auto-closed as invalid on 2020/08/26 06:46
linux-4.19 BUG: soft lockup in tcp_write_timer (3) 2 475d 488d 0/1 upstream: reported on 2022/12/17 21:41
linux-4.19 BUG: soft lockup in tcp_write_timer (2) 2 885d 948d 0/1 auto-closed as invalid on 2022/03/16 10:56
linux-4.19 BUG: soft lockup in tcp_write_timer 1 1114d 1114d 0/1 auto-closed as invalid on 2021/07/30 14:52
upstream BUG: soft lockup in tcp_write_timer (2) kvm 1 724d 724d 0/26 auto-closed as invalid on 2022/06/24 22:31
upstream BUG: soft lockup in tcp_write_timer (3) net 6 101d 209d 0/26 closed as invalid on 2024/03/18 17:07

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	0-...!: (10500 ticks this GP) idle=e92/1/0x4000000000000000 softirq=63532/63532 fqs=7 
	(t=10502 jiffies g=102717 q=194)
rcu: rcu_preempt kthread starved for 10488 jiffies! g102717 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28800 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 context_switch kernel/sched/core.c:4339 [inline]
 __schedule+0x916/0x23e0 kernel/sched/core.c:5147
 schedule+0xcf/0x270 kernel/sched/core.c:5226
 schedule_timeout+0x14a/0x250 kernel/time/timer.c:1892
 rcu_gp_fqs_loop kernel/rcu/tree.c:2004 [inline]
 rcu_gp_kthread+0xd07/0x2300 kernel/rcu/tree.c:2177
 kthread+0x3b1/0x4a0 kernel/kthread.c:313
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9156 Comm: kworker/1:3 Not tainted 5.13.0-rc6-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: wg-crypt-wg0 wg_packet_tx_worker
RIP: 0010:arch_safe_halt arch/x86/include/asm/irqflags.h:90 [inline]
RIP: 0010:kvm_wait arch/x86/kernel/kvm.c:888 [inline]
RIP: 0010:kvm_wait+0xb2/0x100 arch/x86/kernel/kvm.c:871
Code: 89 74 24 0c 48 89 3c 24 e8 cb 30 48 00 8b 74 24 0c 48 8b 3c 24 eb 82 e8 ec 35 48 00 e9 07 00 00 00 0f 00 2d 90 10 36 08 fb f4 <eb> 98 e9 07 00 00 00 0f 00 2d 80 10 36 08 f4 eb bf 89 74 24 0c 48
RSP: 0018:ffffc90001eaf5f0 EFLAGS: 00000206
RAX: 00000000001fd0b2 RBX: 0000000000000000 RCX: 1ffffffff204f18a
RDX: 0000000000000000 RSI: 0000000000000803 RDI: 0000000000000000
RBP: ffff88802be240f0 R08: 0000000000000001 R09: ffffffff90227977
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
R13: ffffed10057c481e R14: 0000000000000001 R15: ffff8880b9d36400
FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000560bd02b4ac8 CR3: 000000002fae4000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 pv_wait arch/x86/include/asm/paravirt.h:597 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline]
 __pv_queued_spin_lock_slowpath+0x8b8/0xb40 kernel/locking/qspinlock.c:508
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:585 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:85 [inline]
 do_raw_spin_lock+0x200/0x2b0 kernel/locking/spinlock_debug.c:113
 spin_lock include/linux/spinlock.h:354 [inline]
 __dev_xmit_skb net/core/dev.c:3874 [inline]
 __dev_queue_xmit+0x178e/0x3130 net/core/dev.c:4214
 neigh_hh_output include/net/neighbour.h:499 [inline]
 neigh_output include/net/neighbour.h:508 [inline]
 ip_finish_output2+0xeec/0x21f0 net/ipv4/ip_output.c:230
 __ip_finish_output net/ipv4/ip_output.c:308 [inline]
 __ip_finish_output+0x396/0x640 net/ipv4/ip_output.c:290
 ip_finish_output+0x35/0x200 net/ipv4/ip_output.c:318
 NF_HOOK_COND include/linux/netfilter.h:290 [inline]
 ip_output+0x196/0x310 net/ipv4/ip_output.c:432
 dst_output include/net/dst.h:448 [inline]
 ip_local_out+0xaf/0x1a0 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x5a3/0x9c0 net/ipv4/ip_tunnel_core.c:82
 send4+0x454/0xe30 drivers/net/wireguard/socket.c:85
 wg_socket_send_skb_to_peer+0x192/0x220 drivers/net/wireguard/socket.c:174
 wg_packet_create_data_done drivers/net/wireguard/send.c:251 [inline]
 wg_packet_tx_worker+0x1a7/0x710 drivers/net/wireguard/send.c:276
 process_one_work+0x98d/0x1600 kernel/workqueue.c:2276
 worker_thread+0x64c/0x1120 kernel/workqueue.c:2422
 kthread+0x3b1/0x4a0 kernel/kthread.c:313
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
NMI backtrace for cpu 0
CPU: 0 PID: 8724 Comm: syz-executor.3 Not tainted 5.13.0-rc6-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0x141/0x1d7 lib/dump_stack.c:120
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_dump_cpu_stacks+0x25e/0x3f0 kernel/rcu/tree_stall.h:341
 print_cpu_stall kernel/rcu/tree_stall.h:624 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:699 [inline]
 rcu_pending kernel/rcu/tree.c:3911 [inline]
 rcu_sched_clock_irq.cold+0x3f1/0x74b kernel/rcu/tree.c:2649
 update_process_times+0x16d/0x200 kernel/time/timer.c:1796
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1374
 __run_hrtimer kernel/time/hrtimer.c:1537 [inline]
 __hrtimer_run_queues+0x1c0/0xe40 kernel/time/hrtimer.c:1601
 hrtimer_interrupt+0x330/0xa00 kernel/time/hrtimer.c:1663
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1089 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x540 arch/x86/kernel/apic/apic.c:1106
 sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1100
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:647
RIP: 0010:list_add_tail include/linux/list.h:100 [inline]
RIP: 0010:list_move_tail include/linux/list.h:228 [inline]
RIP: 0010:fq_codel_dequeue+0x832/0x1f50 net/sched/sch_fq_codel.c:299
Code: 03 42 80 3c 28 00 0f 85 d3 13 00 00 48 89 d8 48 89 5d 08 48 c1 e8 03 42 80 3c 28 00 0f 85 3f 14 00 00 48 89 2b e8 5e 30 3a fa <49> 8d ac 24 40 03 00 00 48 89 e8 48 c1 e8 03 42 80 3c 28 00 0f 85
RSP: 0018:ffffc90000007328 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff88802be24338 RCX: 0000000000000100
RDX: ffff88806804b880 RSI: ffffffff873aab92 RDI: ffff88802be24340
RBP: ffff88802be24338 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff873aab26 R11: 0000000000000000 R12: ffff88802be24000
R13: dffffc0000000000 R14: ffff88806d358110 R15: ffff88802be24338
 dequeue_skb net/sched/sch_generic.c:286 [inline]
 qdisc_restart net/sched/sch_generic.c:391 [inline]
 __qdisc_run+0x1ae/0x1680 net/sched/sch_generic.c:409
 __dev_xmit_skb net/core/dev.c:3893 [inline]
 __dev_queue_xmit+0x276c/0x3130 net/core/dev.c:4214
 neigh_hh_output include/net/neighbour.h:499 [inline]
 neigh_output include/net/neighbour.h:508 [inline]
 ip_finish_output2+0xeec/0x21f0 net/ipv4/ip_output.c:230
 __ip_finish_output net/ipv4/ip_output.c:308 [inline]
 __ip_finish_output+0x396/0x640 net/ipv4/ip_output.c:290
 ip_finish_output+0x35/0x200 net/ipv4/ip_output.c:318
 NF_HOOK_COND include/linux/netfilter.h:290 [inline]
 ip_output+0x196/0x310 net/ipv4/ip_output.c:432
 dst_output include/net/dst.h:448 [inline]
 ip_local_out net/ipv4/ip_output.c:126 [inline]
 __ip_queue_xmit+0x905/0x1a20 net/ipv4/ip_output.c:533
 __tcp_transmit_skb+0x1889/0x38f0 net/ipv4/tcp_output.c:1405
 tcp_transmit_skb net/ipv4/tcp_output.c:1423 [inline]
 __tcp_retransmit_skb+0x5d9/0x2a50 net/ipv4/tcp_output.c:3233
 tcp_retransmit_skb+0x2a/0x360 net/ipv4/tcp_output.c:3256
 tcp_retransmit_timer+0xf5e/0x3450 net/ipv4/tcp_timer.c:539
 tcp_write_timer_handler+0x5b8/0xb00 net/ipv4/tcp_timer.c:622
 tcp_write_timer+0xa2/0x2b0 net/ipv4/tcp_timer.c:642
 call_timer_fn+0x1a5/0x6b0 kernel/time/timer.c:1431
 expire_timers kernel/time/timer.c:1476 [inline]
 __run_timers.part.0+0x67c/0xa50 kernel/time/timer.c:1745
 __run_timers kernel/time/timer.c:1726 [inline]
 run_timer_softirq+0xb3/0x1d0 kernel/time/timer.c:1758
 __do_softirq+0x29b/0x9f6 kernel/softirq.c:559
 invoke_softirq kernel/softirq.c:433 [inline]
 __irq_exit_rcu+0x136/0x200 kernel/softirq.c:637
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649
 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1100
 </IRQ>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:647
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:161 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0x38/0x70 kernel/locking/spinlock.c:191
Code: 74 24 10 e8 6a 36 41 f8 48 89 ef e8 02 af 41 f8 81 e3 00 02 00 00 75 25 9c 58 f6 c4 02 75 2d 48 85 db 74 01 fb bf 01 00 00 00 <e8> 73 a5 35 f8 65 8b 05 6c a7 e8 76 85 c0 74 0a 5b 5d c3 e8 50 a3
RSP: 0018:ffffc9000180f3c8 EFLAGS: 00000206
RAX: 0000000000000006 RBX: 0000000000000200 RCX: 1ffffffff1b925c1
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffffffff8c079840 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff817aebe8 R11: 0000000000000000 R12: 0000607f460bd000
R13: 0000000000100000 R14: 0000000000000000 R15: 0000000000033ed4
 spin_unlock_irqrestore include/linux/spinlock.h:409 [inline]
 free_percpu mm/percpu.c:2121 [inline]
 free_percpu+0x4bf/0xae0 mm/percpu.c:2081
 xt_percpu_counter_free+0x96/0xc0 net/netfilter/x_tables.c:1889
 cleanup_entry+0x24f/0x300 net/ipv4/netfilter/ip_tables.c:655
 __do_replace+0x628/0x870 net/ipv4/netfilter/ip_tables.c:1084
 compat_do_replace.constprop.0+0x29c/0x470 net/ipv6/netfilter/ip6_tables.c:1532
 do_ipt_set_ctl+0x5b0/0xb80 net/ipv4/netfilter/ip_tables.c:1626
 nf_setsockopt+0x83/0xe0 net/netfilter/nf_sockopt.c:101
 ip_setsockopt+0x3c3/0x3a40 net/ipv4/ip_sockglue.c:1435
 tcp_setsockopt+0x136/0x24a0 net/ipv4/tcp.c:3656
 __sys_setsockopt+0x2db/0x610 net/socket.c:2117
 __do_compat_sys_socketcall+0x4ef/0x660 net/compat.c:492
 do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
 __do_fast_syscall_32+0x67/0xe0 arch/x86/entry/common.c:143
 do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:168
 entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
RIP: 0023:0xf7fb2549
Code: 03 74 c0 01 10 05 03 74 b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d b4 26 00 00 00 00 8d b4 26 00 00 00 00
RSP: 002b:00000000086cf650 EFLAGS: 00000206 ORIG_RAX: 0000000000000066
RAX: ffffffffffffffda RBX: 000000000000000e RCX: 00000000086cf668
RDX: 00000000086cf7c0 RSI: 00000000086cf6d8 RDI: 00000000086cf72c
RBP: 0000000008170fdc R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/06/15 13:11 upstream 009c9aa5be65 58636922 .config console log report info ci-upstream-kasan-gce-386 INFO: rcu detected stall in tcp_write_timer
* Struck through repros no longer work on HEAD.