syzbot

 sign-in | mailing list | source | docs
🐞 Open [712] 🐞 Fixed [297] 🐞 Invalid [838] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in tcp_write_timer

Status: auto-closed as invalid on 2020/08/26 06:46
Reported-by: syzbot+fed2dabde7e5d4a0f341@syzkaller.appspotmail.com
First crash: 1512d, last: 1458d
▶ ▼ Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in tcp_write_timer (2) bpf 2 1178d 1233d 0/26 auto-closed as invalid on 2021/05/03 11:59
upstream INFO: rcu detected stall in tcp_write_timer net 3 1727d 1722d 0/26 auto-closed as invalid on 2019/10/25 14:11
upstream INFO: rcu detected stall in tcp_write_timer (3) net 1 1045d 1045d 0/26 auto-closed as invalid on 2021/09/13 13:17
linux-4.14 INFO: rcu detected stall in tcp_write_timer 4 1329d 1572d 0/1 auto-closed as invalid on 2021/01/02 05:45

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	0-...!: (10500 ticks this GP) idle=c82/1/0x4000000000000002 softirq=263306/263306 fqs=0 
rcu: 	 (t=10500 jiffies g=377945 q=56)
rcu: rcu_preempt kthread starved for 10500 jiffies! g377945 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: RCU grace-period kthread stack dump:
rcu_preempt     I29456    10      2 0x80000000
Call Trace:
 schedule+0x8d/0x1b0 kernel/sched/core.c:3559
 schedule_timeout+0x4d1/0xf20 kernel/time/timer.c:1806
 rcu_gp_kthread+0xcee/0x2060 kernel/rcu/tree.c:2202
 kthread+0x34a/0x420 kernel/kthread.c:246
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
NMI backtrace for cpu 0
CPU: 0 PID: 17683 Comm: syz-executor.3 Not tainted 4.19.118-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x188/0x20d lib/dump_stack.c:118
 nmi_cpu_backtrace.cold+0x63/0xa2 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x1a6/0x1eb lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_dump_cpu_stacks+0x170/0x1bb kernel/rcu/tree.c:1340
 print_cpu_stall kernel/rcu/tree.c:1478 [inline]
 check_cpu_stall kernel/rcu/tree.c:1550 [inline]
 __rcu_pending kernel/rcu/tree.c:3293 [inline]
 rcu_pending kernel/rcu/tree.c:3336 [inline]
 rcu_check_callbacks.cold+0x634/0xddc kernel/rcu/tree.c:2682
 update_process_times+0x2a/0x70 kernel/time/timer.c:1638
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:168
 tick_sched_timer+0x42/0x130 kernel/time/tick-sched.c:1278
 __run_hrtimer kernel/time/hrtimer.c:1401 [inline]
 __hrtimer_run_queues+0x2fc/0xd50 kernel/time/hrtimer.c:1463
 hrtimer_interrupt+0x312/0x770 kernel/time/hrtimer.c:1521
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1067 [inline]
 smp_apic_timer_interrupt+0x10c/0x550 arch/x86/kernel/apic/apic.c:1092
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:893
RIP: 0010:sfq_dequeue+0x1ae/0xd40 net/sched/sch_sfq.c:496
Code: 9d e8 02 00 00 48 8d 7b 12 48 89 f8 48 c1 e8 03 42 0f b6 14 30 48 89 f8 83 e0 07 83 c0 01 38 d0 7c 08 84 d2 0f 85 ab 0a 00 00 <44> 0f b7 63 12 48 8b 9d c0 02 00 00 4d 89 e7 4d 6b e4 38 4c 01 e3
RSP: 0018:ffff8880ae6075c0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
RAX: 0000000000000003 RBX: ffff88804c701270 RCX: ffffffff85bb25c7
RDX: 0000000000000000 RSI: ffffffff85bb2502 RDI: ffff88804c701282
RBP: ffff88808e24cc00 R08: ffff88800002c540 R09: ffffed1011c4999e
R10: ffffed1011c4999d R11: ffff88808e24cceb R12: 0000000000000000
R13: ffff88804c70128a R14: dffffc0000000000 R15: 0000000000000002
 dequeue_skb net/sched/sch_generic.c:282 [inline]
 qdisc_restart net/sched/sch_generic.c:385 [inline]
 __qdisc_run+0x1b7/0x1720 net/sched/sch_generic.c:403
 __dev_xmit_skb net/core/dev.c:3492 [inline]
 __dev_queue_xmit+0x1dad/0x3040 net/core/dev.c:3814
 neigh_hh_output include/net/neighbour.h:491 [inline]
 neigh_output include/net/neighbour.h:499 [inline]
 ip_finish_output2+0x104d/0x1740 net/ipv4/ip_output.c:229
 ip_finish_output+0x721/0xcd0 net/ipv4/ip_output.c:317
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip_output+0x1fb/0x5f0 net/ipv4/ip_output.c:405
 dst_output include/net/dst.h:447 [inline]
 ip_local_out+0xaf/0x170 net/ipv4/ip_output.c:124
 __ip_queue_xmit+0x85a/0x1bc0 net/ipv4/ip_output.c:505
 __tcp_transmit_skb+0x19d5/0x3750 net/ipv4/tcp_output.c:1146
 tcp_transmit_skb net/ipv4/tcp_output.c:1162 [inline]
 tcp_write_wakeup+0x4ed/0x610 net/ipv4/tcp_output.c:3720
 tcp_send_probe0+0x46/0x3f5 net/ipv4/tcp_output.c:3742
 tcp_probe_timer net/ipv4/tcp_timer.c:385 [inline]
 tcp_write_timer_handler+0x78a/0x8d0 net/ipv4/tcp_timer.c:602
 tcp_write_timer+0x103/0x1b0 net/ipv4/tcp_timer.c:618
 call_timer_fn+0x177/0x700 kernel/time/timer.c:1326
 expire_timers kernel/time/timer.c:1363 [inline]
 __run_timers kernel/time/timer.c:1684 [inline]
 __run_timers kernel/time/timer.c:1652 [inline]
 run_timer_softirq+0x5d0/0x1540 kernel/time/timer.c:1697
 __do_softirq+0x26c/0x93c kernel/softirq.c:292
 do_softirq_own_stack+0x2a/0x40 arch/x86/entry/entry_64.S:1091
 </IRQ>
 do_softirq.part.0+0x10f/0x160 kernel/softirq.c:336
 do_softirq kernel/softirq.c:328 [inline]
 __local_bh_enable_ip+0x20e/0x270 kernel/softirq.c:189
 local_bh_enable include/linux/bottom_half.h:32 [inline]
 rcu_read_unlock_bh include/linux/rcupdate.h:721 [inline]
 ip6_finish_output2+0xbd2/0x2540 net/ipv6/ip6_output.c:121
 ip6_fragment+0x1386/0x3410 net/ipv6/ip6_output.c:846
 ip6_finish_output+0x6a2/0xbe0 net/ipv6/ip6_output.c:152
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip6_output+0x209/0x760 net/ipv6/ip6_output.c:171
 dst_output include/net/dst.h:447 [inline]
 ip6_local_out+0xaf/0x170 net/ipv6/output_core.c:178
 ip6_send_skb+0xb3/0x300 net/ipv6/ip6_output.c:1699
 ip6_push_pending_frames+0xbd/0xe0 net/ipv6/ip6_output.c:1719
 rawv6_push_pending_frames net/ipv6/raw.c:618 [inline]
 rawv6_sendmsg+0x2a16/0x3640 net/ipv6/raw.c:959
 inet_sendmsg+0x12e/0x590 net/ipv4/af_inet.c:798
 sock_sendmsg_nosec net/socket.c:622 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:632
 __sys_sendto+0x21a/0x330 net/socket.c:1787
 __do_sys_sendto net/socket.c:1799 [inline]
 __se_sys_sendto net/socket.c:1795 [inline]
 __x64_sys_sendto+0xdd/0x1b0 net/socket.c:1795
 do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c829
Code: 0d b7 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fed6635fc78 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 0000000000500d00 RCX: 000000000045c829
RDX: 000000000000ffa7 RSI: 0000000020000000 RDI: 0000000000000004
RBP: 000000000078bf00 R08: 0000000020000180 R09: 0000000000000018
R10: 0000000000006fe0 R11: 0000000000000246 R12: 00000000ffffffff
R13: 0000000000000a09 R14: 00000000004ccbed R15: 00007fed663606d4
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 4724 Comm: kworker/u4:24 Not tainted 4.19.118-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: krdsd rds_connect_worker
RIP: 0010:pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:435 [inline]
RIP: 0010:__pv_queued_spin_lock_slowpath+0x3a2/0xb20 kernel/locking/qspinlock.c:474
Code: eb c6 45 01 01 41 bc 00 80 00 00 48 c1 e9 03 83 e3 07 41 be 01 00 00 00 48 b8 00 00 00 00 00 fc ff df 4c 8d 2c 01 eb 0c f3 90 <41> 83 ec 01 0f 84 a8 04 00 00 41 0f b6 45 00 38 d8 7f 08 84 c0 0f
RSP: 0018:ffff88820224f158 EFLAGS: 00000206
RAX: 0000000000000003 RBX: 0000000000000000 RCX: 1ffff11011c4999d
RDX: 0000000000000001 RSI: ffffffff816cbd15 RDI: 0000000000000286
RBP: ffff88808e24cce8 R08: ffff888055f742c0 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: 00000000000075c7
R13: ffffed1011c4999d R14: 0000000000000001 R15: ffff8880ae72cfc0
FS:  0000000000000000(0000) GS:ffff8880ae700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000043e060 CR3: 0000000050f14000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Call Trace:
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:679 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:53 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:88 [inline]
 do_raw_spin_lock+0x19a/0x240 kernel/locking/spinlock_debug.c:113
 spin_lock include/linux/spinlock.h:329 [inline]
 __dev_xmit_skb net/core/dev.c:3473 [inline]
 __dev_queue_xmit+0x14aa/0x3040 net/core/dev.c:3814
 neigh_hh_output include/net/neighbour.h:491 [inline]
 neigh_output include/net/neighbour.h:499 [inline]
 ip_finish_output2+0x104d/0x1740 net/ipv4/ip_output.c:229
 ip_finish_output+0x721/0xcd0 net/ipv4/ip_output.c:317
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip_output+0x1fb/0x5f0 net/ipv4/ip_output.c:405
 dst_output include/net/dst.h:447 [inline]
 ip_local_out+0xaf/0x170 net/ipv4/ip_output.c:124
 __ip_queue_xmit+0x85a/0x1bc0 net/ipv4/ip_output.c:505
 __tcp_transmit_skb+0x19d5/0x3750 net/ipv4/tcp_output.c:1146
 tcp_transmit_skb net/ipv4/tcp_output.c:1162 [inline]
 tcp_connect+0x2cda/0x3df0 net/ipv4/tcp_output.c:3521
 tcp_v4_connect+0x13fe/0x1a80 net/ipv4/tcp_ipv4.c:315
 __inet_stream_connect+0x7d8/0xdf0 net/ipv4/af_inet.c:655
 inet_stream_connect+0x53/0xa0 net/ipv4/af_inet.c:719
 rds_tcp_conn_path_connect+0x620/0x880 net/rds/tcp_connect.c:172
 rds_connect_worker+0x194/0x2b0 net/rds/threads.c:175
 process_one_work+0x91f/0x1640 kernel/workqueue.c:2155
 worker_thread+0x96/0xe20 kernel/workqueue.c:2298
 kthread+0x34a/0x420 kernel/kthread.c:246
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/04/28 06:45 linux-4.19.y 7edd66cf6167 0ce7569e .config console log report ci2-linux-4-19
2020/03/05 06:10 linux-4.19.y a083db76118d 576fb9bc .config console log report ci2-linux-4-19
* Struck through repros no longer work on HEAD.