syzbot


INFO: rcu detected stall in tipc_release

Status: fixed on 2021/03/10 01:48
Subsystems: tipc
[Documentation on labels]
Reported-by: syzbot+3654c027d861c6df4b06@syzkaller.appspotmail.com
Fix commit: cc00bcaa5899 netfilter: x_tables: Switch synchronization to RCU
First crash: 1351d, last: 1215d
Cause bisection: introduced by (bisect log) :
commit 5e9eeccc58f3e6bcc99b929670665d2ce047e9c9
Author: Tuong Lien <tuong.t.lien@dektech.com.au>
Date: Wed Jun 3 05:06:01 2020 +0000

  tipc: fix NULL pointer dereference in streaming

Crash: INFO: rcu detected stall in tipc_release (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit cc00bcaa589914096edef7fb87ca5cee4a166b5c
Author: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org>
Date: Wed Nov 25 18:27:22 2020 +0000

  netfilter: x_tables: Switch synchronization to RCU

  
Discussions (1)
Title Replies (including bot) Last reply
INFO: rcu detected stall in tipc_release 2 (5) 2020/12/23 10:24
Fix bisection attempts (4)
Created Duration User Patch Repo Result
2020/12/20 04:55 4h40m bisect fix upstream job log (1)
2020/11/19 07:45 31m bisect fix upstream job log (0) log
2020/10/20 07:11 33m bisect fix upstream job log (0) log
2020/09/20 06:40 31m bisect fix upstream job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (1 GPs behind) idle=2c6/1/0x4000000000000000 softirq=8594/8600 fqs=12 
	(t=10501 jiffies g=7945 q=771)
rcu: rcu_preempt kthread starved for 10429 jiffies! g7945 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
rcu_preempt     I29120    10      2 0x00004000
Call Trace:
 context_switch kernel/sched/core.c:3453 [inline]
 __schedule+0x8e1/0x1eb0 kernel/sched/core.c:4178
 schedule+0xd0/0x2a0 kernel/sched/core.c:4253
 schedule_timeout+0x148/0x250 kernel/time/timer.c:1897
 rcu_gp_fqs_loop kernel/rcu/tree.c:1874 [inline]
 rcu_gp_kthread+0xae5/0x1b50 kernel/rcu/tree.c:2044
 kthread+0x3b5/0x4a0 kernel/kthread.c:291
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:293
NMI backtrace for cpu 1
CPU: 1 PID: 6855 Comm: syz-executor167 Not tainted 5.8.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x18f/0x20d lib/dump_stack.c:118
 nmi_cpu_backtrace.cold+0x70/0xb1 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x1b3/0x223 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_dump_cpu_stacks+0x194/0x1cf kernel/rcu/tree_stall.h:320
 print_cpu_stall kernel/rcu/tree_stall.h:553 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:627 [inline]
 rcu_pending kernel/rcu/tree.c:3489 [inline]
 rcu_sched_clock_irq.cold+0x5b3/0xccc kernel/rcu/tree.c:2504
 update_process_times+0x25/0x60 kernel/time/timer.c:1726
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:176
 tick_sched_timer+0x108/0x290 kernel/time/tick-sched.c:1320
 __run_hrtimer kernel/time/hrtimer.c:1520 [inline]
 __hrtimer_run_queues+0x1d5/0xfc0 kernel/time/hrtimer.c:1584
 hrtimer_interrupt+0x32a/0x930 kernel/time/hrtimer.c:1646
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1080 [inline]
 __sysvec_apic_timer_interrupt+0x142/0x5e0 arch/x86/kernel/apic/apic.c:1097
 asm_call_on_stack+0xf/0x20 arch/x86/entry/entry_64.S:711
 </IRQ>
 __run_on_irqstack arch/x86/include/asm/irq_stack.h:22 [inline]
 run_on_irqstack_cond arch/x86/include/asm/irq_stack.h:48 [inline]
 sysvec_apic_timer_interrupt+0xe0/0x120 arch/x86/kernel/apic/apic.c:1091
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:596
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:765 [inline]
RIP: 0010:lock_release+0x481/0x8d0 kernel/locking/lockdep.c:4980
Code: 00 00 00 00 00 fc ff df 48 c1 e8 03 80 3c 10 00 0f 85 7f 03 00 00 48 83 3d 2b d7 5a 08 00 0f 84 a9 01 00 00 48 8b 3c 24 57 9d <0f> 1f 44 00 00 48 b8 00 00 00 00 00 fc ff df 48 01 c5 48 c7 45 00
RSP: 0018:ffffc90001776cd0 EFLAGS: 00000282
RAX: 1ffffffff1369c08 RBX: ffff8880972f25c0 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: ffffffff89bc1180 RDI: 0000000000000282
RBP: 1ffff920002eed9c R08: 0000000000000001 R09: ffff8880972f2e88
R10: fffffbfff155cb29 R11: 0000000000000000 R12: 0000000000000003
R13: ffffffff87a707d5 R14: 0000000000000004 R15: ffff8880972f25c0
 rcu_lock_release include/linux/rcupdate.h:246 [inline]
 rcu_read_unlock include/linux/rcupdate.h:688 [inline]
 tipc_sk_lookup+0x607/0xab0 net/tipc/socket.c:2974
 tipc_sk_rcv+0x27a/0x1ec0 net/tipc/socket.c:2461
 tipc_node_xmit+0x2b0/0xce0 net/tipc/node.c:1652
 tipc_node_xmit_skb+0xd5/0x140 net/tipc/node.c:1712
 tipc_sk_rcv+0x1754/0x1ec0 net/tipc/socket.c:2491
 tipc_node_xmit+0x2b0/0xce0 net/tipc/node.c:1652
 tipc_sk_push_backlog+0x324/0x790 net/tipc/socket.c:1303
 tipc_sk_filter_connect net/tipc/socket.c:2226 [inline]
 tipc_sk_filter_rcv+0xafe/0x3250 net/tipc/socket.c:2335
 tipc_sk_enqueue net/tipc/socket.c:2415 [inline]
 tipc_sk_rcv+0xd1a/0x1ec0 net/tipc/socket.c:2466
 tipc_node_xmit+0x2b0/0xce0 net/tipc/node.c:1652
 tipc_node_xmit_skb net/tipc/node.c:1712 [inline]
 tipc_node_distr_xmit+0x15c/0x3a0 net/tipc/node.c:1727
 tipc_sk_backlog_rcv+0x155/0x1c0 net/tipc/socket.c:2383
 sk_backlog_rcv include/net/sock.h:996 [inline]
 __release_sock+0x134/0x3a0 net/core/sock.c:2550
 release_sock+0x54/0x1b0 net/core/sock.c:3066
 tipc_release+0xbb1/0x1a70 net/tipc/socket.c:638
 __sock_release+0xcd/0x280 net/socket.c:605
 sock_close+0x18/0x20 net/socket.c:1278
 __fput+0x33c/0x880 fs/file_table.c:281
 task_work_run+0xdd/0x190 kernel/task_work.c:135
 tracehook_notify_resume include/linux/tracehook.h:188 [inline]
 exit_to_usermode_loop arch/x86/entry/common.c:216 [inline]
 __prepare_exit_to_usermode+0x1e9/0x1f0 arch/x86/entry/common.c:246
 do_syscall_64+0x6c/0xe0 arch/x86/entry/common.c:368
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x446df9
Code: Bad RIP value.
RSP: 002b:00007f4766792cb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000021
RAX: 0000000000000005 RBX: 00007f4766792cc0 RCX: 0000000000446df9
RDX: 0000000000000001 RSI: 0000000000000005 RDI: 0000000000000003
RBP: 0000000000000006 R08: 0000000000000001 R09: 0000000000000031
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dcc5c
R13: 00007fff2c969dbf R14: 00007f47667939c0 R15: 20c49ba5e353f7cf

Crashes (4):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/07/06 10:31 upstream 7cc2a8ea1048 51095195 .config console log report syz C ci-upstream-kasan-gce
2020/08/21 06:39 upstream da2968ff879b 1d75fe45 .config console log report ci-upstream-kasan-gce-selinux-root
2020/08/04 12:17 upstream 3208167a865e 196277c4 .config console log report ci-upstream-kasan-gce-root
2020/07/06 08:37 upstream 7cc2a8ea1048 51095195 .config console log report ci-upstream-kasan-gce
* Struck through repros no longer work on HEAD.