syzbot


inconsistent lock state in rxrpc_put_client_conn

Status: fixed on 2020/02/18 14:31
Reported-by: syzbot+3f1fd6b8cbf8702d134e@syzkaller.appspotmail.com
Fix commit: 963485d436cc rxrpc: Fix call RCU cleanup using non-bh-safe locks
First crash: 1536d, last: 1530d
Cause bisection: introduced by (bisect log) :
commit 5273a191dca65a675dc0bcf3909e59c6933e2831
Author: David Howells <dhowells@redhat.com>
Date: Thu Jan 30 21:50:36 2020 +0000

  rxrpc: Fix NULL pointer deref due to call->conn being cleared on disconnect

Crash: inconsistent lock state in rxrpc_put_client_conn (log)
Repro: C syz .config
  
Discussions (7)
Title Replies (including bot) Last reply
[PATCH 4.19 00/97] 4.19.107-stable review 108 (108) 2020/02/28 18:05
[PATCH 5.5 000/150] 5.5.7-stable review 166 (166) 2020/02/28 15:12
[PATCH 5.4 000/135] 5.4.23-stable review 140 (140) 2020/02/28 03:42
[PATCH net] rxrpc: Fix call RCU cleanup using non-bh-safe locks 2 (2) 2020/02/07 10:22
Re: inconsistent lock state in rxrpc_put_client_conn 1 (1) 2020/02/07 07:23
Re: inconsistent lock state in rxrpc_put_client_conn 1 (1) 2020/02/06 13:09
inconsistent lock state in rxrpc_put_client_conn 0 (2) 2020/02/04 05:41
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 inconsistent lock state in rxrpc_put_client_conn C done 6584 1511d 1529d 1/1 fixed on 2020/03/30 09:03
linux-4.14 inconsistent lock state in rxrpc_put_client_conn C 15057 410d 1525d 0/1 upstream: reported C repro on 2020/02/14 22:06

Sample crash report:
================================
WARNING: inconsistent lock state
5.5.0-syzkaller #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
swapper/1/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
ffff8880a83001c8 (&(&local->client_conns_lock)->rlock){+.?.}, at: spin_lock include/linux/spinlock.h:338 [inline]
ffff8880a83001c8 (&(&local->client_conns_lock)->rlock){+.?.}, at: rxrpc_put_one_client_conn net/rxrpc/conn_client.c:948 [inline]
ffff8880a83001c8 (&(&local->client_conns_lock)->rlock){+.?.}, at: rxrpc_put_client_conn+0x6ed/0xc90 net/rxrpc/conn_client.c:1001
{SOFTIRQ-ON-W} state was registered at:
  lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
  __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
  _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
  spin_lock include/linux/spinlock.h:338 [inline]
  rxrpc_get_client_conn net/rxrpc/conn_client.c:304 [inline]
  rxrpc_connect_call+0x358/0x4e30 net/rxrpc/conn_client.c:701
  rxrpc_new_client_call+0x9c0/0x1ad0 net/rxrpc/call_object.c:290
  rxrpc_new_client_call_for_sendmsg net/rxrpc/sendmsg.c:595 [inline]
  rxrpc_do_sendmsg+0xffa/0x1d5f net/rxrpc/sendmsg.c:652
  rxrpc_sendmsg+0x4d6/0x5f0 net/rxrpc/af_rxrpc.c:586
  sock_sendmsg_nosec net/socket.c:652 [inline]
  sock_sendmsg+0xd7/0x130 net/socket.c:672
  ____sys_sendmsg+0x358/0x880 net/socket.c:2343
  ___sys_sendmsg+0x100/0x170 net/socket.c:2397
  __sys_sendmmsg+0x1bf/0x4d0 net/socket.c:2487
  __do_sys_sendmmsg net/socket.c:2516 [inline]
  __se_sys_sendmmsg net/socket.c:2513 [inline]
  __x64_sys_sendmmsg+0x9d/0x100 net/socket.c:2513
  do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
  entry_SYSCALL_64_after_hwframe+0x49/0xbe
irq event stamp: 192014
hardirqs last  enabled at (192014): [<ffffffff87edfb56>] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:160 [inline]
hardirqs last  enabled at (192014): [<ffffffff87edfb56>] _raw_spin_unlock_irqrestore+0x66/0xe0 kernel/locking/spinlock.c:191
hardirqs last disabled at (192013): [<ffffffff87edfecf>] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline]
hardirqs last disabled at (192013): [<ffffffff87edfecf>] _raw_spin_lock_irqsave+0x6f/0xcd kernel/locking/spinlock.c:159
softirqs last  enabled at (191930): [<ffffffff8147568c>] _local_bh_enable+0x1c/0x30 kernel/softirq.c:162
softirqs last disabled at (191931): [<ffffffff8147808b>] invoke_softirq kernel/softirq.c:373 [inline]
softirqs last disabled at (191931): [<ffffffff8147808b>] irq_exit+0x19b/0x1e0 kernel/softirq.c:413

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&(&local->client_conns_lock)->rlock);
  <Interrupt>
    lock(&(&local->client_conns_lock)->rlock);

 *** DEADLOCK ***

1 lock held by swapper/1/0:
 #0: ffffffff89bac140 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2176 [inline]
 #0: ffffffff89bac140 (rcu_callback){....}, at: rcu_core+0x562/0x1390 kernel/rcu/tree.c:2410

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.5.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_usage_bug.cold+0x327/0x378 kernel/locking/lockdep.c:3100
 valid_state kernel/locking/lockdep.c:3111 [inline]
 mark_lock_irq kernel/locking/lockdep.c:3308 [inline]
 mark_lock+0xbb4/0x1220 kernel/locking/lockdep.c:3665
 mark_usage kernel/locking/lockdep.c:3565 [inline]
 __lock_acquire+0x1e8e/0x4a00 kernel/locking/lockdep.c:3908
 lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
 spin_lock include/linux/spinlock.h:338 [inline]
 rxrpc_put_one_client_conn net/rxrpc/conn_client.c:948 [inline]
 rxrpc_put_client_conn+0x6ed/0xc90 net/rxrpc/conn_client.c:1001
 rxrpc_put_connection net/rxrpc/ar-internal.h:965 [inline]
 rxrpc_rcu_destroy_call+0xbd/0x200 net/rxrpc/call_object.c:572
 rcu_do_batch kernel/rcu/tree.c:2186 [inline]
 rcu_core+0x5e1/0x1390 kernel/rcu/tree.c:2410
 rcu_core_si+0x9/0x10 kernel/rcu/tree.c:2419
 __do_softirq+0x262/0x98c kernel/softirq.c:292
 invoke_softirq kernel/softirq.c:373 [inline]
 irq_exit+0x19b/0x1e0 kernel/softirq.c:413
 exiting_irq arch/x86/include/asm/apic.h:536 [inline]
 smp_apic_timer_interrupt+0x1a3/0x610 arch/x86/kernel/apic/apic.c:1137
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
 </IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: 38 3e c6 f9 eb 8a cc cc cc cc cc cc e9 07 00 00 00 0f 00 2d 14 99 5a 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d 04 99 5a 00 fb f4 <c3> cc 55 48 89 e5 41 57 41 56 41 55 41 54 53 e8 4e f5 74 f9 e8 89
RSP: 0018:ffffc90000d3fd68 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff1367612 RBX: ffff8880a9a0a340 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000006 RDI: ffff8880a9a0abd4
RBP: ffffc90000d3fd98 R08: ffff8880a9a0a340 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
R13: ffffffff8aa587c0 R14: 0000000000000000 R15: 0000000000000001
 arch_cpu_idle+0xa/0x10 arch/x86/kernel/process.c:686
 default_idle_call+0x84/0xb0 kernel/sched/idle.c:94
 cpuidle_idle_call kernel/sched/idle.c:154 [inline]
 do_idle+0x3c8/0x6e0 kernel/sched/idle.c:269

Crashes (18198):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/02/09 08:22 upstream d4f309ca4118 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/09 07:34 upstream d4f309ca4118 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/09 07:02 upstream d4f309ca4118 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 23:06 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 22:16 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 21:43 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 20:43 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 19:56 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 17:43 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 16:12 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 13:38 upstream f757165705e9 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 11:06 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 10:15 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 06:58 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 05:31 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 04:55 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 04:47 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/08 04:26 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 03:36 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/08 02:50 upstream 41dcd67e8868 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/07 22:18 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/07 22:08 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 21:28 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2020/02/07 20:51 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 17:57 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 17:26 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 17:11 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 16:05 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 14:57 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 12:12 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 11:23 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 11:03 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 10:35 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 10:02 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 09:42 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 09:11 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 09:03 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/07 08:38 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce
2020/02/07 07:56 upstream 90568ecf5615 06150bf1 .config console log report syz C ci-upstream-kasan-gce-root
2020/02/04 00:37 net-old 3d80c653f996 93e5e335 .config console log report syz C ci-upstream-net-this-kasan-gce
2020/02/07 15:24 upstream 90568ecf5615 06150bf1 .config console log report syz ci-upstream-kasan-gce
2020/02/07 10:29 upstream 90568ecf5615 06150bf1 .config console log report syz ci-upstream-kasan-gce
2020/02/09 08:48 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/09 05:00 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce-smack-root
2020/02/09 02:14 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce-root
2020/02/09 05:40 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/07 17:34 net-old f798a5a0a605 06150bf1 .config console log report ci-upstream-net-this-kasan-gce
2020/02/03 21:50 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/03 21:11 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/09 10:38 net-next-old 33b40134e5cf 6ece2ea5 .config console log report ci-upstream-net-kasan-gce
2020/02/09 10:12 net-next-old 33b40134e5cf 6ece2ea5 .config console log report ci-upstream-net-kasan-gce
2020/02/09 09:24 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 09:11 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 08:59 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 08:35 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 08:24 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 08:15 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 08:03 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 07:53 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 07:43 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/10 00:53 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/10 00:30 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/10 00:06 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 23:46 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 23:27 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 23:01 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 22:36 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 22:14 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 21:55 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 21:08 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 20:05 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 19:49 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 18:40 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 17:56 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 16:46 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 16:28 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 15:22 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 14:50 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 13:36 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 13:13 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 12:43 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 12:24 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 12:03 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 11:40 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 11:18 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 10:54 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 10:30 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 10:22 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 10:05 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 09:56 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 09:44 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.