syzbot


inconsistent lock state in rxrpc_put_client_connection_id

Status: fixed on 2020/02/18 14:31
Subsystems: net afs
[Documentation on labels]
Reported-by: syzbot+d82f3ac8d87e7ccbb2c9@syzkaller.appspotmail.com
Fix commit: 963485d436cc rxrpc: Fix call RCU cleanup using non-bh-safe locks
First crash: 1765d, last: 1759d
Cause bisection: introduced by (bisect log) :
commit 5273a191dca65a675dc0bcf3909e59c6933e2831
Author: David Howells <dhowells@redhat.com>
Date: Thu Jan 30 21:50:36 2020 +0000

  rxrpc: Fix NULL pointer deref due to call->conn being cleared on disconnect

Crash: inconsistent lock state in rxrpc_put_client_connection_id (log)
Repro: C syz .config
  
Discussions (5)
Title Replies (including bot) Last reply
[PATCH 4.19 00/97] 4.19.107-stable review 108 (108) 2020/02/28 18:05
[PATCH 5.5 000/150] 5.5.7-stable review 166 (166) 2020/02/28 15:12
[PATCH 5.4 000/135] 5.4.23-stable review 140 (140) 2020/02/28 03:42
[PATCH net] rxrpc: Fix call RCU cleanup using non-bh-safe locks 2 (2) 2020/02/07 10:22
inconsistent lock state in rxrpc_put_client_connection_id 0 (3) 2020/02/06 01:29
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 inconsistent lock state in rxrpc_put_client_connection_id 8 1647d 1745d 0/1 auto-closed as invalid on 2020/09/29 08:26
linux-4.19 inconsistent lock state in rxrpc_put_client_connection_id C done 89 1740d 1757d 1/1 fixed on 2020/03/30 09:03

Sample crash report:
================================
WARNING: inconsistent lock state
5.5.0-syzkaller #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
swapper/1/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
ffffffff8a8c84b8 (rxrpc_conn_id_lock){+.?.}, at: spin_lock include/linux/spinlock.h:338 [inline]
ffffffff8a8c84b8 (rxrpc_conn_id_lock){+.?.}, at: rxrpc_put_client_connection_id net/rxrpc/conn_client.c:138 [inline]
ffffffff8a8c84b8 (rxrpc_conn_id_lock){+.?.}, at: rxrpc_put_client_connection_id+0x73/0xe0 net/rxrpc/conn_client.c:135
{SOFTIRQ-ON-W} state was registered at:
  lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
  __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
  _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
  spin_lock include/linux/spinlock.h:338 [inline]
  rxrpc_get_client_connection_id net/rxrpc/conn_client.c:109 [inline]
  rxrpc_alloc_client_connection net/rxrpc/conn_client.c:193 [inline]
  rxrpc_get_client_conn net/rxrpc/conn_client.c:340 [inline]
  rxrpc_connect_call+0x954/0x4e30 net/rxrpc/conn_client.c:701
  rxrpc_new_client_call+0x9c0/0x1ad0 net/rxrpc/call_object.c:290
  rxrpc_new_client_call_for_sendmsg net/rxrpc/sendmsg.c:595 [inline]
  rxrpc_do_sendmsg+0xffa/0x1d5f net/rxrpc/sendmsg.c:652
  rxrpc_sendmsg+0x4d6/0x5f0 net/rxrpc/af_rxrpc.c:586
  sock_sendmsg_nosec net/socket.c:652 [inline]
  sock_sendmsg+0xd7/0x130 net/socket.c:672
  ____sys_sendmsg+0x358/0x880 net/socket.c:2343
  ___sys_sendmsg+0x100/0x170 net/socket.c:2397
  __sys_sendmmsg+0x1bf/0x4d0 net/socket.c:2487
  __do_sys_sendmmsg net/socket.c:2516 [inline]
  __se_sys_sendmmsg net/socket.c:2513 [inline]
  __x64_sys_sendmmsg+0x9d/0x100 net/socket.c:2513
  do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
  entry_SYSCALL_64_after_hwframe+0x49/0xbe
irq event stamp: 182462
hardirqs last  enabled at (182462): [<ffffffff87ec18d6>] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:160 [inline]
hardirqs last  enabled at (182462): [<ffffffff87ec18d6>] _raw_spin_unlock_irqrestore+0x66/0xe0 kernel/locking/spinlock.c:191
hardirqs last disabled at (182461): [<ffffffff87ec1c4f>] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline]
hardirqs last disabled at (182461): [<ffffffff87ec1c4f>] _raw_spin_lock_irqsave+0x6f/0xcd kernel/locking/spinlock.c:159
softirqs last  enabled at (182386): [<ffffffff8147368c>] _local_bh_enable+0x1c/0x30 kernel/softirq.c:162
softirqs last disabled at (182387): [<ffffffff8147608b>] invoke_softirq kernel/softirq.c:373 [inline]
softirqs last disabled at (182387): [<ffffffff8147608b>] irq_exit+0x19b/0x1e0 kernel/softirq.c:413

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(rxrpc_conn_id_lock);
  <Interrupt>
    lock(rxrpc_conn_id_lock);

 *** DEADLOCK ***

1 lock held by swapper/1/0:
 #0: ffffffff89babac0 (rcu_callback){....}, at: rcu_do_batch kernel/rcu/tree.c:2176 [inline]
 #0: ffffffff89babac0 (rcu_callback){....}, at: rcu_core+0x562/0x1390 kernel/rcu/tree.c:2410

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.5.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_usage_bug.cold+0x327/0x378 kernel/locking/lockdep.c:3100
 valid_state kernel/locking/lockdep.c:3111 [inline]
 mark_lock_irq kernel/locking/lockdep.c:3308 [inline]
 mark_lock+0xbb4/0x1220 kernel/locking/lockdep.c:3665
 mark_usage kernel/locking/lockdep.c:3565 [inline]
 __lock_acquire+0x1e8e/0x4a00 kernel/locking/lockdep.c:3908
 lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
 spin_lock include/linux/spinlock.h:338 [inline]
 rxrpc_put_client_connection_id net/rxrpc/conn_client.c:138 [inline]
 rxrpc_put_client_connection_id+0x73/0xe0 net/rxrpc/conn_client.c:135
 rxrpc_put_one_client_conn net/rxrpc/conn_client.c:955 [inline]
 rxrpc_put_client_conn+0x243/0xc90 net/rxrpc/conn_client.c:1001
 rxrpc_put_connection net/rxrpc/ar-internal.h:965 [inline]
 rxrpc_rcu_destroy_call+0xbd/0x200 net/rxrpc/call_object.c:572
 rcu_do_batch kernel/rcu/tree.c:2186 [inline]
 rcu_core+0x5e1/0x1390 kernel/rcu/tree.c:2410
 rcu_core_si+0x9/0x10 kernel/rcu/tree.c:2419
 __do_softirq+0x262/0x98c kernel/softirq.c:292
 invoke_softirq kernel/softirq.c:373 [inline]
 irq_exit+0x19b/0x1e0 kernel/softirq.c:413
 exiting_irq arch/x86/include/asm/apic.h:536 [inline]
 smp_apic_timer_interrupt+0x1a3/0x610 arch/x86/kernel/apic/apic.c:1137
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
 </IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: 38 ea c7 f9 eb 8a cc cc cc cc cc cc e9 07 00 00 00 0f 00 2d 94 76 5c 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d 84 76 5c 00 fb f4 <c3> cc 55 48 89 e5 41 57 41 56 41 55 41 54 53 e8 ce b7 76 f9 e8 09
RSP: 0018:ffffc90000d3fd68 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff1367542 RBX: ffff8880a99fc340 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000006 RDI: ffff8880a99fcbd4
RBP: ffffc90000d3fd98 R08: ffff8880a99fc340 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
R13: ffffffff8aa509c0 R14: 0000000000000000 R15: 0000000000000001

Crashes (473):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/02/05 21:19 upstream 6992ca0dd017 662cf49a .config console log report syz C ci-upstream-kasan-gce-root
2020/02/09 08:11 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/09 05:21 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce-root
2020/02/09 01:33 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 18:07 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 14:00 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 13:54 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 11:47 upstream 41dcd67e8868 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 02:56 upstream 41dcd67e8868 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/08 01:24 upstream 41dcd67e8868 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/07 23:44 upstream 41dcd67e8868 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/07 23:15 upstream 41dcd67e8868 06150bf1 .config console log report ci-upstream-kasan-gce-selinux-root
2020/02/07 16:10 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/07 14:32 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/07 13:27 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce-selinux-root
2020/02/07 04:39 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce
2020/02/09 02:38 upstream d4f309ca4118 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/08 21:54 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/08 20:40 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/08 15:14 upstream f757165705e9 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/07 19:06 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/07 07:59 upstream 90568ecf5615 06150bf1 .config console log report ci-upstream-kasan-gce-386
2020/02/04 09:37 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/04 03:29 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/03 21:25 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/03 21:19 net-old 3d80c653f996 93e5e335 .config console log report ci-upstream-net-this-kasan-gce
2020/02/09 09:21 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 06:25 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 03:14 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/09 00:32 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 23:14 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 22:05 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 19:11 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 16:24 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 12:50 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 10:32 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 09:32 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 07:41 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 06:09 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 04:35 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/08 00:47 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 22:05 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 20:47 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 17:21 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 17:16 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 11:46 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 09:57 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 08:29 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/07 06:35 net-next-old 33b40134e5cf 06150bf1 .config console log report ci-upstream-net-kasan-gce
2020/02/10 00:31 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 22:04 linux-next 6dff1565d69c 35f5e45e .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 11:35 linux-next 6dff1565d69c 6ece2ea5 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/09 07:58 linux-next 6dff1565d69c 06150bf1 .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.