syzbot


possible deadlock in _destroy_id

Status: fixed on 2021/03/10 01:48
Subsystems: rdma
[Documentation on labels]
Reported-by: syzbot+1bc48bf7f78253f664a9@syzkaller.appspotmail.com
Fix commit: dd37d2f59eb8 RDMA/cma: Fix deadlock on &lock in rdma_cma_listen_on_all() error unwind
First crash: 1248d, last: 1240d
Cause bisection: introduced by (bisect log) [merge commit]:
commit ae0d0bb29b311ee3c71c8796f76d309a3779c8d9
Author: Jakub Kicinski <kuba@kernel.org>
Date: Sat Nov 7 01:32:55 2020 +0000

  Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

Crash: BUG: sleeping function called from invalid context in sta_info_move_state (log)
Repro: C syz .config
  
Duplicate bugs (1)
duplicates (1):
Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
possible deadlock in rdma_destroy_id rdma 2 1245d 1245d 0/26 closed as dup on 2020/11/20 18:32
Discussions (2)
Title Replies (including bot) Last reply
[PATCH 5.10 000/717] 5.10.4-rc1 review 747 (747) 2021/01/05 16:41
possible deadlock in _destroy_id 3 (5) 2020/11/26 00:24

Sample crash report:
virt_wifi0 speed is unknown, defaulting to 1000
virt_wifi0 speed is unknown, defaulting to 1000
iwpm_register_pid: Unable to send a nlmsg (client = 2)
infiniband syz2: RDMA CMA: cma_listen_on_dev, error -98
============================================
WARNING: possible recursive locking detected
5.10.0-rc4-next-20201118-syzkaller #0 Not tainted
--------------------------------------------
syz-executor563/8458 is trying to acquire lock:
ffffffff8c684748 (lock#6){+.+.}-{3:3}, at: cma_release_dev drivers/infiniband/core/cma.c:476 [inline]
ffffffff8c684748 (lock#6){+.+.}-{3:3}, at: _destroy_id+0x299/0xa00 drivers/infiniband/core/cma.c:1852

but task is already holding lock:
ffffffff8c684748 (lock#6){+.+.}-{3:3}, at: cma_add_one+0x55c/0xce0 drivers/infiniband/core/cma.c:4902

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(lock#6);
  lock(lock#6);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

6 locks held by syz-executor563/8458:
 #0: ffffffff8fa76958 (&rdma_nl_types[idx].sem){.+.+}-{3:3}, at: rdma_nl_rcv_msg+0x15b/0x690 drivers/infiniband/core/netlink.c:164
 #1: ffffffff8c66c490 (link_ops_rwsem){++++}-{3:3}, at: nldev_newlink+0x261/0x540 drivers/infiniband/core/nldev.c:1545
 #2: ffffffff8c65bd90 (devices_rwsem){++++}-{3:3}, at: enable_device_and_get+0xfc/0x3c0 drivers/infiniband/core/device.c:1321
 #3: ffffffff8c65bc50 (clients_rwsem){++++}-{3:3}, at: enable_device_and_get+0x163/0x3c0 drivers/infiniband/core/device.c:1331
 #4: ffff888023cbc598 (&device->client_data_rwsem){++++}-{3:3}, at: add_client_context+0x3d0/0x5e0 drivers/infiniband/core/device.c:710
 #5: ffffffff8c684748 (lock#6){+.+.}-{3:3}, at: cma_add_one+0x55c/0xce0 drivers/infiniband/core/cma.c:4902

stack backtrace:
CPU: 0 PID: 8458 Comm: syz-executor563 Not tainted 5.10.0-rc4-next-20201118-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0x107/0x163 lib/dump_stack.c:120
 print_deadlock_bug kernel/locking/lockdep.c:2759 [inline]
 check_deadlock kernel/locking/lockdep.c:2802 [inline]
 validate_chain kernel/locking/lockdep.c:3593 [inline]
 __lock_acquire.cold+0x115/0x39f kernel/locking/lockdep.c:4830
 lock_acquire kernel/locking/lockdep.c:5435 [inline]
 lock_acquire+0x2a3/0x8c0 kernel/locking/lockdep.c:5400
 __mutex_lock_common kernel/locking/mutex.c:956 [inline]
 __mutex_lock+0x134/0x1110 kernel/locking/mutex.c:1103
 cma_release_dev drivers/infiniband/core/cma.c:476 [inline]
 _destroy_id+0x299/0xa00 drivers/infiniband/core/cma.c:1852
 cma_listen_on_dev.cold+0x168/0x16d drivers/infiniband/core/cma.c:2535
 cma_add_one+0x667/0xce0 drivers/infiniband/core/cma.c:4905
 add_client_context+0x405/0x5e0 drivers/infiniband/core/device.c:712
 enable_device_and_get+0x1d5/0x3c0 drivers/infiniband/core/device.c:1333
 ib_register_device drivers/infiniband/core/device.c:1408 [inline]
 ib_register_device+0x7a0/0xa30 drivers/infiniband/core/device.c:1367
 siw_device_register drivers/infiniband/sw/siw/siw_main.c:72 [inline]
 siw_newlink drivers/infiniband/sw/siw/siw_main.c:545 [inline]
 siw_newlink+0xddb/0x1340 drivers/infiniband/sw/siw/siw_main.c:522
 nldev_newlink+0x30e/0x540 drivers/infiniband/core/nldev.c:1555
 rdma_nl_rcv_msg+0x367/0x690 drivers/infiniband/core/netlink.c:195
 rdma_nl_rcv_skb drivers/infiniband/core/netlink.c:239 [inline]
 rdma_nl_rcv+0x2f2/0x440 drivers/infiniband/core/netlink.c:259
 netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
 netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
 sock_sendmsg_nosec net/socket.c:650 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:670
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2339
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2393
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2426
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x443689
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 eb 0d fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffe37189dd8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000443689
RDX: 0000000000000000 RSI: 0000000020000200 RDI: 0000000000000003
RBP: 00007ffe37189de0 R08: 0000000001bbbbbb R09: 0000000001bbbbbb
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffe37189df0
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000

Crashes (28):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/11/19 02:00 linux-next 205292332779 0767f13f .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2020/11/18 14:25 linux-next 205292332779 09323409 .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2020/11/26 11:35 linux-next 6147c83fd749 2f1cec62 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/24 06:03 linux-next 62918e6fd7b5 1ab681a4 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/23 11:22 linux-next 62918e6fd7b5 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/23 10:25 linux-next 62918e6fd7b5 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/23 09:46 linux-next 62918e6fd7b5 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/22 10:14 linux-next 95065cb54210 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/22 08:47 linux-next 95065cb54210 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/22 08:47 linux-next 95065cb54210 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/22 08:47 linux-next 95065cb54210 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/22 08:47 linux-next 95065cb54210 0d27f508 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/21 13:55 linux-next 95065cb54210 68068804 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/21 09:50 linux-next 95065cb54210 68068804 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/21 09:47 linux-next 95065cb54210 68068804 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/21 09:47 linux-next 95065cb54210 68068804 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/21 09:47 linux-next 95065cb54210 68068804 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/20 06:49 linux-next f8550c0d35df 0767f13f .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/20 00:44 linux-next f8550c0d35df 0767f13f .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 22:22 linux-next 205292332779 0767f13f .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 21:24 linux-next 205292332779 0767f13f .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 21:17 linux-next 205292332779 0767f13f .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 10:42 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 10:39 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 10:36 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 10:33 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 09:19 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
2020/11/18 09:10 linux-next 205292332779 09323409 .config console log report info ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.