syzbot


WARNING: bad unlock balance in ucma_event_handler

Status: fixed on 2020/05/10 10:42
Reported-by: syzbot+e5579222b6a3edd96522@syzkaller.appspotmail.com
Fix commit: 7c11910783a1 RDMA/ucma: Put a lock around every call to the rdma_cm layer
First crash: 2168d, last: 1478d
Cause bisection: the issue happens on the oldest tested release (bisect log)
Crash: possible deadlock in ucma_migrate_id (log)
Repro: C syz .config
  
Discussions (12)
Title Replies (including bot) Last reply
[PATCH 4.19 00/54] 4.19.115-rc1 review 70 (70) 2020/05/27 20:33
[PATCH AUTOSEL 5.4 001/108] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 111 (111) 2020/04/17 17:18
[PATCH AUTOSEL 5.6 001/149] net: hns3: drop the WQ_MEM_RECLAIM flag when allocating WQ 152 (152) 2020/04/17 17:06
[PATCH 5.6 00/38] 5.6.4-rc1 review 44 (44) 2020/04/14 10:36
[PATCH 5.5 00/44] 5.5.17-rc1 review 48 (48) 2020/04/14 10:36
[PATCH 5.4 00/41] 5.4.32-rc1 review 45 (45) 2020/04/14 10:36
[PATCH AUTOSEL 5.5 001/121] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 122 (122) 2020/04/12 01:16
[PATCH AUTOSEL 4.19 01/66] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 67 (67) 2020/04/12 01:16
Reminder: 11 open syzbot bugs in RDMA subsystem 1 (1) 2019/07/24 01:48
Reminder: 11 open syzbot bugs in RDMA subsystem 1 (1) 2019/06/25 05:48
WARNING: bad unlock balance in ucma_event_handler 6 (7) 2019/06/11 17:57
[PATCH] infiniband: fix a subtle race condition 13 (13) 2018/06/18 18:40
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 WARNING: bad unlock balance in ucma_event_handler C inconclusive 3 499d 1575d 0/1 upstream: reported C repro on 2020/01/02 01:32
linux-4.19 WARNING: bad unlock balance in ucma_event_handler C error 15 508d 1661d 0/1 upstream: reported C repro on 2019/10/08 08:22
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2019/12/11 11:37 20m bisect fix upstream job log (0) log

Sample crash report:
=====================================
WARNING: bad unlock balance detected!
5.5.0-rc4-syzkaller #0 Not tainted
-------------------------------------
kworker/u4:8/469 is trying to release lock (&file->mut) at:
[<ffffffff85a7a1c1>] ucma_event_handler+0x711/0xef0 drivers/infiniband/core/ucma.c:389
but there are no more locks to release!

other info that might help us debug this:
4 locks held by kworker/u4:8/469:
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: __write_once_size include/linux/compiler.h:226 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: set_work_data kernel/workqueue.c:615 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:642 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: process_one_work+0x88b/0x1740 kernel/workqueue.c:2235
 #1: ffffc90002157dc0 ((work_completion)(&(&req->work)->work)){+.+.}, at: process_one_work+0x8c1/0x1740 kernel/workqueue.c:2239
 #2: ffff8880a878a390 (&id_priv->handler_mutex){+.+.}, at: addr_handler+0xaf/0x3d0 drivers/infiniband/core/cma.c:3059
 #3: ffff88809dc14060 (&file->mut){+.+.}, at: ucma_event_handler+0xb3/0xef0 drivers/infiniband/core/ucma.c:349

stack backtrace:
CPU: 1 PID: 469 Comm: kworker/u4:8 Not tainted 5.5.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: ib_addr process_one_req
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_unlock_imbalance_bug kernel/locking/lockdep.c:4008 [inline]
 print_unlock_imbalance_bug.cold+0x114/0x123 kernel/locking/lockdep.c:3984
 __lock_release kernel/locking/lockdep.c:4242 [inline]
 lock_release+0x5f2/0x960 kernel/locking/lockdep.c:4503
 __mutex_unlock_slowpath+0x86/0x6a0 kernel/locking/mutex.c:1228
 mutex_unlock+0xd/0x10 kernel/locking/mutex.c:740
 ucma_event_handler+0x711/0xef0 drivers/infiniband/core/ucma.c:389
 addr_handler+0x2e9/0x3d0 drivers/infiniband/core/cma.c:3092
 process_one_req+0x106/0x680 drivers/infiniband/core/addr.c:643
 process_one_work+0x9af/0x1740 kernel/workqueue.c:2264
 worker_thread+0x98/0xe40 kernel/workqueue.c:2410
 kthread+0x361/0x430 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Crashes (143):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/01/02 03:35 upstream 738d2902773e 25a0186e .config console log report syz C ci-upstream-kasan-gce-root
2020/01/02 01:53 upstream 738d2902773e 25a0186e .config console log report syz C ci-upstream-kasan-gce
2020/01/02 01:37 upstream 738d2902773e 25a0186e .config console log report syz C ci-upstream-kasan-gce-selinux-root
2018/09/07 02:23 upstream ca16eb342ebe e30d3b52 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2018/06/18 22:51 upstream 9ffc59d57228 45c54f75 .config console log report syz C ci-upstream-kasan-gce-root
2018/05/20 02:52 upstream 0b449a441dac f48c20b8 .config console log report syz C ci-upstream-kasan-gce-root
2018/05/19 16:44 upstream 73fcb1a370c7 849705db .config console log report syz C ci-upstream-kasan-gce
2018/05/19 14:57 upstream 73fcb1a370c7 849705db .config console log report syz C ci-upstream-kasan-gce
2018/05/19 13:58 upstream 73fcb1a370c7 849705db .config console log report syz C ci-upstream-kasan-gce-root
2020/01/08 07:22 linux-next 264673852033 6738e0b3 .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2018/09/12 22:18 upstream 96eddb810b14 71907daf .config console log report syz ci-upstream-kasan-gce-smack-root
2018/09/07 01:54 upstream ca16eb342ebe e30d3b52 .config console log report syz ci-upstream-kasan-gce-root
2020/01/02 01:39 upstream 738d2902773e 25a0186e .config console log report syz ci-upstream-kasan-gce-386
2018/06/18 22:41 upstream 9ffc59d57228 45c54f75 .config console log report syz ci-upstream-kasan-gce-386
2018/05/19 07:17 upstream 2c71d338bef2 849705db .config console log report syz ci-upstream-kasan-gce-386
2018/09/10 13:36 linux-next f2b6e66e9885 6b5120a4 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2020/04/07 21:23 upstream 763dede1b248 db9bcd4b .config console log report ci-upstream-kasan-gce-selinux-root
2020/01/26 23:31 upstream a45ea48e2bcd dd56146d .config console log report ci-upstream-kasan-gce
2020/01/26 11:09 upstream 2821e26f3a0a f4e7270e .config console log report ci-upstream-kasan-gce-root
2020/01/10 14:59 upstream bef1d88263ff 532ec44e .config console log report ci-qemu-upstream
2019/10/09 18:28 upstream e3280b54afed 312c6a5a .config console log report ci-upstream-kasan-gce
2019/09/22 19:26 upstream f7c3bf8fa7e5 d96e88f3 .config console log report ci-upstream-kasan-gce
2019/06/10 17:04 upstream d1fdb6d8f6a4 0159583c .config console log report ci-upstream-kasan-gce
2019/04/02 12:29 upstream 5e7a8ca31926 dfd3394d .config console log report ci-upstream-kasan-gce
2019/02/12 19:36 upstream aa0c38cf39de 6ecc6d0f .config console log report ci-upstream-kasan-gce-root
2019/01/27 06:22 upstream ba6069759381 c73f090a .config console log report ci-upstream-kasan-gce
2019/01/26 15:17 upstream ba6069759381 c73f090a .config console log report ci-upstream-kasan-gce
2019/01/25 04:58 upstream c04e2a780caf bfab9cd8 .config console log report ci-upstream-kasan-gce-root
2019/01/10 15:03 upstream ba422731316d db9b6579 .config console log report ci-upstream-kasan-gce
2019/01/01 12:56 upstream e1ef035d272e 3d85f48c .config console log report ci-upstream-kasan-gce
2018/12/16 15:51 upstream 6531e115b7ab def91db3 .config console log report ci-upstream-kasan-gce
2018/12/13 08:58 upstream f5d582777bcb 02613a41 .config console log report ci-upstream-kasan-gce-root
2018/12/05 20:19 upstream d08970904582 ac6c0578 .config console log report ci-upstream-kasan-gce
2018/12/05 14:11 upstream 0072a0c14d5b ac6c0578 .config console log report ci-upstream-kasan-gce
2018/12/05 07:54 upstream 0072a0c14d5b f162ad97 .config console log report ci-upstream-kasan-gce
2018/11/27 22:46 upstream ef78e5ec9214 4b6d14f2 .config console log report ci-upstream-kasan-gce-root
2018/11/17 21:13 upstream 1ce80e0fe98e adf636a8 .config console log report ci-upstream-kasan-gce
2018/10/27 08:13 upstream 345671ea0f92 a8292de9 .config console log report ci-upstream-kasan-gce
2018/10/25 15:30 upstream bd6bf7c10484 a8292de9 .config console log report ci-upstream-kasan-gce-smack-root
2018/10/25 08:17 upstream 01aa9d518eae a8292de9 .config console log report ci-upstream-kasan-gce
2018/10/17 19:14 upstream c0cff31be705 1ba7fd7e .config console log report ci-upstream-kasan-gce
2018/10/14 00:04 upstream 7ec21823634d caf12900 .config console log report ci-upstream-kasan-gce
2018/10/12 16:33 upstream 6b3944e42e2e caf12900 .config console log report ci-upstream-kasan-gce
2018/10/03 02:26 upstream 1d2ba7fee28b 0f3e0261 .config console log report ci-upstream-kasan-gce-selinux-root
2018/10/02 02:33 upstream 385afbf8c3e8 e06f7713 .config console log report ci-upstream-kasan-gce-smack-root
2018/10/01 19:31 upstream 17b57b1883c1 48a50c6b .config console log report ci-upstream-kasan-gce
2018/09/24 04:30 upstream 6bf4ca7fbc85 28d9ac76 .config console log report ci-upstream-kasan-gce-root
2018/09/23 20:07 upstream 6bf4ca7fbc85 28d9ac76 .config console log report ci-upstream-kasan-gce-selinux-root
2020/04/02 13:44 upstream 919dce24701f a34e2c33 .config console log report ci-upstream-kasan-gce-386
2020/01/01 14:59 upstream 738d2902773e 25a0186e .config console log report ci-upstream-kasan-gce-386
2019/06/08 14:18 upstream 79c3ba3206c7 0159583c .config console log report ci-upstream-kasan-gce-386
2019/04/05 08:03 upstream 8e22ba96d44c e5d1b3ac .config console log report ci-upstream-kasan-gce-386
2019/03/03 05:13 upstream c93d9218ea56 1c0e457a .config console log report ci-upstream-kasan-gce-386
2018/11/29 08:22 upstream 60b548237fed 4b6d14f2 .config console log report ci-upstream-kasan-gce-386
2018/11/14 06:15 upstream ccda4af0f4b9 5f5f6d14 .config console log report ci-upstream-kasan-gce-386
2018/10/19 04:42 upstream fa520c47eaa1 9aba67b5 .config console log report ci-upstream-kasan-gce-386
2020/02/26 02:44 linux-next bdc5461b23ca 59b57593 .config console log report ci-upstream-linux-next-kasan-gce-root
2020/02/05 17:08 linux-next 2747d5fdab78 662cf49a .config console log report ci-upstream-linux-next-kasan-gce-root
2019/03/24 20:04 linux-next e382d91f5f80 acbc5b7d .config console log report ci-upstream-linux-next-kasan-gce-root
2019/01/21 11:19 linux-next a790195efddd badbbeee .config console log report ci-upstream-linux-next-kasan-gce-root
2019/01/13 05:09 linux-next b808822a75a3 c3f3344c .config console log report ci-upstream-linux-next-kasan-gce-root
2018/11/13 20:28 linux-next 442b8cea2477 5f5f6d14 .config console log report ci-upstream-linux-next-kasan-gce-root
2018/10/16 09:23 linux-next 6d5d82417dd6 8cd30605 .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.