syzbot


WARNING: bad unlock balance in ucma_event_handler

Status: fixed on 2020/05/10 10:42
Reported-by: syzbot+e5579222b6a3edd96522@syzkaller.appspotmail.com
Fix commit: 7c11910783a1 RDMA/ucma: Put a lock around every call to the rdma_cm layer
First crash: 1776d, last: 1086d

Cause bisection: the issue happens on the oldest tested release (bisect log)
Crash: possible deadlock in ucma_migrate_id (log)
Repro: C syz .config
similar bugs (2):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 WARNING: bad unlock balance in ucma_event_handler C inconclusive 3 107d 1183d 0/1 upstream: reported C repro on 2020/01/02 01:32
linux-4.19 WARNING: bad unlock balance in ucma_event_handler C error 15 116d 1269d 0/1 upstream: reported C repro on 2019/10/08 08:22

Sample crash report:
=====================================
WARNING: bad unlock balance detected!
5.5.0-rc4-syzkaller #0 Not tainted
-------------------------------------
kworker/u4:8/469 is trying to release lock (&file->mut) at:
[<ffffffff85a7a1c1>] ucma_event_handler+0x711/0xef0 drivers/infiniband/core/ucma.c:389
but there are no more locks to release!

other info that might help us debug this:
4 locks held by kworker/u4:8/469:
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: __write_once_size include/linux/compiler.h:226 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: set_work_data kernel/workqueue.c:615 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:642 [inline]
 #0: ffff88821aa98928 ((wq_completion)ib_addr){+.+.}, at: process_one_work+0x88b/0x1740 kernel/workqueue.c:2235
 #1: ffffc90002157dc0 ((work_completion)(&(&req->work)->work)){+.+.}, at: process_one_work+0x8c1/0x1740 kernel/workqueue.c:2239
 #2: ffff8880a878a390 (&id_priv->handler_mutex){+.+.}, at: addr_handler+0xaf/0x3d0 drivers/infiniband/core/cma.c:3059
 #3: ffff88809dc14060 (&file->mut){+.+.}, at: ucma_event_handler+0xb3/0xef0 drivers/infiniband/core/ucma.c:349

stack backtrace:
CPU: 1 PID: 469 Comm: kworker/u4:8 Not tainted 5.5.0-rc4-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: ib_addr process_one_req
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_unlock_imbalance_bug kernel/locking/lockdep.c:4008 [inline]
 print_unlock_imbalance_bug.cold+0x114/0x123 kernel/locking/lockdep.c:3984
 __lock_release kernel/locking/lockdep.c:4242 [inline]
 lock_release+0x5f2/0x960 kernel/locking/lockdep.c:4503
 __mutex_unlock_slowpath+0x86/0x6a0 kernel/locking/mutex.c:1228
 mutex_unlock+0xd/0x10 kernel/locking/mutex.c:740
 ucma_event_handler+0x711/0xef0 drivers/infiniband/core/ucma.c:389
 addr_handler+0x2e9/0x3d0 drivers/infiniband/core/cma.c:3092
 process_one_req+0x106/0x680 drivers/infiniband/core/addr.c:643
 process_one_work+0x9af/0x1740 kernel/workqueue.c:2264
 worker_thread+0x98/0xe40 kernel/workqueue.c:2410
 kthread+0x361/0x430 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Fix bisection attempts:
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Title
ci-upstream-kasan-gce-selinux-root 2019/12/11 11:58 upstream 6794862a16ef e30d3b52 .config console log report syz C
* Struck through repros no longer work on HEAD.
Crashes (143):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Title
ci-upstream-kasan-gce-root 2020/01/02 03:35 upstream 738d2902773e 25a0186e .config console log report syz C
ci-upstream-kasan-gce 2020/01/02 01:53 upstream 738d2902773e 25a0186e .config console log report syz C
ci-upstream-kasan-gce-selinux-root 2020/01/02 01:37 upstream 738d2902773e 25a0186e .config console log report syz C
ci-upstream-kasan-gce-selinux-root 2018/09/07 02:23 upstream ca16eb342ebe e30d3b52 .config console log report syz C
ci-upstream-kasan-gce-root 2018/06/18 22:51 upstream 9ffc59d57228 45c54f75 .config console log report syz C
ci-upstream-kasan-gce-root 2018/05/20 02:52 upstream 0b449a441dac f48c20b8 .config console log report syz C
ci-upstream-kasan-gce 2018/05/19 16:44 upstream 73fcb1a370c7 849705db .config console log report syz C
ci-upstream-kasan-gce 2018/05/19 14:57 upstream 73fcb1a370c7 849705db .config console log report syz C
ci-upstream-kasan-gce-root 2018/05/19 13:58 upstream 73fcb1a370c7 849705db .config console log report syz C
ci-upstream-linux-next-kasan-gce-root 2020/01/08 07:22 linux-next 264673852033 6738e0b3 .config console log report syz C
ci-upstream-kasan-gce-smack-root 2018/09/12 22:18 upstream 96eddb810b14 71907daf .config console log report syz
ci-upstream-kasan-gce-root 2018/09/07 01:54 upstream ca16eb342ebe e30d3b52 .config console log report syz
ci-upstream-kasan-gce-386 2020/01/02 01:39 upstream 738d2902773e 25a0186e .config console log report syz
ci-upstream-kasan-gce-386 2018/06/18 22:41 upstream 9ffc59d57228 45c54f75 .config console log report syz
ci-upstream-kasan-gce-386 2018/05/19 07:17 upstream 2c71d338bef2 849705db .config console log report syz
ci-upstream-linux-next-kasan-gce-root 2018/09/10 13:36 linux-next f2b6e66e9885 6b5120a4 .config console log report syz
ci-upstream-kasan-gce-selinux-root 2020/04/07 21:23 upstream 763dede1b248 db9bcd4b .config console log report
ci-upstream-kasan-gce 2020/01/26 23:31 upstream a45ea48e2bcd dd56146d .config console log report
ci-upstream-kasan-gce-root 2020/01/26 11:09 upstream 2821e26f3a0a f4e7270e .config console log report
ci-qemu-upstream 2020/01/10 14:59 upstream bef1d88263ff 532ec44e .config console log report
ci-upstream-kasan-gce 2019/10/09 18:28 upstream e3280b54afed 312c6a5a .config console log report
ci-upstream-kasan-gce 2019/09/22 19:26 upstream f7c3bf8fa7e5 d96e88f3 .config console log report
ci-upstream-kasan-gce 2019/06/10 17:04 upstream d1fdb6d8f6a4 0159583c .config console log report
ci-upstream-kasan-gce 2019/04/02 12:29 upstream 5e7a8ca31926 dfd3394d .config console log report
ci-upstream-kasan-gce-root 2019/02/12 19:36 upstream aa0c38cf39de 6ecc6d0f .config console log report
ci-upstream-kasan-gce 2019/01/27 06:22 upstream ba6069759381 c73f090a .config console log report
ci-upstream-kasan-gce 2019/01/26 15:17 upstream ba6069759381 c73f090a .config console log report
ci-upstream-kasan-gce-root 2019/01/25 04:58 upstream c04e2a780caf bfab9cd8 .config console log report
ci-upstream-kasan-gce 2019/01/10 15:03 upstream ba422731316d db9b6579 .config console log report
ci-upstream-kasan-gce 2019/01/01 12:56 upstream e1ef035d272e 3d85f48c .config console log report
ci-upstream-kasan-gce 2018/12/16 15:51 upstream 6531e115b7ab def91db3 .config console log report
ci-upstream-kasan-gce-root 2018/12/13 08:58 upstream f5d582777bcb 02613a41 .config console log report
ci-upstream-kasan-gce 2018/12/05 20:19 upstream d08970904582 ac6c0578 .config console log report
ci-upstream-kasan-gce 2018/12/05 14:11 upstream 0072a0c14d5b ac6c0578 .config console log report
ci-upstream-kasan-gce 2018/12/05 07:54 upstream 0072a0c14d5b f162ad97 .config console log report
ci-upstream-kasan-gce-root 2018/11/27 22:46 upstream ef78e5ec9214 4b6d14f2 .config console log report
ci-upstream-kasan-gce 2018/11/17 21:13 upstream 1ce80e0fe98e adf636a8 .config console log report
ci-upstream-kasan-gce 2018/10/27 08:13 upstream 345671ea0f92 a8292de9 .config console log report
ci-upstream-kasan-gce-smack-root 2018/10/25 15:30 upstream bd6bf7c10484 a8292de9 .config console log report
ci-upstream-kasan-gce 2018/10/25 08:17 upstream 01aa9d518eae a8292de9 .config console log report
ci-upstream-kasan-gce 2018/10/17 19:14 upstream c0cff31be705 1ba7fd7e .config console log report
ci-upstream-kasan-gce 2018/10/14 00:04 upstream 7ec21823634d caf12900 .config console log report
ci-upstream-kasan-gce 2018/10/12 16:33 upstream 6b3944e42e2e caf12900 .config console log report
ci-upstream-kasan-gce-selinux-root 2018/10/03 02:26 upstream 1d2ba7fee28b 0f3e0261 .config console log report
ci-upstream-kasan-gce-smack-root 2018/10/02 02:33 upstream 385afbf8c3e8 e06f7713 .config console log report
ci-upstream-kasan-gce 2018/10/01 19:31 upstream 17b57b1883c1 48a50c6b .config console log report
ci-upstream-kasan-gce-root 2018/09/24 04:30 upstream 6bf4ca7fbc85 28d9ac76 .config console log report
ci-upstream-kasan-gce-selinux-root 2018/09/23 20:07 upstream 6bf4ca7fbc85 28d9ac76 .config console log report
ci-upstream-kasan-gce-386 2020/04/02 13:44 upstream 919dce24701f a34e2c33 .config console log report
ci-upstream-kasan-gce-386 2020/01/01 14:59 upstream 738d2902773e 25a0186e .config console log report
ci-upstream-kasan-gce-386 2019/06/08 14:18 upstream 79c3ba3206c7 0159583c .config console log report
ci-upstream-kasan-gce-386 2019/04/05 08:03 upstream 8e22ba96d44c e5d1b3ac .config console log report
ci-upstream-kasan-gce-386 2019/03/03 05:13 upstream c93d9218ea56 1c0e457a .config console log report
ci-upstream-kasan-gce-386 2018/11/29 08:22 upstream 60b548237fed 4b6d14f2 .config console log report
ci-upstream-kasan-gce-386 2018/11/14 06:15 upstream ccda4af0f4b9 5f5f6d14 .config console log report
ci-upstream-kasan-gce-386 2018/10/19 04:42 upstream fa520c47eaa1 9aba67b5 .config console log report
ci-upstream-linux-next-kasan-gce-root 2020/02/26 02:44 linux-next bdc5461b23ca 59b57593 .config console log report
ci-upstream-linux-next-kasan-gce-root 2020/02/05 17:08 linux-next 2747d5fdab78 662cf49a .config console log report
ci-upstream-linux-next-kasan-gce-root 2019/03/24 20:04 linux-next e382d91f5f80 acbc5b7d .config console log report
ci-upstream-linux-next-kasan-gce-root 2019/01/21 11:19 linux-next a790195efddd badbbeee .config console log report
ci-upstream-linux-next-kasan-gce-root 2019/01/13 05:09 linux-next b808822a75a3 c3f3344c .config console log report
ci-upstream-linux-next-kasan-gce-root 2018/11/13 20:28 linux-next 442b8cea2477 5f5f6d14 .config console log report
ci-upstream-linux-next-kasan-gce-root 2018/10/16 09:23 linux-next 6d5d82417dd6 8cd30605 .config console log report
* Struck through repros no longer work on HEAD.