WARNING: bad unlock balance in ucma_destroy_id

Status: fixed on 2020/05/10 10:42
Reported-by: syzbot+4b628fcc748474003457@syzkaller.appspotmail.com
Fix commit: 7c11910783a1 RDMA/ucma: Put a lock around every call to the rdma_cm layer
First crash: 2889d, last: 2335d
Cause bisection: introduced by (bisect log) :
commit e1c30298ccab87151a0c4241fc5985c591598361
Author: Yishai Hadas <yishaih@mellanox.com>
Date: Thu Aug 13 15:32:07 2015 +0000

  IB/ucma: HW Device hot-removal support

Crash: BUG: bad unlock balance in mutex_unlock (log)
Repro: syz .config
  
▶ ▼ Discussions (11)
Title Replies (including bot) Last reply
[PATCH 4.19 00/54] 4.19.115-rc1 review 70 (70) 2020/05/27 20:33
[PATCH AUTOSEL 5.4 001/108] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 111 (111) 2020/04/17 17:18
[PATCH AUTOSEL 5.6 001/149] net: hns3: drop the WQ_MEM_RECLAIM flag when allocating WQ 152 (152) 2020/04/17 17:06
[PATCH 5.6 00/38] 5.6.4-rc1 review 44 (44) 2020/04/14 10:36
[PATCH 5.5 00/44] 5.5.17-rc1 review 48 (48) 2020/04/14 10:36
[PATCH 5.4 00/41] 5.4.32-rc1 review 45 (45) 2020/04/14 10:36
[PATCH AUTOSEL 5.5 001/121] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 122 (122) 2020/04/12 01:16
[PATCH AUTOSEL 4.19 01/66] net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware 67 (67) 2020/04/12 01:16
Reminder: 11 open syzbot bugs in RDMA subsystem 1 (1) 2019/07/24 01:48
Reminder: 11 open syzbot bugs in RDMA subsystem 1 (1) 2019/06/25 05:48
WARNING: bad unlock balance in ucma_destroy_id 0 (2) 2019/04/11 06:53
▶ ▼ Similar bugs (2)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 WARNING: bad unlock balance in ucma_destroy_id 4 C error 7 1308d 2586d 0/1 upstream: reported C repro on 2019/06/27 11:05
linux-4.14 WARNING: bad unlock balance in ucma_destroy_id 4 C inconclusive 12 1307d 2581d 0/1 upstream: reported C repro on 2019/07/01 17:24
▶ ▼ Fix bisection attempts (4)
Created Duration User Patch Repo Result
2020/02/12 07:58 22m bisect fix upstream OK (0) job log log
2020/01/13 07:38 20m bisect fix upstream OK (0) job log log
2019/11/09 19:13 16m bisect fix upstream OK (0) job log log
2019/08/22 22:24 17m bisect fix upstream OK (0) job log log

Sample crash report:
=====================================
WARNING: bad unlock balance detected!
5.6.0-rc2-syzkaller #0 Not tainted
-------------------------------------
syz-executor.0/9856 is trying to release lock (&file->mut) at:
[<ffffffff85c2570a>] ucma_destroy_id+0x24a/0x490 drivers/infiniband/core/ucma.c:621
but there are no more locks to release!

other info that might help us debug this:
1 lock held by syz-executor.0/9856:
 #0: ffff8880a11dd460 (&file->mut){+.+.}, at: ucma_destroy_id+0x1e7/0x490 drivers/infiniband/core/ucma.c:619

stack backtrace:
CPU: 1 PID: 9856 Comm: syz-executor.0 Not tainted 5.6.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_unlock_imbalance_bug kernel/locking/lockdep.c:4007 [inline]
 print_unlock_imbalance_bug.cold+0x114/0x123 kernel/locking/lockdep.c:3983
 __lock_release kernel/locking/lockdep.c:4241 [inline]
 lock_release+0x5f2/0x960 kernel/locking/lockdep.c:4502
 __mutex_unlock_slowpath+0x86/0x6a0 kernel/locking/mutex.c:1228
 mutex_unlock+0xd/0x10 kernel/locking/mutex.c:740
 ucma_destroy_id+0x24a/0x490 drivers/infiniband/core/ucma.c:621
 ucma_write+0x2d7/0x3c0 drivers/infiniband/core/ucma.c:1684
 __vfs_write+0x8a/0x110 fs/read_write.c:494
 vfs_write+0x268/0x5d0 fs/read_write.c:558
 ksys_write+0x220/0x290 fs/read_write.c:611
 __do_sys_write fs/read_write.c:623 [inline]
 __se_sys_write fs/read_write.c:620 [inline]
 __x64_sys_write+0x73/0xb0 fs/read_write.c:620
 do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c449
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f5c79d18c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007f5c79d196d4 RCX: 000000000045c449
RDX: 0000000000000018 RSI: 0000000020001380 RDI: 0000000000000003
RBP: 000000000076bfc0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 0000000000000c8a R14: 00000000004d7660 R15: 000000000076bfcc
==================================================================
BUG: KASAN: use-after-free in atomic64_read include/asm-generic/atomic-instrumented.h:836 [inline]
BUG: KASAN: use-after-free in atomic_long_read include/asm-generic/atomic-long.h:28 [inline]
BUG: KASAN: use-after-free in __mutex_unlock_slowpath+0x93/0x6a0 kernel/locking/mutex.c:1237
Read of size 8 at addr ffff8880a11dd000 by task syz-executor.0/9856

CPU: 1 PID: 9856 Comm: syz-executor.0 Not tainted 5.6.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x197/0x210 lib/dump_stack.c:118
 print_address_description.constprop.0.cold+0xd4/0x30b mm/kasan/report.c:374
 __kasan_report.cold+0x1b/0x32 mm/kasan/report.c:506
 kasan_report+0x12/0x20 mm/kasan/common.c:641
 check_memory_region_inline mm/kasan/generic.c:185 [inline]
 check_memory_region+0x134/0x1a0 mm/kasan/generic.c:192
 __kasan_check_read+0x11/0x20 mm/kasan/common.c:95
 atomic64_read include/asm-generic/atomic-instrumented.h:836 [inline]
 atomic_long_read include/asm-generic/atomic-long.h:28 [inline]
 __mutex_unlock_slowpath+0x93/0x6a0 kernel/locking/mutex.c:1237
 mutex_unlock+0xd/0x10 kernel/locking/mutex.c:740
 ucma_destroy_id+0x24a/0x490 drivers/infiniband/core/ucma.c:621
 ucma_write+0x2d7/0x3c0 drivers/infiniband/core/ucma.c:1684
 __vfs_write+0x8a/0x110 fs/read_write.c:494
 vfs_write+0x268/0x5d0 fs/read_write.c:558
 ksys_write+0x220/0x290 fs/read_write.c:611
 __do_sys_write fs/read_write.c:623 [inline]
 __se_sys_write fs/read_write.c:620 [inline]
 __x64_sys_write+0x73/0xb0 fs/read_write.c:620
 do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45c449
Code: ad b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 7b b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f5c79d18c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007f5c79d196d4 RCX: 000000000045c449
RDX: 0000000000000018 RSI: 0000000020001380 RDI: 0000000000000003
RBP: 000000000076bfc0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 0000000000000c8a R14: 00000000004d7660 R15: 000000000076bfcc

Allocated by task 9856:
 save_stack+0x23/0x90 mm/kasan/common.c:72
 set_track mm/kasan/common.c:80 [inline]
 __kasan_kmalloc mm/kasan/common.c:515 [inline]
 __kasan_kmalloc.constprop.0+0xcf/0xe0 mm/kasan/common.c:488
 kasan_kmalloc+0x9/0x10 mm/kasan/common.c:529
 kmem_cache_alloc_trace+0x158/0x790 mm/slab.c:3551
 kmalloc include/linux/slab.h:555 [inline]
 ucma_open+0x4f/0x290 drivers/infiniband/core/ucma.c:1716
 misc_open+0x395/0x4c0 drivers/char/misc.c:141
 chrdev_open+0x245/0x6b0 fs/char_dev.c:414
 do_dentry_open+0x4e6/0x1380 fs/open.c:797
 vfs_open+0xa0/0xd0 fs/open.c:914
 do_last fs/namei.c:3490 [inline]
 path_openat+0x12ee/0x3490 fs/namei.c:3607
 do_filp_open+0x192/0x260 fs/namei.c:3637
 do_sys_openat2+0x5eb/0x7e0 fs/open.c:1149
 do_sys_open+0xf2/0x180 fs/open.c:1165
 __do_sys_openat fs/open.c:1179 [inline]
 __se_sys_openat fs/open.c:1174 [inline]
 __x64_sys_openat+0x9d/0x100 fs/open.c:1174
 do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

Freed by task 9850:
 save_stack+0x23/0x90 mm/kasan/common.c:72
 set_track mm/kasan/common.c:80 [inline]
 kasan_set_free_info mm/kasan/common.c:337 [inline]
 __kasan_slab_free+0x102/0x150 mm/kasan/common.c:476
 kasan_slab_free+0xe/0x10 mm/kasan/common.c:485
 __cache_free mm/slab.c:3426 [inline]
 kfree+0x10a/0x2c0 mm/slab.c:3757
 ucma_close+0x275/0x310 drivers/infiniband/core/ucma.c:1772
 __fput+0x2ff/0x890 fs/file_table.c:280
 ____fput+0x16/0x20 fs/file_table.c:313
 task_work_run+0x145/0x1c0 kernel/task_work.c:113
 tracehook_notify_resume include/linux/tracehook.h:188 [inline]
 exit_to_usermode_loop+0x316/0x380 arch/x86/entry/common.c:164
 prepare_exit_to_usermode arch/x86/entry/common.c:195 [inline]
 syscall_return_slowpath arch/x86/entry/common.c:278 [inline]
 do_syscall_64+0x676/0x790 arch/x86/entry/common.c:304
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

The buggy address belongs to the object at ffff8880a11dd000
 which belongs to the cache kmalloc-256 of size 256
The buggy address is located 0 bytes inside of
 256-byte region [ffff8880a11dd000, ffff8880a11dd100)
The buggy address belongs to the page:
page:ffffea0002847740 refcount:1 mapcount:0 mapping:ffff8880aa4008c0 index:0x0
flags: 0xfffe0000000200(slab)
raw: 00fffe0000000200 ffffea0002527cc8 ffffea000251dbc8 ffff8880aa4008c0
raw: 0000000000000000 ffff8880a11dd000 0000000100000008 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
 ffff8880a11dcf00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff8880a11dcf80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff8880a11dd000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                   ^
 ffff8880a11dd080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff8880a11dd100: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

Crashes (44):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/02/20 19:04 upstream ca7e1fd1026c 81230308 .config console log report syz ci-upstream-kasan-gce-root
2020/02/18 17:47 upstream b1da3acc781c 012fbc32 .config console log report syz ci-upstream-kasan-gce-smack-root
2019/11/30 20:04 upstream 81b6b96475ac 3a75be00 .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/11/30 15:01 upstream 81b6b96475ac 3a75be00 .config console log report syz ci-upstream-kasan-gce-smack-root
2019/11/30 09:18 upstream 81b6b96475ac 3a75be00 .config console log report syz ci-upstream-kasan-gce-root
2019/11/29 23:19 upstream 81b6b96475ac d29b9e84 .config console log report syz ci-upstream-kasan-gce
2019/02/18 16:29 upstream 2fee036af043 59f36113 .config console log report syz ci-upstream-kasan-gce
2019/02/18 16:26 upstream 2fee036af043 59f36113 .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/02/18 16:26 upstream 2fee036af043 59f36113 .config console log report syz ci-upstream-kasan-gce-root
2019/02/18 16:14 upstream 2fee036af043 59f36113 .config console log report syz ci-upstream-kasan-gce-smack-root
2019/01/04 18:28 upstream 96d4f267e40f 0127e3ba .config console log report syz ci-upstream-kasan-gce-selinux-root
2019/01/04 02:57 upstream 645ff1e8e704 7da23925 .config console log report syz ci-upstream-kasan-gce-root
2019/01/04 02:47 upstream 645ff1e8e704 7da23925 .config console log report syz ci-upstream-kasan-gce
2019/01/04 02:11 upstream 645ff1e8e704 7da23925 .config console log report syz ci-upstream-kasan-gce-smack-root
2018/10/22 04:39 upstream 467e050e9760 ecb386fe .config console log report syz ci-upstream-kasan-gce-smack-root
2018/10/21 14:47 upstream 23469de647c4 ecb386fe .config console log report syz ci-upstream-kasan-gce
2018/10/20 18:05 upstream 270b77a0f30e ecb386fe .config console log report syz ci-upstream-kasan-gce-selinux-root
2018/08/27 19:26 upstream 5b394b2ddf03 758cd203 .config console log report syz ci-upstream-kasan-gce
2018/08/27 19:03 upstream 5b394b2ddf03 758cd203 .config console log report syz ci-upstream-kasan-gce-root
2020/02/18 09:54 upstream 11a48a5a18c6 1ce142dc .config console log report syz ci-upstream-kasan-gce-386
2019/11/30 14:13 upstream 81b6b96475ac 3a75be00 .config console log report syz ci-upstream-kasan-gce-386
2019/02/18 16:16 upstream a3b22b9f11d9 59f36113 .config console log report syz ci-upstream-kasan-gce-386
2019/01/04 07:04 upstream 645ff1e8e704 7da23925 .config console log report syz ci-upstream-kasan-gce-386
2018/10/21 06:15 upstream b0d04fb56b31 ecb386fe .config console log report syz ci-upstream-kasan-gce-386
2018/08/27 19:41 upstream 5b394b2ddf03 758cd203 .config console log report syz ci-upstream-kasan-gce-386
2020/03/04 09:04 linux-next c99b17ac0399 c88c7b75 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/12/14 07:38 linux-next 32b8acf85223 5b2ca5da .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/02/18 16:27 linux-next cb916fc5eabf 59f36113 .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/01/04 18:09 linux-next a4983672f9ca 0127e3ba .config console log report syz ci-upstream-linux-next-kasan-gce-root
2018/10/21 20:08 linux-next 8c60c36d0b8c ecb386fe .config console log report syz ci-upstream-linux-next-kasan-gce-root
2019/06/26 06:52 upstream 249155c20f9b 0a8d1a96 .config console log report ci-upstream-kasan-gce-selinux-root
2019/03/23 17:56 upstream a5ed1e96cafd a2cef203 .config console log report ci-upstream-kasan-gce-smack-root
2019/03/17 01:05 upstream 9c7dc824d9a4 bab43553 .config console log report ci-upstream-kasan-gce-root
2019/03/12 14:40 upstream ea295481b6e3 a71bfb62 .config console log report ci-upstream-kasan-gce
2019/03/07 10:44 upstream f90d64483ebd 8c085c5e .config console log report ci-upstream-kasan-gce
2019/03/04 21:43 upstream 736706bee329 7c693b52 .config console log report ci-upstream-kasan-gce
2019/02/26 10:54 upstream 7d762d69145a a36ecd98 .config console log report ci-upstream-kasan-gce
2019/02/22 20:48 upstream 8a61716ff2ab 6a5fcca4 .config console log report ci-upstream-kasan-gce
2019/02/11 04:41 upstream aadaa8061189 b4f792e4 .config console log report ci-upstream-kasan-gce
2019/02/09 17:04 upstream 46c291e277f9 d75f7686 .config console log report ci-upstream-kasan-gce
2019/02/01 04:20 upstream 9f789567142c 0e8ea0a3 .config console log report ci-upstream-kasan-gce
2019/01/04 01:24 upstream 645ff1e8e704 7da23925 .config console log report ci-upstream-kasan-gce-smack-root
2018/08/27 18:32 upstream 5b394b2ddf03 758cd203 .config console log report ci-upstream-kasan-gce-root
2019/02/16 22:08 linux-next 7a92eb7cc1dc f42dee6d .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.