syzbot


possible deadlock in perf_trace_destroy

Status: closed as invalid on 2017/11/01 19:39
Subsystems: trace
[Documentation on labels]
Reported-by: syzbot+c0dc7dbcb186ca1f017281cb9cf6ea999d058146@syzkaller.appspotmail.com
First crash: 2430d, last: 2376d
Similar bugs (5)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in perf_trace_destroy (3) trace 12 1951d 2080d 0/26 auto-closed as invalid on 2019/06/14 09:18
upstream possible deadlock in perf_trace_destroy (2) trace C 2006 2269d 2355d 5/26 fixed on 2018/04/24 21:47
android-414 possible deadlock in perf_trace_destroy 1 2048d 2048d 0/1 auto-closed as invalid on 2019/03/09 03:11
android-414 possible deadlock in perf_trace_destroy (2) 1 1838d 1832d 0/1 auto-closed as invalid on 2019/10/05 09:33
linux-4.14 possible deadlock in perf_trace_destroy C inconclusive 829 1419d 1831d 0/1 upstream: reported C repro on 2019/04/15 13:38

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
4.14.0-rc2-mm1+ #11 Not tainted
------------------------------------------------------
syzkaller685541/7117 is trying to acquire lock:
 (event_mutex){+.+.}, at: [<ffffffff817730d8>] perf_trace_destroy+0x28/0x100 kernel/trace/trace_event_perf.c:234

but task is already holding lock:
 (&mm->mmap_sem){++++}, at: [<ffffffff8192c9f8>] vm_mmap_pgoff+0x198/0x280 mm/util.c:331

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #7 (&mm->mmap_sem){++++}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __might_fault+0x13a/0x1d0 mm/memory.c:4522
       _copy_to_user+0x2c/0xc0 lib/usercopy.c:24
       copy_to_user include/linux/uaccess.h:154 [inline]
       filldir+0x1a7/0x320 fs/readdir.c:196
       dir_emit_dot include/linux/fs.h:3341 [inline]
       dir_emit_dots include/linux/fs.h:3352 [inline]
       dcache_readdir+0x12d/0x5e0 fs/libfs.c:193
       iterate_dir+0x4b2/0x5d0 fs/readdir.c:51
       SYSC_getdents fs/readdir.c:231 [inline]
       SyS_getdents+0x225/0x450 fs/readdir.c:212
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #6 (&sb->s_type->i_mutex_key#5){++++}:
       down_write+0x87/0x120 kernel/locking/rwsem.c:53
       inode_lock include/linux/fs.h:712 [inline]
       handle_create+0x30c/0x760 drivers/base/devtmpfs.c:218
       handle drivers/base/devtmpfs.c:372 [inline]
       devtmpfsd+0x3b4/0x4b0 drivers/base/devtmpfs.c:398
       kthread+0x3c9/0x4b0 kernel/kthread.c:242
       ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431

-> #5 ((complete)&req.done){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       complete_acquire include/linux/completion.h:39 [inline]
       __wait_for_common kernel/sched/completion.c:108 [inline]
       wait_for_common kernel/sched/completion.c:122 [inline]
       wait_for_completion+0xcb/0x7b0 kernel/sched/completion.c:143
       devtmpfs_create_node+0x32b/0x4a0 drivers/base/devtmpfs.c:114
       device_add+0x120f/0x1640 drivers/base/core.c:1824
       device_create_groups_vargs+0x1f3/0x250 drivers/base/core.c:2430
       device_create_vargs drivers/base/core.c:2470 [inline]
       device_create+0xda/0x110 drivers/base/core.c:2506
       msr_device_create+0x26/0x40 arch/x86/kernel/msr.c:188
       cpuhp_invoke_callback+0x2ea/0x1d20 kernel/cpu.c:182
       cpuhp_thread_fun+0x48b/0x7e0 kernel/cpu.c:571
       smpboot_thread_fn+0x450/0x7c0 kernel/smpboot.c:164
       kthread+0x3c9/0x4b0 kernel/kthread.c:242
       ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431

-> #4 (cpuhp_state-up){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       cpuhp_lock_acquire kernel/cpu.c:85 [inline]
       cpuhp_invoke_ap_callback kernel/cpu.c:605 [inline]
       cpuhp_issue_call+0x1e6/0x4b0 kernel/cpu.c:1490
       __cpuhp_setup_state_cpuslocked+0x2c7/0x5f0 kernel/cpu.c:1637
       __cpuhp_setup_state+0xb0/0x140 kernel/cpu.c:1666
       cpuhp_setup_state include/linux/cpuhotplug.h:196 [inline]
       page_writeback_init+0x4d/0x71 mm/page-writeback.c:2084
       pagecache_init+0x48/0x4f mm/filemap.c:879
       start_kernel+0x6cd/0x760 init/main.c:693
       x86_64_start_reservations+0x2a/0x2c arch/x86/kernel/head64.c:377
       x86_64_start_kernel+0x77/0x7a arch/x86/kernel/head64.c:358
       secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:238

-> #3 (cpuhp_state_mutex){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x19d0 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       __cpuhp_setup_state_cpuslocked+0x5b/0x5f0 kernel/cpu.c:1612
       __cpuhp_setup_state+0xb0/0x140 kernel/cpu.c:1666
       cpuhp_setup_state_nocalls include/linux/cpuhotplug.h:224 [inline]
       kvm_guest_init+0x1f3/0x20f arch/x86/kernel/kvm.c:488
       setup_arch+0x17d1/0x19eb arch/x86/kernel/setup.c:1264
       start_kernel+0xa5/0x760 init/main.c:533
       x86_64_start_reservations+0x2a/0x2c arch/x86/kernel/head64.c:377
       x86_64_start_kernel+0x77/0x7a arch/x86/kernel/head64.c:358
       secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:238

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:35 [inline]
       percpu_down_read include/linux/percpu-rwsem.h:58 [inline]
       cpus_read_lock+0x42/0x90 kernel/cpu.c:293
       static_key_slow_inc+0x9d/0x3c0 kernel/jump_label.c:123
       tracepoint_add_func kernel/tracepoint.c:223 [inline]
       tracepoint_probe_register_prio+0x80d/0x9a0 kernel/tracepoint.c:283
       tracepoint_probe_register+0x2a/0x40 kernel/tracepoint.c:304
       trace_event_reg+0x167/0x320 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x4ef/0xab0 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8037
       perf_try_init_event+0xc9/0x1f0 kernel/events/core.c:9262
       perf_init_event kernel/events/core.c:9300 [inline]
       perf_event_alloc+0x1c5b/0x2a00 kernel/events/core.c:9559
       SYSC_perf_event_open+0x84e/0x2e00 kernel/events/core.c:10014
       SyS_perf_event_open+0x39/0x50 kernel/events/core.c:9900
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #1 (tracepoints_mutex){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x19d0 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       tracepoint_probe_register_prio+0xa0/0x9a0 kernel/tracepoint.c:279
       tracepoint_probe_register+0x2a/0x40 kernel/tracepoint.c:304
       trace_event_reg+0x167/0x320 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x4ef/0xab0 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8037
       perf_try_init_event+0xc9/0x1f0 kernel/events/core.c:9262
       perf_init_event kernel/events/core.c:9300 [inline]
       perf_event_alloc+0x1c5b/0x2a00 kernel/events/core.c:9559
       SYSC_perf_event_open+0x84e/0x2e00 kernel/events/core.c:10014
       SyS_perf_event_open+0x39/0x50 kernel/events/core.c:9900
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #0 (event_mutex){+.+.}:
       check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x19d0 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       perf_trace_destroy+0x28/0x100 kernel/trace/trace_event_perf.c:234
       tp_perf_event_destroy+0x15/0x20 kernel/events/core.c:8021
       _free_event+0x41d/0x1170 kernel/events/core.c:4193
       put_event+0x24/0x30 kernel/events/core.c:4276
       perf_mmap_close+0x60d/0xf90 kernel/events/core.c:5224
       remove_vma+0xb4/0x1b0 mm/mmap.c:172
       remove_vma_list mm/mmap.c:2475 [inline]
       do_munmap+0x82a/0xdf0 mm/mmap.c:2714
       mmap_region+0x59e/0x15a0 mm/mmap.c:1631
       do_mmap+0x6a1/0xd50 mm/mmap.c:1468
       do_mmap_pgoff include/linux/mm.h:2150 [inline]
       vm_mmap_pgoff+0x1de/0x280 mm/util.c:333
       SYSC_mmap_pgoff mm/mmap.c:1518 [inline]
       SyS_mmap_pgoff+0x23b/0x5f0 mm/mmap.c:1476
       SYSC_mmap arch/x86/kernel/sys_x86_64.c:99 [inline]
       SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:90
       entry_SYSCALL_64_fastpath+0x1f/0xbe

other info that might help us debug this:

Chain exists of:
  event_mutex --> &sb->s_type->i_mutex_key#5 --> &mm->mmap_sem

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&mm->mmap_sem);
                               lock(&sb->s_type->i_mutex_key#5);
                               lock(&mm->mmap_sem);
  lock(event_mutex);

 *** DEADLOCK ***

1 lock held by syzkaller685541/7117:
 #0:  (&mm->mmap_sem){++++}, at: [<ffffffff8192c9f8>] vm_mmap_pgoff+0x198/0x280 mm/util.c:331

stack backtrace:
CPU: 1 PID: 7117 Comm: syzkaller685541 Not tainted 4.14.0-rc2-mm1+ #11
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x194/0x257 lib/dump_stack.c:52
 print_circular_bug+0x503/0x710 kernel/locking/lockdep.c:1259
 check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894
 check_prevs_add kernel/locking/lockdep.c:2020 [inline]
 validate_chain kernel/locking/lockdep.c:2469 [inline]
 __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
 __mutex_lock_common kernel/locking/mutex.c:756 [inline]
 __mutex_lock+0x16f/0x19d0 kernel/locking/mutex.c:893
 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
 perf_trace_destroy+0x28/0x100 kernel/trace/trace_event_perf.c:234
 tp_perf_event_destroy+0x15/0x20 kernel/events/core.c:8021
 _free_event+0x41d/0x1170 kernel/events/core.c:4193
 put_event+0x24/0x30 kernel/events/core.c:4276
 perf_mmap_close+0x60d/0xf90 kernel/events/core.c:5224
 remove_vma+0xb4/0x1b0 mm/mmap.c:172
 remove_vma_list mm/mmap.c:2475 [inline]
 do_munmap+0x82a/0xdf0 mm/mmap.c:2714
 mmap_region+0x59e/0x15a0 mm/mmap.c:1631

Crashes (525):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2017/10/02 14:20 mmots c0232cb3af70 c26ea367 .config console log report syz C ci-upstream-mmots-kasan-gce
2017/09/18 00:25 mmots 720bbe532b7c c26ea367 .config console log report syz C ci-upstream-mmots-kasan-gce
2017/10/02 12:44 upstream 9e66317d3c92 c26ea367 .config console log report syz ci-upstream-kasan-gce-386
2017/09/29 08:07 upstream 02a2b05395dd c26ea367 .config console log report syz ci-upstream-kasan-gce-386
2017/09/29 07:55 upstream 02a2b05395dd c26ea367 .config console log report syz ci-upstream-kasan-gce-386
2017/09/28 11:19 upstream 9cd6681cb116 c26ea367 .config console log report syz ci-upstream-kasan-gce-386
2017/09/18 00:38 mmots 720bbe532b7c c26ea367 .config console log report syz ci-upstream-mmots-kasan-gce
2017/09/17 00:41 mmots 720bbe532b7c c26ea367 .config console log report syz ci-upstream-mmots-kasan-gce
2017/08/27 12:21 linux-next 7159188b70e3 4074aed7 .config console log report syz skylake-linux-next-kasan-qemu
2017/08/27 12:12 linux-next 7159188b70e3 4074aed7 .config console log report syz skylake-linux-next-kasan-qemu
2017/08/27 12:07 linux-next 7159188b70e3 4074aed7 .config console log report syz skylake-linux-next-kasan-qemu
2017/08/27 11:52 linux-next 7159188b70e3 4074aed7 .config console log report syz skylake-linux-next-kasan-qemu
2017/08/24 09:26 linux-next 9506597de2cd 3f1aca48 .config console log report syz ci-upstream-next-kasan-gce
2017/08/24 09:08 linux-next 9506597de2cd 3f1aca48 .config console log report syz skylake-linux-next-kasan-qemu
2017/10/02 08:31 upstream 368f89984bb9 c26ea367 .config console log report ci-upstream-kasan-gce
2017/10/02 07:57 upstream 368f89984bb9 c26ea367 .config console log report ci-upstream-kasan-gce
2017/09/05 01:04 upstream b1b6f83ac938 f400a0da .config console log report ci-upstream-kasan-gce
2017/10/04 19:15 upstream b7e141644178 c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 17:31 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 16:42 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 16:08 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 11:38 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 11:24 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 11:04 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 10:47 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 10:45 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 06:51 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 06:48 upstream d81fa669e3de c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/04 04:50 upstream 887c8ba753fb c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/03 22:27 upstream 887c8ba753fb c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/10/03 10:12 upstream 9e66317d3c92 c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/09/26 17:01 upstream e365806ac289 c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/09/26 16:10 upstream e365806ac289 c26ea367 .config console log report ci-upstream-kasan-gce-386
2017/09/26 09:11 upstream 19240e6b2a6c c26ea367 .config console log report ci-upstream-kasan-gce-386
* Struck through repros no longer work on HEAD.