syzbot


possible deadlock in ftrace_profile_set_filter (2)

Status: fixed on 2018/04/24 21:47
Subsystems: trace
[Documentation on labels]
Reported-by: syzbot+7b44b47914a5b3dc2038ff8f39e13a0e0f9e5f78@syzkaller.appspotmail.com
Fix commit: 43fa87f7deed perf/core: Fix another perf,trace,cpuhp lock inversion
First crash: 2341d, last: 2277d
Discussions (1)
Title Replies (including bot) Last reply
Re: possible deadlock in ftrace_profile_set_filter (2) 1 (1) 2018/04/21 18:29
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 possible deadlock in ftrace_profile_set_filter C 14295 417d 1837d 0/1 upstream: reported C repro on 2019/04/16 20:57
upstream possible deadlock in ftrace_profile_set_filter 2 2418d 2370d 0/26 closed as invalid on 2017/10/30 19:48

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
4.15.0-rc8+ #1 Not tainted
------------------------------------------------------
syzkaller301315/3887 is trying to acquire lock:
 (event_mutex){+.+.}, at: [<00000000dfa1326b>] ftrace_profile_set_filter+0x7a/0x270 kernel/trace/trace_events_filter.c:2266

but task is already holding lock:
 (&cpuctx_mutex){+.+.}, at: [<00000000e276bad6>] perf_event_ctx_lock_nested+0x21b/0x450 kernel/events/core.c:1249

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 (&cpuctx_mutex){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       perf_event_init_cpu+0xb6/0x160 kernel/events/core.c:11076
       perf_event_init+0x4e9/0x549 kernel/events/core.c:11123
       start_kernel+0x4cc/0x819 init/main.c:627
       x86_64_start_reservations+0x2a/0x2c arch/x86/kernel/head64.c:378
       x86_64_start_kernel+0x77/0x7a arch/x86/kernel/head64.c:359
       secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:237

-> #3 (pmus_lock){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       perf_event_init_cpu+0x2f/0x160 kernel/events/core.c:11070
       cpuhp_invoke_callback+0x2ea/0x1d20 kernel/cpu.c:182
       cpuhp_up_callbacks kernel/cpu.c:477 [inline]
       _cpu_up+0x216/0x510 kernel/cpu.c:1036
       do_cpu_up+0x73/0xa0 kernel/cpu.c:1066
       cpu_up+0x18/0x20 kernel/cpu.c:1074
       smp_init+0x13a/0x152 kernel/smp.c:578
       kernel_init_freeable+0x2fe/0x521 init/main.c:1067
       kernel_init+0x13/0x180 init/main.c:999
       ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:541

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
       percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline]
       percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
       cpus_read_lock+0x42/0x90 kernel/cpu.c:293
       static_key_slow_inc+0x9d/0x3c0 kernel/jump_label.c:123
       tracepoint_add_func kernel/tracepoint.c:223 [inline]
       tracepoint_probe_register_prio+0x80d/0x9a0 kernel/tracepoint.c:283
       tracepoint_probe_register+0x2a/0x40 kernel/tracepoint.c:304
       trace_event_reg+0x167/0x320 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x4ef/0xab0 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:7959
       perf_try_init_event+0xc9/0x1f0 kernel/events/core.c:9192
       perf_init_event kernel/events/core.c:9230 [inline]
       perf_event_alloc+0x1cc6/0x2b00 kernel/events/core.c:9494
       SYSC_perf_event_open+0x84e/0x2e00 kernel/events/core.c:9949

Crashes (1007):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2018/01/22 17:48 bpf-next ebdd7b491b8a 228e3d95 .config console log report syz C ci-upstream-bpf-next-kasan-gce
2018/02/01 09:57 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:52 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:46 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:44 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:36 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:34 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:33 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:31 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:22 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:21 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:13 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 09:04 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:56 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:53 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:50 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:43 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:35 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:34 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:33 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:31 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:15 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:12 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 08:08 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:52 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:44 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:39 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:35 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:22 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:18 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:16 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:15 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:13 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 07:06 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:58 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:55 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:48 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:26 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:23 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:18 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
2018/02/01 06:15 bpf-next 868c36dcc949 02553e22 .config console log report ci-upstream-bpf-next-kasan-gce
* Struck through repros no longer work on HEAD.