syzbot


possible deadlock in perf_event_read_value

Status: upstream: reported syz repro on 2019/04/18 16:03
Reported-by: syzbot+55050b0f87558b5401b7@syzkaller.appspotmail.com
First crash: 2050d, last: 760d
Fix bisection: failed (error log, bisect log)
  
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-414 possible deadlock in perf_event_read_value 1 2116d 2054d 0/1 auto-closed as invalid on 2019/08/10 05:35
Fix bisection attempts (15)
Created Duration User Patch Repo Result
2023/02/03 12:04 1m bisect fix linux-4.14.y error job log
2022/10/29 03:32 28m bisect fix linux-4.14.y OK (0) job log log
2022/09/01 23:53 27m bisect fix linux-4.14.y OK (0) job log log
2022/08/02 23:25 27m bisect fix linux-4.14.y OK (0) job log log
2022/07/03 10:35 27m bisect fix linux-4.14.y OK (0) job log log
2022/06/03 10:08 26m bisect fix linux-4.14.y OK (0) job log log
2022/02/11 19:50 39m bisect fix linux-4.14.y OK (0) job log log
2022/01/12 17:56 27m bisect fix linux-4.14.y OK (0) job log log
2021/12/13 16:57 30m bisect fix linux-4.14.y OK (0) job log log
2021/11/13 16:06 26m bisect fix linux-4.14.y OK (0) job log log
2021/10/14 15:36 27m bisect fix linux-4.14.y OK (0) job log log
2021/09/14 13:10 30m bisect fix linux-4.14.y OK (0) job log log
2021/08/15 12:46 23m bisect fix linux-4.14.y OK (0) job log log
2021/07/16 12:15 30m bisect fix linux-4.14.y OK (0) job log log
2021/06/16 11:53 21m bisect fix linux-4.14.y OK (0) job log log

Sample crash report:
batman_adv: batadv0: Interface activated: batadv_slave_1
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
======================================================
WARNING: possible circular locking dependency detected
4.14.176-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.0/7960 is trying to acquire lock:
 (&event->child_mutex){+.+.}, at: [<ffffffff8168e018>] perf_event_read_value+0x78/0x410 kernel/events/core.c:4447

but task is already holding lock:
 (&cpuctx_mutex){+.+.}, at: [<ffffffff8168c52d>] perf_event_ctx_lock_nested+0x14d/0x2c0 kernel/events/core.c:1235

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #5 (&cpuctx_mutex){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
       perf_event_init_cpu+0xb7/0x170 kernel/events/core.c:11234
       perf_event_init+0x2cc/0x308 kernel/events/core.c:11281
       start_kernel+0x3b1/0x676 init/main.c:620
       secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:240

-> #4 (pmus_lock){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
       perf_event_init_cpu+0x2c/0x170 kernel/events/core.c:11228
       cpuhp_invoke_callback+0x1e6/0x1ad0 kernel/cpu.c:184
       cpuhp_up_callbacks kernel/cpu.c:572 [inline]
       _cpu_up+0x21a/0x520 kernel/cpu.c:1140
       do_cpu_up kernel/cpu.c:1175 [inline]
       do_cpu_up+0x9a/0x160 kernel/cpu.c:1147
       smp_init+0x149/0x15e kernel/smp.c:578
       kernel_init_freeable+0x305/0x526 init/main.c:1066
       kernel_init+0xd/0x15b init/main.c:998
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #3 (cpu_hotplug_lock.rw_sem){++++}:
       percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline]
       percpu_down_read include/linux/percpu-rwsem.h:59 [inline]
       cpus_read_lock+0x39/0xc0 kernel/cpu.c:295
       static_key_slow_inc+0xe/0x20 kernel/jump_label.c:123
       tracepoint_add_func kernel/tracepoint.c:223 [inline]
       tracepoint_probe_register_prio+0x4d4/0x6d0 kernel/tracepoint.c:283
       trace_event_reg+0x272/0x330 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x424/0xa50 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x79/0xf0 kernel/events/core.c:8117
       perf_try_init_event+0x15b/0x1f0 kernel/events/core.c:9353
       perf_init_event kernel/events/core.c:9391 [inline]
       perf_event_alloc.part.0+0xd89/0x25a0 kernel/events/core.c:9651
       perf_event_alloc kernel/events/core.c:10004 [inline]
       SYSC_perf_event_open+0xa80/0x24a0 kernel/events/core.c:10108
       do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
       entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #2 (tracepoints_mutex){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
       tracepoint_probe_register_prio+0x35/0x6d0 kernel/tracepoint.c:279
       trace_event_reg+0x272/0x330 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x424/0xa50 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x79/0xf0 kernel/events/core.c:8117
       perf_try_init_event+0x15b/0x1f0 kernel/events/core.c:9353
       perf_init_event kernel/events/core.c:9391 [inline]
       perf_event_alloc.part.0+0xd89/0x25a0 kernel/events/core.c:9651
       perf_event_alloc kernel/events/core.c:10004 [inline]
       SYSC_perf_event_open+0xa80/0x24a0 kernel/events/core.c:10108
       do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
       entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #1 (event_mutex){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
       perf_trace_destroy+0x23/0xf0 kernel/trace/trace_event_perf.c:234
       _free_event+0x328/0xe30 kernel/events/core.c:4238
       free_event+0x32/0x40 kernel/events/core.c:4265
       perf_event_release_kernel+0x35c/0x870 kernel/events/core.c:4409
       perf_release+0x33/0x40 kernel/events/core.c:4435
       __fput+0x25f/0x790 fs/file_table.c:210
       task_work_run+0x113/0x190 kernel/task_work.c:113
       tracehook_notify_resume include/linux/tracehook.h:191 [inline]
       exit_to_usermode_loop+0x1d6/0x220 arch/x86/entry/common.c:164
       prepare_exit_to_usermode arch/x86/entry/common.c:199 [inline]
       syscall_return_slowpath arch/x86/entry/common.c:270 [inline]
       do_syscall_64+0x4a3/0x640 arch/x86/entry/common.c:297
       entry_SYSCALL_64_after_hwframe+0x42/0xb7

-> #0 (&event->child_mutex){+.+.}:
       lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3994
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
       perf_event_read_value+0x78/0x410 kernel/events/core.c:4447
       perf_read_one kernel/events/core.c:4569 [inline]
       __perf_read kernel/events/core.c:4620 [inline]
       perf_read+0x3e2/0x7c0 kernel/events/core.c:4633
       do_loop_readv_writev fs/read_write.c:695 [inline]
       do_loop_readv_writev fs/read_write.c:682 [inline]
       do_iter_read+0x3e3/0x5a0 fs/read_write.c:919
       vfs_readv+0xd3/0x130 fs/read_write.c:981
       do_readv+0xfc/0x2c0 fs/read_write.c:1014
       do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
       entry_SYSCALL_64_after_hwframe+0x42/0xb7

other info that might help us debug this:

Chain exists of:
  &event->child_mutex --> pmus_lock --> &cpuctx_mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&cpuctx_mutex);
                               lock(pmus_lock);
                               lock(&cpuctx_mutex);
  lock(&event->child_mutex);

 *** DEADLOCK ***

1 lock held by syz-executor.0/7960:
 #0:  (&cpuctx_mutex){+.+.}, at: [<ffffffff8168c52d>] perf_event_ctx_lock_nested+0x14d/0x2c0 kernel/events/core.c:1235

stack backtrace:
CPU: 1 PID: 7960 Comm: syz-executor.0 Not tainted 4.14.176-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x13e/0x194 lib/dump_stack.c:58
 print_circular_bug.isra.0.cold+0x1c4/0x282 kernel/locking/lockdep.c:1258
 check_prev_add kernel/locking/lockdep.c:1901 [inline]
 check_prevs_add kernel/locking/lockdep.c:2018 [inline]
 validate_chain kernel/locking/lockdep.c:2460 [inline]
 __lock_acquire+0x2cb3/0x4620 kernel/locking/lockdep.c:3487
 lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3994
 __mutex_lock_common kernel/locking/mutex.c:756 [inline]
 __mutex_lock+0xe8/0x1470 kernel/locking/mutex.c:893
 perf_event_read_value+0x78/0x410 kernel/events/core.c:4447
 perf_read_one kernel/events/core.c:4569 [inline]
 __perf_read kernel/events/core.c:4620 [inline]
 perf_read+0x3e2/0x7c0 kernel/events/core.c:4633
 do_loop_readv_writev fs/read_write.c:695 [inline]
 do_loop_readv_writev fs/read_write.c:682 [inline]
 do_iter_read+0x3e3/0x5a0 fs/read_write.c:919
 vfs_readv+0xd3/0x130 fs/read_write.c:981
 do_readv+0xfc/0x2c0 fs/read_write.c:1014
 do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x45c869
RSP: 002b:00007f87dbb46c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000013
RAX: ffffffffffffffda RBX: 00007f87dbb476d4 RCX: 000000000045c869
RDX: 0000000000000001 RSI: 0000000020000280 RDI: 0000000000000008
RBP: 000000000076bf00 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 000000000000088c R14: 00000000004cb270 R15: 000000000076bf0c

Crashes (445):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/04/19 03:08 linux-4.14.y c10b57a567e4 365fba24 .config console log report syz ci2-linux-4-14
2022/05/04 10:08 linux-4.14.y e3a56aaade89 dc9e5259 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/04/11 17:06 linux-4.14.y 74766a973637 af01ee7d .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/04/07 04:24 linux-4.14.y 74766a973637 53c67432 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/04/06 10:31 linux-4.14.y 74766a973637 86b4b7f8 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/04/01 02:10 linux-4.14.y af1af6ebca0e 68fc921a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/03/09 00:11 linux-4.14.y c9fcc1545c3b 9e8eaa75 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2022/02/11 21:14 linux-4.14.y 8034e99d1a01 8b9ca619 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/17 11:51 linux-4.14.y 7d7d1c0ab3eb a2eb125d .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/16 04:24 linux-4.14.y 7d7d1c0ab3eb f54a5c09 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/15 20:08 linux-4.14.y 7d7d1c0ab3eb 93f844de .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/12 06:59 linux-4.14.y 7d7d1c0ab3eb 86294427 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/11 21:31 linux-4.14.y 7d7d1c0ab3eb b3c3bb8e .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/08 13:13 linux-4.14.y 7d7d1c0ab3eb bc5434be .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/07 23:52 linux-4.14.y 7d7d1c0ab3eb bc5434be .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/05/07 09:33 linux-4.14.y 7d7d1c0ab3eb f6da8120 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/26 22:00 linux-4.14.y cf256fbcbe34 8ee2dea6 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/26 12:27 linux-4.14.y cf256fbcbe34 e60b7df1 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/22 03:46 linux-4.14.y cf256fbcbe34 2bc8999a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/21 21:44 linux-4.14.y cf256fbcbe34 2bc8999a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/21 10:20 linux-4.14.y cf256fbcbe34 95777977 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/15 18:23 linux-4.14.y 958e517f4e16 c59079a6 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/14 22:01 linux-4.14.y 958e517f4e16 fcdb12ba .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/10 15:07 linux-4.14.y 958e517f4e16 6a81331a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/04/10 06:17 linux-4.14.y 0cc244011f40 6a81331a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/29 00:21 linux-4.14.y 670d6552eda8 a8529b82 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/27 20:59 linux-4.14.y 670d6552eda8 a8529b82 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/25 11:36 linux-4.14.y 670d6552eda8 6a383ecf .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/15 15:07 linux-4.14.y c7150cd2fa8c fdb2bb2c .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/12 21:54 linux-4.14.y c7150cd2fa8c 429d8a6b .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/11 16:04 linux-4.14.y c7150cd2fa8c c2ca1f2a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/10 17:31 linux-4.14.y 1d177c0872ab 764067f3 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/10 16:01 linux-4.14.y 1d177c0872ab 764067f3 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/09 17:07 linux-4.14.y 1d177c0872ab 26967e35 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/06 19:45 linux-4.14.y 397a88b2cc86 e4b4d570 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/02 08:55 linux-4.14.y 3242aa3a635c 92ead296 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/03/01 20:24 linux-4.14.y 3242aa3a635c 183afb6c .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/28 15:02 linux-4.14.y 3242aa3a635c 4c37c133 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/25 14:53 linux-4.14.y 3242aa3a635c 76f7fc95 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/25 07:39 linux-4.14.y 3242aa3a635c fcc6d71b .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/23 15:27 linux-4.14.y 3242aa3a635c fcc6d71b .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/23 14:13 linux-4.14.y 3242aa3a635c fcc6d71b .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/23 03:15 linux-4.14.y 29c52025152b fcc6d71b .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/17 07:41 linux-4.14.y 2c8a3fceddf0 052f8d9f .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/10 22:12 linux-4.14.y 2c8a3fceddf0 a52ee10a .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/10 14:31 linux-4.14.y 2c8a3fceddf0 9c8b8541 .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/02/08 03:25 linux-4.14.y 2c8a3fceddf0 2ce644fc .config console log report info ci2-linux-4-14 possible deadlock in perf_event_read_value
2021/01/17 11:43 linux-4.14.y f79dc86058bc 813be542 .config console log report info ci2-linux-4-14
2019/04/18 15:02 linux-4.14.y 58b454ebf81e b0e8efcb .config console log report ci2-linux-4-14
* Struck through repros no longer work on HEAD.