syzbot


possible deadlock in ftrace_profile_set_filter

Status: closed as invalid on 2017/10/30 19:48
Reported-by: syzbot+157d0a1a9b300d40d179b5d90ae27e6c1295cb9b@syzkaller.appspotmail.com
First crash: 2422d, last: 2415d
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in ftrace_profile_set_filter (2) trace C 1007 2274d 2334d 5/26 fixed on 2018/04/24 21:47
linux-4.14 possible deadlock in ftrace_profile_set_filter C 14295 414d 1834d 0/1 upstream: reported C repro on 2019/04/16 20:57

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
4.13.0-next-20170912+ #20 Not tainted
------------------------------------------------------
syz-executor4/3313 is trying to acquire lock:
 (event_mutex){+.+.}, at: [<ffffffff81766f1a>] ftrace_profile_set_filter+0x7a/0x270 kernel/trace/trace_events_filter.c:2266

but task is already holding lock:
 (&ctx->mutex){+.+.}, at: [<ffffffff8180923c>] perf_event_ctx_lock_nested+0x1dc/0x3c0 kernel/events/core.c:1210

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #8 (&ctx->mutex){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       perf_event_ctx_lock_nested+0x1dc/0x3c0 kernel/events/core.c:1210
       perf_event_ctx_lock kernel/events/core.c:1223 [inline]
       perf_read+0xb9/0x970 kernel/events/core.c:4584
       do_loop_readv_writev fs/read_write.c:693 [inline]
       do_iter_read+0x3db/0x5b0 fs/read_write.c:917
       vfs_readv+0x121/0x1c0 fs/read_write.c:979
       kernel_readv fs/splice.c:361 [inline]
       default_file_splice_read+0x508/0xae0 fs/splice.c:416
       do_splice_to+0x110/0x170 fs/splice.c:880
       do_splice fs/splice.c:1173 [inline]
       SYSC_splice fs/splice.c:1402 [inline]
       SyS_splice+0x11a8/0x1630 fs/splice.c:1382
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #7 (&pipe->mutex/1){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       pipe_lock_nested fs/pipe.c:66 [inline]
       pipe_lock+0x56/0x70 fs/pipe.c:74
       iter_file_splice_write+0x264/0xf50 fs/splice.c:699
       do_splice_from fs/splice.c:851 [inline]
       do_splice fs/splice.c:1147 [inline]
       SYSC_splice fs/splice.c:1402 [inline]
       SyS_splice+0x7d5/0x1630 fs/splice.c:1382
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #6 (sb_writers){.+.+}:
       percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:35 [inline]
       percpu_down_read include/linux/percpu-rwsem.h:58 [inline]
       __sb_start_write+0x18f/0x290 fs/super.c:1341
       sb_start_write include/linux/fs.h:1541 [inline]
       mnt_want_write+0x3f/0xb0 fs/namespace.c:387
       filename_create+0x12b/0x520 fs/namei.c:3628
       kern_path_create+0x33/0x40 fs/namei.c:3674
       handle_create+0xc0/0x760 drivers/base/devtmpfs.c:203

-> #5 ((complete)&req.done){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       complete_acquire include/linux/completion.h:39 [inline]
       __wait_for_common kernel/sched/completion.c:108 [inline]
       wait_for_common kernel/sched/completion.c:122 [inline]
       wait_for_completion+0xc8/0x770 kernel/sched/completion.c:143
       devtmpfs_create_node+0x32b/0x4a0 drivers/base/devtmpfs.c:115
       device_add+0x120f/0x1640 drivers/base/core.c:1824
       device_create_groups_vargs+0x1f3/0x250 drivers/base/core.c:2430
       device_create_vargs drivers/base/core.c:2470 [inline]
       device_create+0xda/0x110 drivers/base/core.c:2506
       msr_device_create+0x26/0x40 arch/x86/kernel/msr.c:188
       cpuhp_invoke_callback+0x256/0x14d0 kernel/cpu.c:145
       cpuhp_thread_fun+0x265/0x520 kernel/cpu.c:434
       smpboot_thread_fn+0x489/0x850 kernel/smpboot.c:164
       kthread+0x39c/0x470 kernel/kthread.c:231
       ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431

-> #4 (cpuhp_state){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       cpuhp_invoke_ap_callback kernel/cpu.c:467 [inline]
       cpuhp_issue_call+0x1a2/0x3e0 kernel/cpu.c:1308
       __cpuhp_setup_state_cpuslocked+0x2d6/0x5f0 kernel/cpu.c:1455
       __cpuhp_setup_state+0xb0/0x140 kernel/cpu.c:1484
       cpuhp_setup_state include/linux/cpuhotplug.h:177 [inline]
       page_writeback_init+0x4d/0x71 mm/page-writeback.c:2082
       pagecache_init+0x48/0x4f mm/filemap.c:871
       start_kernel+0x6c1/0x754 init/main.c:690
       x86_64_start_reservations+0x2a/0x2c arch/x86/kernel/head64.c:377
       x86_64_start_kernel+0x87/0x8a arch/x86/kernel/head64.c:358
       verify_cpu+0x0/0xfb

-> #3 (cpuhp_state_mutex){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       __cpuhp_setup_state_cpuslocked+0x5b/0x5f0 kernel/cpu.c:1430
       __cpuhp_setup_state+0xb0/0x140 kernel/cpu.c:1484
       cpuhp_setup_state_nocalls include/linux/cpuhotplug.h:205 [inline]
       kvm_guest_init+0x1f3/0x20f arch/x86/kernel/kvm.c:488
       setup_arch+0x1899/0x1ab3 arch/x86/kernel/setup.c:1295
       start_kernel+0xa5/0x754 init/main.c:530
       x86_64_start_reservations+0x2a/0x2c arch/x86/kernel/head64.c:377
       x86_64_start_kernel+0x87/0x8a arch/x86/kernel/head64.c:358
       verify_cpu+0x0/0xfb

-> #2 (cpu_hotplug_lock.rw_sem){++++}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:35 [inline]
       percpu_down_read include/linux/percpu-rwsem.h:58 [inline]
       cpus_read_lock+0x42/0x90 kernel/cpu.c:218
       static_key_slow_inc+0x9d/0x3c0 kernel/jump_label.c:123
       tracepoint_add_func kernel/tracepoint.c:223 [inline]
       tracepoint_probe_register_prio+0x80d/0x9a0 kernel/tracepoint.c:283
       tracepoint_probe_register+0x2a/0x40 kernel/tracepoint.c:304
       trace_event_reg+0x167/0x320 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x4f9/0xab0 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8037
       perf_try_init_event+0xc9/0x1f0 kernel/events/core.c:9261
       perf_init_event kernel/events/core.c:9299 [inline]
       perf_event_alloc+0x1c5b/0x2a00 kernel/events/core.c:9558
       SYSC_perf_event_open+0x84e/0x2e00 kernel/events/core.c:10013
       SyS_perf_event_open+0x39/0x50 kernel/events/core.c:9899
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #1 (tracepoints_mutex){+.+.}:
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       tracepoint_probe_register_prio+0xa0/0x9a0 kernel/tracepoint.c:279
       tracepoint_probe_register+0x2a/0x40 kernel/tracepoint.c:304
       trace_event_reg+0x167/0x320 kernel/trace/trace_events.c:305
       perf_trace_event_reg kernel/trace/trace_event_perf.c:122 [inline]
       perf_trace_event_init kernel/trace/trace_event_perf.c:197 [inline]
       perf_trace_init+0x4f9/0xab0 kernel/trace/trace_event_perf.c:221
       perf_tp_event_init+0x7d/0xf0 kernel/events/core.c:8037
       perf_try_init_event+0xc9/0x1f0 kernel/events/core.c:9261
       perf_init_event kernel/events/core.c:9299 [inline]
       perf_event_alloc+0x1c5b/0x2a00 kernel/events/core.c:9558
       SYSC_perf_event_open+0x84e/0x2e00 kernel/events/core.c:10013
       SyS_perf_event_open+0x39/0x50 kernel/events/core.c:9899
       entry_SYSCALL_64_fastpath+0x1f/0xbe

-> #0 (event_mutex){+.+.}:
       check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894
       check_prevs_add kernel/locking/lockdep.c:2020 [inline]
       validate_chain kernel/locking/lockdep.c:2469 [inline]
       __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       ftrace_profile_set_filter+0x7a/0x270 kernel/trace/trace_events_filter.c:2266
       perf_event_set_filter kernel/events/core.c:8622 [inline]
       _perf_ioctl kernel/events/core.c:4785 [inline]
       perf_ioctl+0x1026/0x1360 kernel/events/core.c:4822
       vfs_ioctl fs/ioctl.c:45 [inline]
       do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:685
       SYSC_ioctl fs/ioctl.c:700 [inline]
       SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
       entry_SYSCALL_64_fastpath+0x1f/0xbe

other info that might help us debug this:

Chain exists of:
  event_mutex --> &pipe->mutex/1 --> &ctx->mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&ctx->mutex);
                               lock(&pipe->mutex/1);
                               lock(&ctx->mutex);
  lock(event_mutex);

 *** DEADLOCK ***

1 lock held by syz-executor4/3313:
 #0:  (&ctx->mutex){+.+.}, at: [<ffffffff8180923c>] perf_event_ctx_lock_nested+0x1dc/0x3c0 kernel/events/core.c:1210

stack backtrace:
CPU: 1 PID: 3313 Comm: syz-executor4 Not tainted 4.13.0-next-20170912+ #20
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x194/0x257 lib/dump_stack.c:52
 print_circular_bug+0x503/0x710 kernel/locking/lockdep.c:1259
 check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894
 check_prevs_add kernel/locking/lockdep.c:2020 [inline]
 validate_chain kernel/locking/lockdep.c:2469 [inline]
 __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498
 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002
 __mutex_lock_common kernel/locking/mutex.c:756 [inline]
 __mutex_lock+0x16f/0x1870 kernel/locking/mutex.c:893
 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
 ftrace_profile_set_filter+0x7a/0x270 kernel/trace/trace_events_filter.c:2266
 perf_event_set_filter kernel/events/core.c:8622 [inline]
 _perf_ioctl kernel/events/core.c:4785 [inline]
 perf_ioctl+0x1026/0x1360 kernel/events/core.c:4822
 vfs_ioctl fs/ioctl.c:45 [inline]
 do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:685
 SYSC_ioctl fs/ioctl.c:700 [inline]
 SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691
 entry_SYSCALL_64_fastpath+0x1f/0xbe
RIP: 0033:0x451e59
RSP: 002b:00007fc2f55a7c08 EFLAGS: 00000216 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000718000 RCX: 0000000000451e59
RDX: 00000000208bb000 RSI: 0000000040082406 RDI: 0000000000000006
RBP: 0000000000000082 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000216 R12: 0000000000000000
R13: 0000000000a6f7ef R14: 00007fc2f55a89c0 R15: 0000000000000000
mmap: syz-executor5 (3330) uses deprecated remap_file_pages() syscall. See Documentation/vm/remap_file_pages.txt.
capability: warning: `syz-executor0' uses deprecated v2 capabilities in a way that may be insecure
FAULT_INJECTION: forcing a failure.
name fail_page_alloc, interval 1, probability 0, space 0, times 1
CPU: 1 PID: 4181 Comm: syz-executor5 Not tainted 4.13.0-next-20170912+ #20
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x194/0x257 lib/dump_stack.c:52
 fail_dump lib/fault-inject.c:51 [inline]
 should_fail+0x8c0/0xa40 lib/fault-inject.c:149
 should_fail_alloc_page mm/page_alloc.c:2915 [inline]
 prepare_alloc_pages mm/page_alloc.c:4151 [inline]
 __alloc_pages_nodemask+0x338/0xd80 mm/page_alloc.c:4187
 alloc_pages_current+0xb6/0x1e0 mm/mempolicy.c:2035
 alloc_pages include/linux/gfp.h:505 [inline]
 skb_page_frag_refill+0x358/0x5f0 net/core/sock.c:2196
 tun_build_skb.isra.42+0x2a2/0x1690 drivers/net/tun.c:1289
 tun_get_user+0x1dad/0x2150 drivers/net/tun.c:1455
 tun_chr_write_iter+0xde/0x190 drivers/net/tun.c:1579
 call_write_iter include/linux/fs.h:1770 [inline]
 new_sync_write fs/read_write.c:468 [inline]
 __vfs_write+0x68a/0x970 fs/read_write.c:481
 vfs_write+0x18f/0x510 fs/read_write.c:543
 SYSC_write fs/read_write.c:588 [inline]
 SyS_write+0xef/0x220 fs/read_write.c:580
 entry_SYSCALL_64_fastpath+0x1f/0xbe
RIP: 0033:0x40c2c1
RSP: 002b:00007f8bf0184c10 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000718000 RCX: 000000000040c2c1
RDX: 000000000000003e RSI: 0000000020d67f77 RDI: 0000000000000015
RBP: 00007f8bf0184a10 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000000f4240 R11: 0000000000000293 R12: 00000000004b69f7
R13: 00007f8bf0184b48 R14: 00000000004b6a07 R15: 0000000000000000
FAULT_INJECTION: forcing a failure.
name fail_page_alloc, interval 1, probability 0, space 0, times 0
CPU: 0 PID: 4341 Comm: syz-executor4 Not tainted 4.13.0-next-20170912+ #20
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:16 [inline]
 dump_stack+0x194/0x257 lib/dump_stack.c:52
 fail_dump lib/fault-inject.c:51 [inline]
 should_fail+0x8c0/0xa40 lib/fault-inject.c:149
 should_fail_alloc_page mm/page_alloc.c:2915 [inline]
 prepare_alloc_pages mm/page_alloc.c:4151 [inline]
 __alloc_pages_nodemask+0x338/0xd80 mm/page_alloc.c:4187
 alloc_pages_current+0xb6/0x1e0 mm/mempolicy.c:2035
 alloc_pages include/linux/gfp.h:505 [inline]
 skb_page_frag_refill+0x358/0x5f0 net/core/sock.c:2196
 tun_build_skb.isra.42+0x2a2/0x1690 drivers/net/tun.c:1289
 tun_get_user+0x1dad/0x2150 drivers/net/tun.c:1455
 tun_chr_write_iter+0xde/0x190 drivers/net/tun.c:1579
 call_write_iter include/linux/fs.h:1770 [inline]
 new_sync_write fs/read_write.c:468 [inline]
 __vfs_write+0x68a/0x970 fs/read_write.c:481
 vfs_write+0x18f/0x510 fs/read_write.c:543
 SYSC_write fs/read_write.c:588 [inline]
 SyS_write+0xef/0x220 fs/read_write.c:580
 entry_SYSCALL_64_fastpath+0x1f/0xbe
RIP: 0033:0x40c2c1
RSP: 002b:00007fc2f55a7c10 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000718000 RCX: 000000000040c2c1
RDX: 0000000000000046 RSI: 0000000020000000 RDI: 0000000000000015
RBP: 00007fc2f55a7a10 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000000f4240 R11: 0000000000000293 R12: 00000000004b69f7
R13: 00007fc2f55a7b48 R14: 00000000004b6a07 R15: 0000000000000000
netlink: 2 bytes leftover after parsing attributes in process `syz-executor5'.
IPv6: Can't replace route, no match found
netlink: 2 bytes leftover after parsing attributes in process `syz-executor5'.
IPv6: Can't replace route, no match found
capability: warning: `syz-executor5' uses 32-bit capabilities (legacy support in use)
audit: type=1326 audit(1505239333.837:2): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=5973 comm="syz-executor4" exe="/root/syz-executor4" sig=31 arch=c000003e syscall=202 compat=0 ip=0x451e59 code=0xffff0000
audit: type=1326 audit(1505239333.922:3): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=5973 comm="syz-executor4" exe="/root/syz-executor4" sig=31 arch=c000003e syscall=202 compat=0 ip=0x451e59 code=0xffff0000

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2017/09/12 18:02 linux-next 0d71e2d4aa14 0bd6a0a5 .config console log report skylake-linux-next-kasan-qemu
2017/09/06 15:48 mmots c97efd59aceb 0ed1da4a .config console log report ci-upstream-mmots-kasan-gce
* Struck through repros no longer work on HEAD.