syzbot


possible deadlock in __mmap_lock_do_trace_start_locking

Status: upstream: reported on 2024/04/26 02:33
Subsystems: mm
[Documentation on labels]
Reported-by: syzbot+6ff90931779bcdfc840c@syzkaller.appspotmail.com
First crash: 54d, last: 7d01h
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [mm?] possible deadlock in __mmap_lock_do_trace_start_locking 0 (1) 2024/04/26 02:33

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.10.0-rc1-syzkaller-g7d0b3953f6d8 #0 Not tainted
------------------------------------------------------
syz-executor.2/8056 is trying to acquire lock:
ffff8880b9438828 (lock#10){+.+.}-{2:2}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
ffff8880b9438828 (lock#10){+.+.}-{2:2}, at: __mmap_lock_do_trace_start_locking+0x83/0x620 mm/mmap_lock.c:230

but task is already holding lock:
ffffe8ffffc64c88 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}, at: psi_task_switch+0x199/0x770 kernel/sched/psi.c:938

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       do_write_seqcount_begin_nested include/linux/seqlock.h:469 [inline]
       do_write_seqcount_begin include/linux/seqlock.h:495 [inline]
       psi_group_change+0x113/0x12d0 kernel/sched/psi.c:787
       psi_task_change+0xfd/0x280 kernel/sched/psi.c:912
       psi_enqueue kernel/sched/stats.h:139 [inline]
       enqueue_task+0x2a6/0x2f0 kernel/sched/core.c:2112
       activate_task kernel/sched/core.c:2146 [inline]
       wake_up_new_task+0x563/0xc30 kernel/sched/core.c:4900
       kernel_clone+0x4ee/0x8f0 kernel/fork.c:2828
       user_mode_thread+0x132/0x1a0 kernel/fork.c:2875
       rest_init+0x23/0x300 init/main.c:712
       start_kernel+0x47a/0x500 init/main.c:1103
       x86_64_start_reservations+0x2a/0x30 arch/x86/kernel/head64.c:507
       x86_64_start_kernel+0x99/0xa0 arch/x86/kernel/head64.c:488
       common_startup_64+0x13e/0x147

-> #2 (&rq->__lock){-.-.}-{2:2}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       _raw_spin_lock_nested+0x31/0x40 kernel/locking/spinlock.c:378
       raw_spin_rq_lock_nested+0x2a/0x140 kernel/sched/core.c:559
       raw_spin_rq_lock kernel/sched/sched.h:1406 [inline]
       _raw_spin_rq_lock_irqsave kernel/sched/sched.h:1425 [inline]
       rq_lock_irqsave kernel/sched/sched.h:1686 [inline]
       class_rq_lock_irqsave_constructor kernel/sched/sched.h:1740 [inline]
       sched_mm_cid_exit_signals+0x17b/0x4b0 kernel/sched/core.c:12004
       exit_signals+0x2a1/0x5c0 kernel/signal.c:3014
       do_exit+0x6b4/0x27e0 kernel/exit.c:833
       __do_sys_exit kernel/exit.c:990 [inline]
       __se_sys_exit kernel/exit.c:988 [inline]
       __pfx___ia32_sys_exit+0x0/0x10 kernel/exit.c:988
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #1 (&sighand->siglock){-.-.}-{2:2}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
       _raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
       __lock_task_sighand+0x149/0x2d0 kernel/signal.c:1412
       lock_task_sighand include/linux/sched/signal.h:746 [inline]
       do_send_sig_info kernel/signal.c:1300 [inline]
       group_send_sig_info+0x274/0x310 kernel/signal.c:1451
       bpf_send_signal_common+0x2dd/0x430 kernel/trace/bpf_trace.c:881
       ____bpf_send_signal_thread kernel/trace/bpf_trace.c:898 [inline]
       bpf_send_signal_thread+0x16/0x20 kernel/trace/bpf_trace.c:896
       bpf_prog_e6cf5f9c69743609+0x1d/0x46
       bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
       __bpf_prog_run include/linux/filter.h:691 [inline]
       bpf_prog_run include/linux/filter.h:698 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2403 [inline]
       bpf_trace_run4+0x334/0x590 kernel/trace/bpf_trace.c:2446
       trace_mmap_lock_acquire_returned include/trace/events/mmap_lock.h:52 [inline]
       __mmap_lock_do_trace_acquire_returned+0x5c8/0x630 mm/mmap_lock.c:237
       __mmap_lock_trace_acquire_returned include/linux/mmap_lock.h:36 [inline]
       mmap_read_trylock include/linux/mmap_lock.h:164 [inline]
       get_mmap_lock_carefully mm/memory.c:5715 [inline]
       lock_mm_and_find_vma+0x213/0x2f0 mm/memory.c:5775
       do_user_addr_fault arch/x86/mm/fault.c:1361 [inline]
       handle_page_fault arch/x86/mm/fault.c:1481 [inline]
       exc_page_fault+0x1bf/0x8c0 arch/x86/mm/fault.c:1539
       asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

-> #0 (lock#10){+.+.}-{2:2}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3869
       __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
       local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
       __mmap_lock_do_trace_start_locking+0x9c/0x620 mm/mmap_lock.c:230
       __mmap_lock_trace_start_locking include/linux/mmap_lock.h:29 [inline]
       mmap_read_trylock include/linux/mmap_lock.h:162 [inline]
       stack_map_get_build_id_offset+0x98a/0x9d0 kernel/bpf/stackmap.c:141
       __bpf_get_stack+0x4ad/0x5a0 kernel/bpf/stackmap.c:449
       bpf_prog_e6cf5f9c69743609+0x42/0x46
       bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
       __bpf_prog_run include/linux/filter.h:691 [inline]
       bpf_prog_run include/linux/filter.h:698 [inline]
       bpf_prog_run_array include/linux/bpf.h:2103 [inline]
       trace_call_bpf+0x369/0x8a0 kernel/trace/bpf_trace.c:147
       perf_trace_run_bpf_submit+0x7c/0x1d0 kernel/events/core.c:10269
       perf_trace_lock+0x388/0x490 include/trace/events/lock.h:50
       trace_lock_release include/trace/events/lock.h:69 [inline]
       lock_release+0x986/0x9f0 kernel/locking/lockdep.c:5765
       do_write_seqcount_end include/linux/seqlock.h:515 [inline]
       psi_group_change+0xace/0x12d0 kernel/sched/psi.c:864
       psi_task_switch+0x199/0x770 kernel/sched/psi.c:938
       psi_sched_switch kernel/sched/stats.h:185 [inline]
       __schedule+0x24ef/0x4a20 kernel/sched/core.c:6740
       preempt_schedule_notrace+0x100/0x140 kernel/sched/core.c:7017
       preempt_schedule_notrace_thunk+0x1a/0x30 arch/x86/entry/thunk.S:13
       trace_lock_release include/trace/events/lock.h:69 [inline]
       lock_release+0x9aa/0x9f0 kernel/locking/lockdep.c:5765
       __mutex_unlock_slowpath+0xe2/0x750 kernel/locking/mutex.c:912
       perf_event_ctx_unlock kernel/events/core.c:1299 [inline]
       perf_ioctl+0x229c/0x25f0 kernel/events/core.c:6021
       vfs_ioctl fs/ioctl.c:51 [inline]
       __do_sys_ioctl fs/ioctl.c:907 [inline]
       __se_sys_ioctl+0xfc/0x170 fs/ioctl.c:893
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

Chain exists of:
  lock#10 --> &rq->__lock --> &per_cpu_ptr(group->pcpu, cpu)->seq

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&per_cpu_ptr(group->pcpu, cpu)->seq);
                               lock(&rq->__lock);
                               lock(&per_cpu_ptr(group->pcpu, cpu)->seq);
  lock(lock#10);

 *** DEADLOCK ***

4 locks held by syz-executor.2/8056:
 #0: ffff888063305ca8 (&ctx->mutex){+.+.}-{3:3}, at: perf_event_ctx_lock_nested+0x416/0x590 kernel/events/core.c:1280
 #1: ffff8880b943e7d8 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x2a/0x140 kernel/sched/core.c:559
 #2: ffffe8ffffc64c88 (&per_cpu_ptr(group->pcpu, cpu)->seq){-.-.}-{0:0}, at: psi_task_switch+0x199/0x770 kernel/sched/psi.c:938
 #3: ffffffff8e333f60 (rcu_read_lock){....}-{1:2}, at: trace_call_bpf+0xbc/0x8a0

stack backtrace:
CPU: 0 PID: 8056 Comm: syz-executor.2 Not tainted 6.10.0-rc1-syzkaller-g7d0b3953f6d8 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3869
 __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
 local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
 __mmap_lock_do_trace_start_locking+0x9c/0x620 mm/mmap_lock.c:230
 __mmap_lock_trace_start_locking include/linux/mmap_lock.h:29 [inline]
 mmap_read_trylock include/linux/mmap_lock.h:162 [inline]
 stack_map_get_build_id_offset+0x98a/0x9d0 kernel/bpf/stackmap.c:141
 __bpf_get_stack+0x4ad/0x5a0 kernel/bpf/stackmap.c:449
 bpf_prog_e6cf5f9c69743609+0x42/0x46
 bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
 __bpf_prog_run include/linux/filter.h:691 [inline]
 bpf_prog_run include/linux/filter.h:698 [inline]
 bpf_prog_run_array include/linux/bpf.h:2103 [inline]
 trace_call_bpf+0x369/0x8a0 kernel/trace/bpf_trace.c:147
 perf_trace_run_bpf_submit+0x7c/0x1d0 kernel/events/core.c:10269
 perf_trace_lock+0x388/0x490 include/trace/events/lock.h:50
 trace_lock_release include/trace/events/lock.h:69 [inline]
 lock_release+0x986/0x9f0 kernel/locking/lockdep.c:5765
 do_write_seqcount_end include/linux/seqlock.h:515 [inline]
 psi_group_change+0xace/0x12d0 kernel/sched/psi.c:864
 psi_task_switch+0x199/0x770 kernel/sched/psi.c:938
 psi_sched_switch kernel/sched/stats.h:185 [inline]
 __schedule+0x24ef/0x4a20 kernel/sched/core.c:6740
 preempt_schedule_notrace+0x100/0x140 kernel/sched/core.c:7017
 preempt_schedule_notrace_thunk+0x1a/0x30 arch/x86/entry/thunk.S:13
 trace_lock_release include/trace/events/lock.h:69 [inline]
 lock_release+0x9aa/0x9f0 kernel/locking/lockdep.c:5765
 __mutex_unlock_slowpath+0xe2/0x750 kernel/locking/mutex.c:912
 perf_event_ctx_unlock kernel/events/core.c:1299 [inline]
 perf_ioctl+0x229c/0x25f0 kernel/events/core.c:6021
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:907 [inline]
 __se_sys_ioctl+0xfc/0x170 fs/ioctl.c:893
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f5ad5a7cee9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f5ad67d00c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f5ad5bb4070 RCX: 00007f5ad5a7cee9
RDX: 0000000000000009 RSI: 0000000040042408 RDI: 0000000000000003
RBP: 00007f5ad5ac947f R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000006e R14: 00007f5ad5bb4070 R15: 00007ffd3a14d418
 </TASK>

Crashes (7):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/06/03 01:53 bpf 7d0b3953f6d8 3113787f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/20 20:25 bpf 4b377b4868ef c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/04/22 02:30 bpf 443574b03387 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/08 03:40 bpf-next f85af9d955ac 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/22 04:00 bpf-next 6f130e4d4a5f 1014eca7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/20 10:18 bpf-next 6f130e4d4a5f c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/17 06:05 bpf-next 83eea61776c9 c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
* Struck through repros no longer work on HEAD.