syzbot


possible deadlock in __mmap_lock_do_trace_start_locking

Status: fixed on 2024/08/28 15:06
Subsystems: mm
[Documentation on labels]
Reported-by: syzbot+6ff90931779bcdfc840c@syzkaller.appspotmail.com
Fix commit: 7d6be67cfdd4 mm: mmap_lock: replace get_memcg_path_buf() with on-stack buffer
First crash: 177d, last: 79d
Cause bisection: introduced by (bisect log) :
commit 21c38a3bd4ee3fb7337d013a638302fb5e5f9dc2
Author: Jesper Dangaard Brouer <hawk@kernel.org>
Date: Wed May 1 14:04:11 2024 +0000

  cgroup/rstat: add cgroup_rstat_cpu_lock helpers and tracepoints

Crash: inconsistent lock state in __mmap_lock_do_trace_acquire_returned (log)
Repro: C syz .config
  
Fix bisection: fixed by (bisect log) :
commit 7d6be67cfdd4a53cea7147313ca13c531e3a470f
Author: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Date: Fri Jun 21 01:08:41 2024 +0000

  mm: mmap_lock: replace get_memcg_path_buf() with on-stack buffer

  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [mm?] possible deadlock in __mmap_lock_do_trace_start_locking 2 (6) 2024/08/28 03:45
Last patch testing requests (1)
Created Duration User Patch Repo Result
2024/08/11 18:49 23m retest repro bpf OK log

Sample crash report:
============================================
WARNING: possible recursive locking detected
6.10.0-rc2-syzkaller-00242-g36534d3c5453 #0 Not tainted
--------------------------------------------
syz-executor181/5090 is trying to acquire lock:
ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: __mmap_lock_do_trace_start_locking+0x83/0x620 mm/mmap_lock.c:230

but task is already holding lock:
ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: __mmap_lock_do_trace_start_locking+0x83/0x620 mm/mmap_lock.c:230

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(lock#9);
  lock(lock#9);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

3 locks held by syz-executor181/5090:
 #0: ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
 #0: ffff8880b9538828 (lock#9){+.+.}-{2:2}, at: __mmap_lock_do_trace_start_locking+0x83/0x620 mm/mmap_lock.c:230
 #1: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:329 [inline]
 #1: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:781 [inline]
 #1: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: get_mem_cgroup_from_mm+0x38/0x2a0 mm/memcontrol.c:1265
 #2: ffffffff8e333fa0 (rcu_read_lock){....}-{1:2}, at: trace_call_bpf+0xbc/0x8a0

stack backtrace:
CPU: 1 PID: 5090 Comm: syz-executor181 Not tainted 6.10.0-rc2-syzkaller-00242-g36534d3c5453 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 check_deadlock kernel/locking/lockdep.c:3062 [inline]
 validate_chain+0x15d3/0x5900 kernel/locking/lockdep.c:3856
 __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5137
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5754
 local_lock_acquire include/linux/local_lock_internal.h:29 [inline]
 __mmap_lock_do_trace_start_locking+0x9c/0x620 mm/mmap_lock.c:230
 __mmap_lock_trace_start_locking include/linux/mmap_lock.h:29 [inline]
 mmap_read_trylock include/linux/mmap_lock.h:162 [inline]
 stack_map_get_build_id_offset+0x98a/0x9d0 kernel/bpf/stackmap.c:141
 __bpf_get_stack+0x4ad/0x5a0 kernel/bpf/stackmap.c:449
 bpf_prog_e6cf5f9c69743609+0x42/0x46
 bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
 __bpf_prog_run include/linux/filter.h:691 [inline]
 bpf_prog_run include/linux/filter.h:698 [inline]
 bpf_prog_run_array include/linux/bpf.h:2103 [inline]
 trace_call_bpf+0x369/0x8a0 kernel/trace/bpf_trace.c:147
 perf_trace_run_bpf_submit+0x7c/0x1d0 kernel/events/core.c:10269
 perf_trace_lock+0x388/0x490 include/trace/events/lock.h:50
 trace_lock_release include/trace/events/lock.h:69 [inline]
 lock_release+0x986/0x9f0 kernel/locking/lockdep.c:5765
 rcu_lock_release include/linux/rcupdate.h:339 [inline]
 rcu_read_unlock include/linux/rcupdate.h:812 [inline]
 get_mem_cgroup_from_mm+0x1ad/0x2a0 mm/memcontrol.c:1271
 get_mm_memcg_path+0x1b/0x600 mm/mmap_lock.c:202
 __mmap_lock_do_trace_start_locking+0x134/0x620 mm/mmap_lock.c:230
 __mmap_lock_trace_start_locking include/linux/mmap_lock.h:29 [inline]
 mmap_read_lock include/linux/mmap_lock.h:143 [inline]
 acct_collect+0x7e7/0x830 kernel/acct.c:563
 do_exit+0x936/0x27e0 kernel/exit.c:853
 do_group_exit+0x207/0x2c0 kernel/exit.c:1023
 __do_sys_exit_group kernel/exit.c:1034 [inline]
 __se_sys_exit_group kernel/exit.c:1032 [inline]
 __x64_sys_exit_group+0x3f/0x40 kernel/exit.c:1032
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f9c47742279
Code: 90 49 c7 c0 b8 ff ff ff be e7 00 00 00 ba 3c 00 00 00 eb 12 0f 1f 44 00 00 89 d0 0f 05 48 3d 00 f0 ff ff 77 1c f4 89 f0 0f 05 <48> 3d 00 f0 ff ff 76 e7 f7 d8 64 41 89 00 eb df 0f 1f 80 00 00 00
RSP: 002b:00007ffd321ae558 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9c47742279
RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
RBP: 00007f9c477bd2b0 R08: ffffffffffffffb8 R09: 00000000000000a0
R10: 00000000000000a0 R11: 0000000000000246 R12: 00007f9c477bd2b0
R13: 0000000000000000 R14: 00007f9c477bdd20 R15: 00007f9c47713400
 </TASK>

Crashes (30):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/06/16 05:53 bpf 36534d3c5453 f429ab00 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/25 21:49 bpf fa5ef655615a 32fcf98f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/08 21:42 bpf 83c36e7cfd74 cde64f7d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/28 17:56 bpf dc6be0b73f4f b62c7d46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/03 01:53 bpf 7d0b3953f6d8 3113787f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/20 20:25 bpf 4b377b4868ef c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/04/22 02:30 bpf 443574b03387 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/28 18:21 bpf-next 7d30b8aa4fc3 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/28 16:54 bpf-next 7d30b8aa4fc3 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/27 16:32 bpf-next 7d30b8aa4fc3 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/25 22:26 bpf-next 6189fa6faa09 32fcf98f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/25 10:22 bpf-next 7244100e0389 b24754ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/23 06:46 bpf-next 229d6db14942 708c10c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/23 06:45 bpf-next 229d6db14942 708c10c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/18 04:13 bpf-next 51835949dda3 03114f55 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/16 16:33 bpf-next 69cf87304dcb b66b37bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/16 15:23 bpf-next 69cf87304dcb b66b37bd .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/11 23:08 bpf-next 19d3c179a377 c699c2eb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/11 09:30 bpf-next 18a8a4c88fb4 c699c2eb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/10 01:42 bpf-next 7b769adc2612 79d68ada .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/10 01:41 bpf-next 7b769adc2612 79d68ada .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/07 17:03 bpf-next fd8db07705c5 bc4ebbb5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/07 10:47 bpf-next fd8db07705c5 bc4ebbb5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/07/07 04:39 bpf-next fd8db07705c5 bc4ebbb5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/28 18:29 bpf-next a12978712d90 b62c7d46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/21 12:47 bpf-next bf977ee4a9e2 dac2aa43 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/06/08 03:40 bpf-next f85af9d955ac 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/22 04:00 bpf-next 6f130e4d4a5f 1014eca7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/20 10:18 bpf-next 6f130e4d4a5f c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
2024/05/17 06:05 bpf-next 83eea61776c9 c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce possible deadlock in __mmap_lock_do_trace_start_locking
* Struck through repros no longer work on HEAD.