syzbot


possible deadlock in sock_map_update_common

Status: upstream: reported C repro on 2024/04/13 07:12
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+f64311c8c313b19f4b96@syzkaller.appspotmail.com
First crash: 252d, last: 188d
Fix commit to backport (bisect log) :
tree: upstream
commit 98e948fb60d41447fd8d2d0c3b8637fc6b6dc26d
Author: Jakub Sitnicki <jakub@cloudflare.com>
Date: Mon May 27 11:20:07 2024 +0000

  bpf: Allow delete from sockmap/sockhash only if update is allowed

  
Bug presence (3)
Date Name Commit Repro Result
2024/06/12 linux-5.15.y (ToT) c61bd26ae81a C [report] possible deadlock in sock_map_update_common
2024/04/28 upstream (ToT) 2c8159388952 C [report] possible deadlock in sock_map_update_common
2024/06/12 upstream (ToT) 2ef5971ff345 C Didn't crash
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in sock_map_update_common (2) bpf net C done unreliable 12 196d 207d 0/28 auto-obsoleted due to no activity on 2024/08/31 16:12
linux-6.1 possible deadlock in sock_map_update_common origin:upstream C error 11 164d 266d 0/3 upstream: reported C repro on 2024/03/29 17:11
Last patch testing requests (2)
Created Duration User Patch Repo Result
2024/10/15 00:00 0m retest repro linux-5.15.y error
2024/10/15 00:00 0m retest repro linux-5.15.y error
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/06/13 03:43 4h00m fix candidate upstream OK (1) job log

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
5.15.158-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor232/3498 is trying to acquire lock:
ffff88807473a290 (&psock->link_lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:368 [inline]
ffff88807473a290 (&psock->link_lock){+...}-{2:2}, at: sock_map_add_link net/core/sock_map.c:136 [inline]
ffff88807473a290 (&psock->link_lock){+...}-{2:2}, at: sock_map_update_common+0x31c/0x5b0 net/core/sock_map.c:505

but task is already holding lock:
ffff88801f6575c0 (&stab->lock){+...}-{2:2}, at: sock_map_update_common+0x1b6/0x5b0 net/core/sock_map.c:495

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&stab->lock){+...}-{2:2}:
       lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
       __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
       _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
       __sock_map_delete net/core/sock_map.c:419 [inline]
       sock_map_delete_elem+0x161/0x230 net/core/sock_map.c:451
       bpf_prog_330576d32f01b34b+0x5b/0x718
       bpf_dispatcher_nop_func include/linux/bpf.h:790 [inline]
       __bpf_prog_run include/linux/filter.h:628 [inline]
       bpf_prog_run include/linux/filter.h:635 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:1880 [inline]
       bpf_trace_run2+0x19e/0x340 kernel/trace/bpf_trace.c:1917
       __bpf_trace_kfree+0x6e/0x90 include/trace/events/kmem.h:118
       trace_kfree include/trace/events/kmem.h:118 [inline]
       kfree+0x22f/0x270 mm/slub.c:4549
       sk_psock_free_link include/linux/skmsg.h:422 [inline]
       sock_map_del_link net/core/sock_map.c:160 [inline]
       sock_map_unref+0x3ac/0x5d0 net/core/sock_map.c:182
       sock_hash_update_common+0x911/0xa30 net/core/sock_map.c:1034
       sock_map_update_elem_sys+0x485/0x770 net/core/sock_map.c:590
       map_update_elem+0x6a0/0x7c0 kernel/bpf/syscall.c:1185
       __sys_bpf+0x2fd/0x670 kernel/bpf/syscall.c:4639
       __do_sys_bpf kernel/bpf/syscall.c:4755 [inline]
       __se_sys_bpf kernel/bpf/syscall.c:4753 [inline]
       __x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:4753
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> #0 (&psock->link_lock){+...}-{2:2}:
       check_prev_add kernel/locking/lockdep.c:3053 [inline]
       check_prevs_add kernel/locking/lockdep.c:3172 [inline]
       validate_chain+0x1649/0x5930 kernel/locking/lockdep.c:3788
       __lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5012
       lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
       __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
       _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
       spin_lock_bh include/linux/spinlock.h:368 [inline]
       sock_map_add_link net/core/sock_map.c:136 [inline]
       sock_map_update_common+0x31c/0x5b0 net/core/sock_map.c:505
       sock_map_update_elem_sys+0x440/0x770 net/core/sock_map.c:588
       map_update_elem+0x6a0/0x7c0 kernel/bpf/syscall.c:1185
       __sys_bpf+0x2fd/0x670 kernel/bpf/syscall.c:4639
       __do_sys_bpf kernel/bpf/syscall.c:4755 [inline]
       __se_sys_bpf kernel/bpf/syscall.c:4753 [inline]
       __x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:4753
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x66/0xd0

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&stab->lock);
                               lock(&psock->link_lock);
                               lock(&stab->lock);
  lock(&psock->link_lock);

 *** DEADLOCK ***

3 locks held by syz-executor232/3498:
 #0: ffff88801da61920 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff88801da61920 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: sock_map_sk_acquire net/core/sock_map.c:119 [inline]
 #0: ffff88801da61920 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: sock_map_update_elem_sys+0x1c8/0x770 net/core/sock_map.c:584
 #1: ffffffff8c91fae0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x5/0x30 include/linux/rcupdate.h:311
 #2: ffff88801f6575c0 (&stab->lock){+...}-{2:2}, at: sock_map_update_common+0x1b6/0x5b0 net/core/sock_map.c:495

stack backtrace:
CPU: 1 PID: 3498 Comm: syz-executor232 Not tainted 5.15.158-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 check_noncircular+0x2f8/0x3b0 kernel/locking/lockdep.c:2133
 check_prev_add kernel/locking/lockdep.c:3053 [inline]
 check_prevs_add kernel/locking/lockdep.c:3172 [inline]
 validate_chain+0x1649/0x5930 kernel/locking/lockdep.c:3788
 __lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5012
 lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
 _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
 spin_lock_bh include/linux/spinlock.h:368 [inline]
 sock_map_add_link net/core/sock_map.c:136 [inline]
 sock_map_update_common+0x31c/0x5b0 net/core/sock_map.c:505
 sock_map_update_elem_sys+0x440/0x770 net/core/sock_map.c:588
 map_update_elem+0x6a0/0x7c0 kernel/bpf/syscall.c:1185
 __sys_bpf+0x2fd/0x670 kernel/bpf/syscall.c:4639
 __do_sys_bpf kernel/bpf/syscall.c:4755 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:4753 [inline]
 __x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:4753
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7f9a3d891b69
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffc3226fa88 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffc3226fc58 RCX: 00007f9a3d891b69
RDX: 0000000000000020 RSI: 00000000200000c0 RDI: 0000000000000002
RBP: 00007f9a3d905610 R08: 00007ffc3226fc58 R09: 000

Crashes (12):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/14 09:42 linux-5.15.y 284087d4f7d5 fdb4c10c .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/04/13 07:11 linux-5.15.y cdfd0a7f0139 c8349e48 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/06/16 00:20 linux-5.15.y c61bd26ae81a f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/06/15 21:17 linux-5.15.y c61bd26ae81a f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/06/15 19:14 linux-5.15.y c61bd26ae81a f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/06/14 18:23 linux-5.15.y c61bd26ae81a 8d849073 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/06/14 11:16 linux-5.15.y c61bd26ae81a 8d849073 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/05/24 19:22 linux-5.15.y 83655231580b a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/05/24 19:22 linux-5.15.y 83655231580b a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/05/15 09:27 linux-5.15.y 284087d4f7d5 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
2024/05/04 11:54 linux-5.15.y 284087d4f7d5 610f2a54 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in sock_map_update_common
2024/04/21 20:43 linux-5.15.y c52b9710c83d af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf possible deadlock in sock_map_update_common
* Struck through repros no longer work on HEAD.