syzbot


possible deadlock in sock_map_delete_elem

Status: upstream: reported C repro on 2024/04/06 16:22
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+3a838a382d1ea3f3a4d7@syzkaller.appspotmail.com
First crash: 27d, last: 1h33m
Bug presence (1)
Date Name Commit Repro Result
2024/04/28 upstream (ToT) 2c8159388952 C [report] possible deadlock in sock_map_delete_elem
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in sock_map_delete_elem bpf net C 290 1h48m 46d 0/26 upstream: reported C repro on 2024/03/18 16:03
linux-5.15 possible deadlock in sock_map_delete_elem origin:upstream C 40 21h10m 43d 0/3 upstream: reported C repro on 2024/03/21 17:46

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.1.89-syzkaller #0 Not tainted
------------------------------------------------------
kworker/u4:1/11 is trying to acquire lock:
ffff88807dccca40 (&stab->lock){+.-.}-{2:2}, at: __sock_map_delete net/core/sock_map.c:419 [inline]
ffff88807dccca40 (&stab->lock){+.-.}-{2:2}, at: sock_map_delete_elem+0x161/0x230 net/core/sock_map.c:451

but task is already holding lock:
ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:356 [inline]
ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: sock_map_del_link net/core/sock_map.c:147 [inline]
ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: sock_map_unref+0xcc/0x5d0 net/core/sock_map.c:182

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&psock->link_lock){+...}-{2:2}:
       lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
       __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
       _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
       spin_lock_bh include/linux/spinlock.h:356 [inline]
       sock_map_add_link net/core/sock_map.c:136 [inline]
       sock_map_update_common+0x31c/0x5b0 net/core/sock_map.c:505
       sock_map_update_elem_sys+0x55b/0x910 net/core/sock_map.c:584
       map_update_elem+0x503/0x680 kernel/bpf/syscall.c:1448
       __sys_bpf+0x337/0x6c0 kernel/bpf/syscall.c:4993
       __do_sys_bpf kernel/bpf/syscall.c:5109 [inline]
       __se_sys_bpf kernel/bpf/syscall.c:5107 [inline]
       __x64_sys_bpf+0x78/0x90 kernel/bpf/syscall.c:5107
       do_syscall_x64 arch/x86/entry/common.c:51 [inline]
       do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
       entry_SYSCALL_64_after_hwframe+0x68/0xd2

-> #0 (&stab->lock){+.-.}-{2:2}:
       check_prev_add kernel/locking/lockdep.c:3090 [inline]
       check_prevs_add kernel/locking/lockdep.c:3209 [inline]
       validate_chain+0x1661/0x5950 kernel/locking/lockdep.c:3825
       __lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5049
       lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
       __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
       _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
       __sock_map_delete net/core/sock_map.c:419 [inline]
       sock_map_delete_elem+0x161/0x230 net/core/sock_map.c:451
       bpf_prog_25adc7951638c19b+0x3b/0x45
       bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
       __bpf_prog_run include/linux/filter.h:603 [inline]
       bpf_prog_run include/linux/filter.h:610 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2273 [inline]
       bpf_trace_run2+0x1fd/0x410 kernel/trace/bpf_trace.c:2312
       __bpf_trace_kfree+0x6e/0x90 include/trace/events/kmem.h:94
       trace_kfree include/trace/events/kmem.h:94 [inline]
       kfree+0xf6/0x190 mm/slab_common.c:994
       sk_psock_free_link include/linux/skmsg.h:416 [inline]
       sock_map_del_link net/core/sock_map.c:160 [inline]
       sock_map_unref+0x3ac/0x5d0 net/core/sock_map.c:182
       sock_map_free+0x1e3/0x3d0 net/core/sock_map.c:353
       process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
       worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
       kthread+0x28d/0x320 kernel/kthread.c:376
       ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&psock->link_lock);
                               lock(&stab->lock);
                               lock(&psock->link_lock);
  lock(&stab->lock);

 *** DEADLOCK ***

6 locks held by kworker/u4:1/11:
 #0: ffff888012479138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7a9/0x11d0 kernel/workqueue.c:2267
 #1: ffffc90000107d20 ((work_completion)(&map->work)){+.+.}-{0:0}, at: process_one_work+0x7a9/0x11d0 kernel/workqueue.c:2267
 #2: ffff88807cf55130 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1745 [inline]
 #2: ffff88807cf55130 (sk_lock-AF_UNIX){+.+.}-{0:0}, at: sock_map_free+0x11a/0x3d0 net/core/sock_map.c:351
 #3: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #3: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #3: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: sock_map_free+0x126/0x3d0 net/core/sock_map.c:352
 #4: ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:356 [inline]
 #4: ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: sock_map_del_link net/core/sock_map.c:147 [inline]
 #4: ffff888072b9c290 (&psock->link_lock){+...}-{2:2}, at: sock_map_unref+0xcc/0x5d0 net/core/sock_map.c:182
 #5: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #5: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #5: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: __bpf_trace_run kernel/trace/bpf_trace.c:2272 [inline]
 #5: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: bpf_trace_run2+0x110/0x410 kernel/trace/bpf_trace.c:2312

stack backtrace:
CPU: 0 PID: 11 Comm: kworker/u4:1 Not tainted 6.1.89-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Workqueue: events_unbound bpf_map_free_deferred
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 check_noncircular+0x2fa/0x3b0 kernel/locking/lockdep.c:2170
 check_prev_add kernel/locking/lockdep.c:3090 [inline]
 check_prevs_add kernel/locking/lockdep.c:3209 [inline]
 validate_chain+0x1661/0x5950 kernel/locking/lockdep.c:3825
 __lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5049
 lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
 _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
 __sock_map_delete net/core/sock_map.c:419 [inline]
 sock_map_delete_elem+0x161/0x230 net/core/sock_map.c:451
 bpf_prog_25adc7951638c19b+0x3b/0x45
 bpf_dispatcher_nop_func include/linux/bpf.h:989 [inline]
 __bpf_prog_run include/linux/filter.h:603 [inline]
 bpf_prog_run include/linux/filter.h:610 [inline]
 __bpf_trace_run kernel/trace/bpf_trace.c:2273 [inline]
 bpf_trace_run2+0x1fd/0x410 kernel/trace/bpf_trace.c:2312
 __bpf_trace_kfree+0x6e/0x90 include/trace/events/kmem.h:94
 trace_kfree include/trace/events/kmem.h:94 [inline]
 kfree+0xf6/0x190 mm/slab_common.c:994
 sk_psock_free_link include/linux/skmsg.h:416 [inline]
 sock_map_del_link net/core/sock_map.c:160 [inline]
 sock_map_unref+0x3ac/0x5d0 net/core/sock_map.c:182
 sock_map_free+0x1e3/0x3d0 net/core/sock_map.c:353
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (43):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/04/30 15:24 linux-6.1.y dcbc050cb0d3 3ce4924c .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/24 02:55 linux-6.1.y 6741e066ec76 21339d7b .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/22 18:10 linux-6.1.y 6741e066ec76 36c961ad .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/22 15:41 linux-6.1.y 6741e066ec76 36c961ad .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/22 15:04 linux-6.1.y 6741e066ec76 36c961ad .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/16 07:03 linux-6.1.y cd5d98c0556c 0d592ce4 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/13 09:33 linux-6.1.y bf1e3b1cb1e0 c8349e48 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/13 03:44 linux-6.1.y bf1e3b1cb1e0 c8349e48 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/12 05:00 linux-6.1.y bf1e3b1cb1e0 27de0a5c .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/10 22:18 linux-6.1.y bf1e3b1cb1e0 4320ec32 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/05/03 21:54 linux-6.1.y 909ba1f1b414 dd26401e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/05/03 13:28 linux-6.1.y 909ba1f1b414 dd26401e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/30 13:04 linux-6.1.y dcbc050cb0d3 3ce4924c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/29 11:45 linux-6.1.y dcbc050cb0d3 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/26 20:32 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/26 02:26 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/25 18:28 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/25 00:00 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/24 17:02 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/23 10:40 linux-6.1.y 6741e066ec76 21339d7b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/23 08:45 linux-6.1.y 6741e066ec76 21339d7b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/23 05:42 linux-6.1.y 6741e066ec76 36c961ad .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/22 01:13 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/21 19:25 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/21 16:57 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/21 16:49 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/21 16:49 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/21 14:31 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/21 12:13 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/19 23:52 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/18 08:52 linux-6.1.y 6741e066ec76 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/16 17:03 linux-6.1.y cd5d98c0556c 18f6e127 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/16 09:10 linux-6.1.y cd5d98c0556c 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/16 09:05 linux-6.1.y cd5d98c0556c 0d592ce4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/15 15:53 linux-6.1.y cd5d98c0556c b9af7e61 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in sock_map_delete_elem
2024/04/15 09:51 linux-6.1.y cd5d98c0556c c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/14 17:51 linux-6.1.y cd5d98c0556c c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/14 16:01 linux-6.1.y cd5d98c0556c c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/14 12:04 linux-6.1.y cd5d98c0556c c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/13 10:15 linux-6.1.y bf1e3b1cb1e0 c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/09 21:00 linux-6.1.y 347385861c50 171ec371 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
2024/04/06 16:21 linux-6.1.y 347385861c50 ca620dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in sock_map_delete_elem
* Struck through repros no longer work on HEAD.