syzbot


possible deadlock in kvfree_call_rcu

Status: upstream: reported C repro on 2024/03/21 20:02
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+e5952d44ee3e1993cddf@syzkaller.appspotmail.com
First crash: 425d, last: 4d18h
Bug presence (3)
Date Name Commit Repro Result
2024/12/29 linux-6.1.y (ToT) 563edd786f0a C [report] possible deadlock in kvfree_call_rcu
2024/05/05 upstream (ToT) 7367539ad4b0 C [report] possible deadlock in kvfree_call_rcu
2024/12/29 upstream (ToT) 059dd502b263 C Didn't crash
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in kvfree_call_rcu (2) bpf C unreliable inconclusive 1097 165d 344d 0/28 upstream: reported C repro on 2024/06/10 22:50
upstream possible deadlock in kvfree_call_rcu bpf C 1431 350d 420d 25/28 fixed on 2024/06/05 13:52
linux-5.15 possible deadlock in kvfree_call_rcu origin:upstream missing-backport C done 891 3d22h 420d 0/3 upstream: reported C repro on 2024/03/25 23:20

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.1.138-syzkaller #0 Not tainted
------------------------------------------------------
kworker/u4:5/1120 is trying to acquire lock:
ffff8880b8f281d8 (krc.lock){....}-{2:2}, at: krc_this_cpu_lock kernel/rcu/tree.c:2991 [inline]
ffff8880b8f281d8 (krc.lock){....}-{2:2}, at: add_ptr_to_bulk_krc_lock kernel/rcu/tree.c:3311 [inline]
ffff8880b8f281d8 (krc.lock){....}-{2:2}, at: kvfree_call_rcu+0x184/0x870 kernel/rcu/tree.c:3402

but task is already holding lock:
ffff8880b8f28418 (&base->lock){-.-.}-{2:2}, at: __mod_timer+0x674/0xd20 kernel/time/timer.c:1096

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&base->lock){-.-.}-{2:2}:
       __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
       _raw_spin_lock_irqsave+0xa4/0xf0 kernel/locking/spinlock.c:162
       lock_timer_base+0x123/0x270 kernel/time/timer.c:999
       __mod_timer+0x117/0xd20 kernel/time/timer.c:1072
       queue_delayed_work_on+0x126/0x1e0 kernel/workqueue.c:1704
       queue_delayed_work include/linux/workqueue.h:527 [inline]
       schedule_delayed_monitor_work kernel/rcu/tree.c:3172 [inline]
       kvfree_call_rcu+0x4cb/0x870 kernel/rcu/tree.c:3428
       rtnl_register_internal+0x489/0x590 net/core/rtnetlink.c:260
       rtnl_register+0x2e/0x70 net/core/rtnetlink.c:310
       ip_rt_init+0x323/0x3b5 net/ipv4/route.c:3789
       ip_init+0xa/0x14 net/ipv4/ip_output.c:1770
       inet_init+0x2bd/0x3cf net/ipv4/af_inet.c:2033
       do_one_initcall+0x214/0x7a0 init/main.c:1298
       do_initcall_level+0x137/0x1e4 init/main.c:1371
       do_initcalls+0x4b/0x8a init/main.c:1387
       kernel_init_freeable+0x3fa/0x5ac init/main.c:1626
       kernel_init+0x19/0x1b0 init/main.c:1514
       ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

-> #0 (krc.lock){....}-{2:2}:
       check_prev_add kernel/locking/lockdep.c:3090 [inline]
       check_prevs_add kernel/locking/lockdep.c:3209 [inline]
       validate_chain kernel/locking/lockdep.c:3825 [inline]
       __lock_acquire+0x2cf8/0x7c50 kernel/locking/lockdep.c:5049
       lock_acquire+0x1b4/0x490 kernel/locking/lockdep.c:5662
       __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
       _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
       krc_this_cpu_lock kernel/rcu/tree.c:2991 [inline]
       add_ptr_to_bulk_krc_lock kernel/rcu/tree.c:3311 [inline]
       kvfree_call_rcu+0x184/0x870 kernel/rcu/tree.c:3402
       trie_delete_elem+0x52d/0x690 kernel/bpf/lpm_trie.c:545
       bpf_prog_2c29ac5cdc6b1842+0x3a/0x3e
       bpf_dispatcher_nop_func include/linux/bpf.h:1001 [inline]
       __bpf_prog_run include/linux/filter.h:603 [inline]
       bpf_prog_run include/linux/filter.h:610 [inline]
       __bpf_trace_run kernel/trace/bpf_trace.c:2285 [inline]
       bpf_trace_run3+0x1e3/0x400 kernel/trace/bpf_trace.c:2325
       trace_timer_start include/trace/events/timer.h:53 [inline]
       enqueue_timer+0x411/0x5c0 kernel/time/timer.c:609
       internal_add_timer kernel/time/timer.c:634 [inline]
       __mod_timer+0x8e1/0xd20 kernel/time/timer.c:1115
       queue_delayed_work_on+0x126/0x1e0 kernel/workqueue.c:1704
       queue_delayed_work include/linux/workqueue.h:527 [inline]
       toggle_allocation_gate+0x3d6/0x430 mm/kfence/core.c:821
       process_one_work+0x898/0x1160 kernel/workqueue.c:2292
       worker_thread+0xaa2/0x1250 kernel/workqueue.c:2439
       kthread+0x29d/0x330 kernel/kthread.c:376
       ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&base->lock);
                               lock(krc.lock);
                               lock(&base->lock);
  lock(krc.lock);

 *** DEADLOCK ***

4 locks held by kworker/u4:5/1120:
 #0: ffff888017479138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7a1/0x1160 kernel/workqueue.c:2267
 #1: ffffc9000470fd00 ((work_completion)(&(&kfence_timer)->work)){+.+.}-{0:0}, at: process_one_work+0x7a1/0x1160 kernel/workqueue.c:2267
 #2: ffff8880b8f28418 (&base->lock){-.-.}-{2:2}, at: __mod_timer+0x674/0xd20 kernel/time/timer.c:1096
 #3: ffffffff8c92a960 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #3: ffffffff8c92a960 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #3: ffffffff8c92a960 (rcu_read_lock){....}-{1:2}, at: __bpf_trace_run kernel/trace/bpf_trace.c:2284 [inline]
 #3: ffffffff8c92a960 (rcu_read_lock){....}-{1:2}, at: bpf_trace_run3+0xf0/0x400 kernel/trace/bpf_trace.c:2325

stack backtrace:
CPU: 1 PID: 1120 Comm: kworker/u4:5 Not tainted 6.1.138-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/19/2025
Workqueue: events_unbound toggle_allocation_gate
Call Trace:
 <TASK>
 dump_stack_lvl+0x168/0x22e lib/dump_stack.c:106
 check_noncircular+0x274/0x310 kernel/locking/lockdep.c:2170
 check_prev_add kernel/locking/lockdep.c:3090 [inline]
 check_prevs_add kernel/locking/lockdep.c:3209 [inline]
 validate_chain kernel/locking/lockdep.c:3825 [inline]
 __lock_acquire+0x2cf8/0x7c50 kernel/locking/lockdep.c:5049
 lock_acquire+0x1b4/0x490 kernel/locking/lockdep.c:5662
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 krc_this_cpu_lock kernel/rcu/tree.c:2991 [inline]
 add_ptr_to_bulk_krc_lock kernel/rcu/tree.c:3311 [inline]
 kvfree_call_rcu+0x184/0x870 kernel/rcu/tree.c:3402
 trie_delete_elem+0x52d/0x690 kernel/bpf/lpm_trie.c:545
 bpf_prog_2c29ac5cdc6b1842+0x3a/0x3e
 bpf_dispatcher_nop_func include/linux/bpf.h:1001 [inline]
 __bpf_prog_run include/linux/filter.h:603 [inline]
 bpf_prog_run include/linux/filter.h:610 [inline]
 __bpf_trace_run kernel/trace/bpf_trace.c:2285 [inline]
 bpf_trace_run3+0x1e3/0x400 kernel/trace/bpf_trace.c:2325
 trace_timer_start include/trace/events/timer.h:53 [inline]
 enqueue_timer+0x411/0x5c0 kernel/time/timer.c:609
 internal_add_timer kernel/time/timer.c:634 [inline]
 __mod_timer+0x8e1/0xd20 kernel/time/timer.c:1115
 queue_delayed_work_on+0x126/0x1e0 kernel/workqueue.c:1704
 queue_delayed_work include/linux/workqueue.h:527 [inline]
 toggle_allocation_gate+0x3d6/0x430 mm/kfence/core.c:821
 process_one_work+0x898/0x1160 kernel/workqueue.c:2292
 worker_thread+0xaa2/0x1250 kernel/workqueue.c:2439
 kthread+0x29d/0x330 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>

Crashes (795):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/05/16 05:07 linux-6.1.y 02b72ccb5f9d cfde8269 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/24 00:57 linux-6.1.y 420102835862 73a168d0 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/22 00:23 linux-6.1.y 3a8358583626 d34966d1 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/12/08 14:53 linux-6.1.y e4d90d63d385 9ac0fdc6 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/06/07 04:29 linux-6.1.y 88690811da69 121701b6 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/06/07 03:41 linux-6.1.y 88690811da69 121701b6 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/04/02 07:03 linux-6.1.y e5cd595e23c1 6baf5069 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/04/02 05:53 linux-6.1.y e5cd595e23c1 6baf5069 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/04/02 03:44 linux-6.1.y e5cd595e23c1 6baf5069 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2024/04/01 21:44 linux-6.1.y e5cd595e23c1 6baf5069 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/05/12 08:58 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/05/12 06:09 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/05/11 02:04 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/05/10 23:52 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/05/10 01:47 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/05/09 18:26 linux-6.1.y 02b72ccb5f9d 77908e5f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/05/09 05:25 linux-6.1.y ac7079a42ea5 bb813bcc .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/05/05 17:26 linux-6.1.y ac7079a42ea5 6ca47dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/28 10:47 linux-6.1.y 535ec20c5027 c6b4fb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/28 09:34 linux-6.1.y 535ec20c5027 c6b4fb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/27 05:33 linux-6.1.y 535ec20c5027 c6b4fb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/26 05:44 linux-6.1.y 535ec20c5027 c6b4fb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/26 00:20 linux-6.1.y 535ec20c5027 c6b4fb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/22 13:45 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/22 10:27 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/20 13:34 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/20 13:33 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/19 09:57 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/18 06:02 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/04/18 02:26 linux-6.1.y 420102835862 2a20f901 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/17 14:16 linux-6.1.y 420102835862 229db4cf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/04/07 05:45 linux-6.1.y 8e60a714ba3b 1c65791e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/31 07:25 linux-6.1.y 8e60a714ba3b d3999433 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/31 02:16 linux-6.1.y 8e60a714ba3b d3999433 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/29 08:48 linux-6.1.y 8e60a714ba3b cf25e2c2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/28 09:45 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/28 09:44 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/28 07:33 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/28 02:12 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/27 22:41 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/27 18:33 linux-6.1.y 344a09659766 6c09fb82 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/03/16 12:04 linux-6.1.y 344a09659766 e2826670 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/14 11:38 linux-6.1.y 344a09659766 e2826670 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/03/14 04:45 linux-6.1.y 344a09659766 e2826670 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/25 20:39 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/02/25 17:48 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/25 17:48 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/24 11:09 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/24 10:01 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/02/23 22:09 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/23 19:03 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2025/02/23 07:17 linux-6.1.y 3a8358583626 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/17 14:33 linux-6.1.y 0cbb5f65e52f 4121cf9d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf possible deadlock in kvfree_call_rcu
2025/02/17 02:47 linux-6.1.y 0cbb5f65e52f 40a34ec9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
2024/03/21 20:02 linux-6.1.y d7543167affd 7a239ce7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in kvfree_call_rcu
* Struck through repros no longer work on HEAD.