syzbot


INFO: rcu detected stall in sys_bpf (9)

Status: upstream: reported C repro on 2024/05/28 13:32
Subsystems: bpf
[Documentation on labels]
Reported-by: syzbot+4fe86fa6110c580ea1f5@syzkaller.appspotmail.com
First crash: 84d, last: 21h15m
Cause bisection: failed (error log, bisect log)
  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [bpf?] [net?] INFO: rcu detected stall in sys_bpf (9) 0 (1) 2024/05/28 13:32
Similar bugs (13)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_bpf (5) bpf C unreliable 56 885d 978d 0/27 closed as invalid on 2022/02/08 10:34
linux-5.15 INFO: rcu detected stall in sys_bpf (2) 5 8d09h 77d 0/3 upstream: reported on 2024/04/09 19:57
upstream INFO: rcu detected stall in sys_bpf (3) bpf 4 1630d 1630d 0/27 closed as invalid on 2020/01/09 08:13
linux-5.15 INFO: rcu detected stall in sys_bpf 2 201d 244d 0/3 auto-obsoleted due to no activity on 2024/03/16 17:33
upstream INFO: rcu detected stall in sys_bpf bpf net 3 1797d 1917d 0/27 auto-closed as invalid on 2019/11/23 00:18
upstream INFO: rcu detected stall in sys_bpf (2) bpf 12 1665d 1667d 0/27 closed as invalid on 2019/12/04 14:14
upstream INFO: rcu detected stall in sys_bpf (8) bpf 1 219d 219d 0/27 auto-obsoleted due to no activity on 2024/02/17 03:10
linux-6.1 INFO: rcu detected stall in sys_bpf 4 22h05m 66d 0/3 upstream: reported on 2024/04/20 10:27
linux-4.19 INFO: rcu detected stall in sys_bpf 3 1614d 1744d 0/1 auto-closed as invalid on 2020/05/23 14:47
upstream INFO: rcu detected stall in sys_bpf (6) net 7 696d 846d 0/27 auto-obsoleted due to no activity on 2022/10/28 06:32
upstream INFO: rcu detected stall in sys_bpf (4) bpf net 3 1583d 1622d 0/27 auto-closed as invalid on 2020/05/24 13:03
android-5-15 BUG: soft lockup in sys_bpf origin:lts C 96 1d17h 88d 0/2 upstream: reported C repro on 2024/03/29 12:25
android-5-10 BUG: soft lockup in sys_bpf 2 1d15h 5d20h 0/2 premoderation: reported on 2024/06/20 06:43

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P5156
rcu: 	(detected by 0, t=10503 jiffies, g=9261, q=651 ncpus=2)
task:syz-executor302 state:R  running task     stack:23696 pid:5156  tgid:5156  ppid:5104   flags:0x00004002
Call Trace:
 <IRQ>
 sched_show_task+0x50c/0x6d0 kernel/sched/core.c:9191
 rcu_print_detail_task_stall_rnp kernel/rcu/tree_stall.h:262 [inline]
 print_other_cpu_stall+0x122d/0x15e0 kernel/rcu/tree_stall.h:639
 check_cpu_stall kernel/rcu/tree_stall.h:799 [inline]
 rcu_pending kernel/rcu/tree.c:4309 [inline]
 rcu_sched_clock_irq+0x9f4/0x10a0 kernel/rcu/tree.c:2636
 update_process_times+0x1ce/0x230 kernel/time/timer.c:2485
 tick_sched_handle kernel/time/tick-sched.c:276 [inline]
 tick_nohz_handler+0x37c/0x500 kernel/time/tick-sched.c:297
 __run_hrtimer kernel/time/hrtimer.c:1687 [inline]
 __hrtimer_run_queues+0x55b/0xd50 kernel/time/hrtimer.c:1751
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1813
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline]
 __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1049
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:arch_atomic_read arch/x86/include/asm/atomic.h:23 [inline]
RIP: 0010:raw_atomic_read include/linux/atomic/atomic-arch-fallback.h:457 [inline]
RIP: 0010:rcu_dynticks_curr_cpu_in_eqs include/linux/context_tracking.h:122 [inline]
RIP: 0010:rcu_is_watching+0x5e/0xb0 kernel/rcu/tree.c:724
Code: 03 42 80 3c 38 00 74 08 4c 89 f7 e8 bc bb 7f 00 48 c7 c3 08 7d 03 00 49 03 1e 48 89 d8 48 c1 e8 03 42 0f b6 04 38 84 c0 75 22 <8b> 03 65 ff 0d 91 46 87 7e 74 10 83 e0 04 c1 e8 02 5b 41 5e 41 5f
RSP: 0018:ffffc900044070e8 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8880b9437d08 RCX: dffffc0000000000
RDX: ffff8880223cbc00 RSI: ffffffff8c1fe2e0 RDI: ffffffff8c1fe2a0
RBP: 0000000000000001 R08: ffffffff8100b50d R09: ffffffff814128cf
R10: 0000000000000003 R11: ffff8880223cbc00 R12: ffff8880223cbc00
R13: ffffffff81821c30 R14: ffffffff8ddbb9e0 R15: dffffc0000000000
 kernel_text_address+0x82/0xe0 kernel/extable.c:113
 __kernel_text_address+0xd/0x40 kernel/extable.c:79
 unwind_get_return_address+0x5d/0xc0 arch/x86/kernel/unwind_orc.c:369
 arch_stack_walk+0x125/0x1b0 arch/x86/kernel/stacktrace.c:26
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:312 [inline]
 __kasan_slab_alloc+0x66/0x80 mm/kasan/common.c:338
 kasan_slab_alloc include/linux/kasan.h:201 [inline]
 slab_post_alloc_hook mm/slub.c:3940 [inline]
 slab_alloc_node mm/slub.c:4000 [inline]
 kmem_cache_alloc_node_noprof+0x16b/0x320 mm/slub.c:4043
 kmalloc_reserve+0xa8/0x2a0 net/core/skbuff.c:575
 pskb_expand_head+0x202/0x1390 net/core/skbuff.c:2240
 __bpf_try_make_writable net/core/filter.c:1668 [inline]
 bpf_try_make_writable net/core/filter.c:1674 [inline]
 bpf_try_make_head_writable net/core/filter.c:1682 [inline]
 ____bpf_clone_redirect net/core/filter.c:2456 [inline]
 bpf_clone_redirect+0x119/0x370 net/core/filter.c:2434
 bpf_prog_de5959beb1c8948f+0x5a/0x5f
 bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
 __bpf_prog_run include/linux/filter.h:691 [inline]
 bpf_prog_run include/linux/filter.h:698 [inline]
 bpf_test_run+0x409/0x910 net/bpf/test_run.c:425
 bpf_prog_test_run_skb+0xafa/0x13a0 net/bpf/test_run.c:1066
 bpf_prog_test_run+0x33a/0x3b0 kernel/bpf/syscall.c:4291
 __sys_bpf+0x48d/0x810 kernel/bpf/syscall.c:5705
 __do_sys_bpf kernel/bpf/syscall.c:5794 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5792 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5792
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7ffa8dbee859
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 d1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffe8675d058 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007ffa8dbee859
RDX: 0000000000000050 RSI: 0000000020000080 RDI: 000000000000000a
RBP: 0000000000000000 R08: 0000000100000000 R09: 0000000100000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10553 jiffies! g9261 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:26512 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5408 [inline]
 __schedule+0x17e8/0x4a50 kernel/sched/core.c:6745
 __schedule_loop kernel/sched/core.c:6822 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6837
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2581
 rcu_gp_fqs_loop+0x2df/0x1370 kernel/rcu/tree.c:2000
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2202
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 5156 Comm: syz-executor302 Not tainted 6.9.0-next-20240521-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
RIP: 0010:arch_atomic_read arch/x86/include/asm/atomic.h:23 [inline]
RIP: 0010:raw_atomic_read include/linux/atomic/atomic-arch-fallback.h:457 [inline]
RIP: 0010:rcu_dynticks_curr_cpu_in_eqs include/linux/context_tracking.h:122 [inline]
RIP: 0010:rcu_is_watching+0x5e/0xb0 kernel/rcu/tree.c:724
Code: 03 42 80 3c 38 00 74 08 4c 89 f7 e8 bc bb 7f 00 48 c7 c3 08 7d 03 00 49 03 1e 48 89 d8 48 c1 e8 03 42 0f b6 04 38 84 c0 75 22 <8b> 03 65 ff 0d 91 46 87 7e 74 10 83 e0 04 c1 e8 02 5b 41 5e 41 5f
RSP: 0018:ffffc900044070e8 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8880b9437d08 RCX: dffffc0000000000
RDX: ffff8880223cbc00 RSI: ffffffff8c1fe2e0 RDI: ffffffff8c1fe2a0
RBP: 0000000000000001 R08: ffffffff8100b50d R09: ffffffff814128cf
R10: 0000000000000003 R11: ffff8880223cbc00 R12: ffff8880223cbc00
R13: ffffffff81821c30 R14: ffffffff8ddbb9e0 R15: dffffc0000000000
FS:  000055557e93c380(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffa8dc6b0f0 CR3: 000000007b0de000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 kernel_text_address+0x82/0xe0 kernel/extable.c:113
 __kernel_text_address+0xd/0x40 kernel/extable.c:79
 unwind_get_return_address+0x5d/0xc0 arch/x86/kernel/unwind_orc.c:369
 arch_stack_walk+0x125/0x1b0 arch/x86/kernel/stacktrace.c:26
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:312 [inline]
 __kasan_slab_alloc+0x66/0x80 mm/kasan/common.c:338
 kasan_slab_alloc include/linux/kasan.h:201 [inline]
 slab_post_alloc_hook mm/slub.c:3940 [inline]
 slab_alloc_node mm/slub.c:4000 [inline]
 kmem_cache_alloc_node_noprof+0x16b/0x320 mm/slub.c:4043
 kmalloc_reserve+0xa8/0x2a0 net/core/skbuff.c:575
 pskb_expand_head+0x202/0x1390 net/core/skbuff.c:2240
 __bpf_try_make_writable net/core/filter.c:1668 [inline]
 bpf_try_make_writable net/core/filter.c:1674 [inline]
 bpf_try_make_head_writable net/core/filter.c:1682 [inline]
 ____bpf_clone_redirect net/core/filter.c:2456 [inline]
 bpf_clone_redirect+0x119/0x370 net/core/filter.c:2434
 bpf_prog_de5959beb1c8948f+0x5a/0x5f
 bpf_dispatcher_nop_func include/linux/bpf.h:1243 [inline]
 __bpf_prog_run include/linux/filter.h:691 [inline]
 bpf_prog_run include/linux/filter.h:698 [inline]
 bpf_test_run+0x409/0x910 net/bpf/test_run.c:425
 bpf_prog_test_run_skb+0xafa/0x13a0 net/bpf/test_run.c:1066
 bpf_prog_test_run+0x33a/0x3b0 kernel/bpf/syscall.c:4291
 __sys_bpf+0x48d/0x810 kernel/bpf/syscall.c:5705
 __do_sys_bpf kernel/bpf/syscall.c:5794 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5792 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5792
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf5/0x240 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7ffa8dbee859
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 d1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffe8675d058 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007ffa8dbee859
RDX: 0000000000000050 RSI: 0000000020000080 RDI: 000000000000000a
RBP: 0000000000000000 R08: 0000000100000000 R09: 0000000100000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>
sched: RT throttling activated

Crashes (17):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/24 13:20 linux-next 124cfbcd6d18 8f98448e .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/06/23 03:07 upstream 5f583a3162ff edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_bpf
2024/06/21 08:44 upstream 50736169ecc8 dac2aa43 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_bpf
2024/06/15 21:59 upstream 2ccbdf43d5e7 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_bpf
2024/06/15 15:52 upstream 2ccbdf43d5e7 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_bpf
2024/06/13 19:58 upstream 2ccbdf43d5e7 a9616ff5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/06/10 01:27 upstream 771ed66105de 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/06/05 11:28 upstream 32f88d65f01b e1e2c66e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_bpf
2024/05/19 18:47 upstream 0450d2083be6 c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_bpf
2024/05/08 15:07 upstream dccb07f2914c 4cf3f9b3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/05/07 03:41 upstream ee5b455b0ada c035c6de .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_bpf
2024/04/28 15:46 upstream 2c8159388952 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_bpf
2024/04/02 06:32 upstream fe46a7dd189e 6baf5069 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/06/25 05:39 linux-next 62c97045b8f7 215eef4a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/06/06 11:30 linux-next ee78a17615ad 121701b6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_bpf
2024/05/24 16:38 bpf 4b377b4868ef 8f98448e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in sys_bpf
2024/05/12 06:20 bpf 3e9bc0472b91 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in sys_bpf
* Struck through repros no longer work on HEAD.