syzbot

 sign-in | mailing list | source | docs
🐞 Open [984] ≡ Subsystems 🐞 Fixed [5216] 🐞 Invalid [12474] ⬇ Missing Backports [82] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in bpf

Status: auto-closed as invalid on 2021/09/25 21:47
Subsystems: bpf
[Documentation on labels]
First crash: 1143d, last: 1026d
▶ ▼ Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 INFO: rcu detected stall in bpf 2 1442d 1506d 0/1 auto-closed as invalid on 2020/09/05 11:39
linux-4.19 BUG: soft lockup in bpf 1 1088d 1088d 0/1 auto-closed as invalid on 2021/08/25 11:47

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=8de/1/0x4000000000000000 softirq=113496/113497 fqs=0 
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1):
	(detected by 0, t=10502 jiffies, g=211033, q=29)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9462 Comm: syz-executor.5 Not tainted 5.13.0-rc5-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:kasan_check_range+0x134/0x180 mm/kasan/generic.c:187
Code: 85 d2 75 0b 48 89 da 48 29 c2 e9 55 ff ff ff 49 39 d2 75 17 49 0f be 02 41 83 e1 07 49 39 c1 7d 0a 5b b8 01 00 00 00 5d 41 5c <c3> 44 89 c2 e8 73 f0 ff ff 5b 83 f0 01 5d 41 5c c3 48 29 c3 48 89
RSP: 0018:ffffc90000dd8d90 EFLAGS: 00000097
RAX: 0000000000000001 RBX: 1ffff920001bb1b4 RCX: ffffffff815a28a0
RDX: fffff520001bb1b8 RSI: 0000000000000004 RDI: ffffc90000dd8dc0
RBP: ffff8880b9d26380 R08: 0000000000000001 R09: 0000000000000003
R10: fffff520001bb1b8 R11: 0000000000000000 R12: ffff8880b9d26388
R13: ffff8880b9d26390 R14: ffff8880b9d26380 R15: 0000000000000001
FS:  00007fe9b787e700(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b33b2c000 CR3: 00000000170a2000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
 atomic_try_cmpxchg_acquire include/asm-generic/atomic-instrumented.h:706 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:82 [inline]
 do_raw_spin_lock+0x120/0x2b0 kernel/locking/spinlock_debug.c:113
 __run_hrtimer kernel/time/hrtimer.c:1541 [inline]
 __hrtimer_run_queues+0x243/0xe40 kernel/time/hrtimer.c:1601
 hrtimer_interrupt+0x330/0xa00 kernel/time/hrtimer.c:1663
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1089 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x540 arch/x86/kernel/apic/apic.c:1106
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1100
 </IRQ>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:647
RIP: 0010:kfree+0x133/0x2c0 mm/slab.c:3805
Code: 0e 4c 89 ea 48 89 ee 4c 89 e7 e8 48 f7 ff ff 48 85 db 0f 85 b3 00 00 00 9c 58 f6 c4 02 0f 85 4b 01 00 00 48 85 db 74 01 fb 5b <5d> 41 5c 41 5d c3 65 8b 05 a8 6c 49 7e 83 f8 07 0f 87 38 01 00 00
RSP: 0018:ffffc90001c27670 EFLAGS: 00000206
RAX: 0000000000000006 RBX: 0000000000000000 RCX: 1ffffffff20512e2
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff88806b0c8000 R08: 0000000000000001 R09: ffffffff9022c997
R10: 0000000000000001 R11: 0000000000084087 R12: ffff888011040800
R13: ffffffff81829db4 R14: ffff888033ec4000 R15: ffff88806b0c8000
 free_func_state kernel/bpf/verifier.c:875 [inline]
 free_func_state kernel/bpf/verifier.c:869 [inline]
 free_verifier_state+0xc4/0x250 kernel/bpf/verifier.c:891
 pop_stack+0x1c8/0x530 kernel/bpf/verifier.c:1000
 do_check kernel/bpf/verifier.c:10831 [inline]
 do_check_common+0x41d4/0xd540 kernel/bpf/verifier.c:12794
 do_check_main kernel/bpf/verifier.c:12860 [inline]
 bpf_check+0xa26d/0xd9f0 kernel/bpf/verifier.c:13419
 bpf_prog_load+0xe47/0x1f30 kernel/bpf/syscall.c:2217
 __do_sys_bpf+0x1272/0x4f40 kernel/bpf/syscall.c:4411
 do_syscall_64+0x3a/0xb0 arch/x86/entry/common.c:47
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x4665d9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fe9b787e188 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 000000000056bf80 RCX: 00000000004665d9
RDX: 0000000000000048 RSI: 00000000200017c0 RDI: 0000000000000005
RBP: 00000000004bfcb9 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056bf80
R13: 00007ffda577169f R14: 00007fe9b787e300 R15: 0000000000022000
task:systemd-udevd   state:R  running task     stack:23472 pid: 4854 ppid:     1 flags:0x00000000
Call Trace:
 context_switch kernel/sched/core.c:4339 [inline]
 __schedule+0x916/0x23e0 kernel/sched/core.c:5147
 preempt_schedule_irq+0x4e/0x90 kernel/sched/core.c:5535
 irqentry_exit+0x31/0x80 kernel/entry/common.c:427
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:647
RIP: 0010:lock_acquire+0x1ef/0x740 kernel/locking/lockdep.c:5480
Code: 3e a8 7e 83 f8 01 0f 85 ee 03 00 00 9c 58 f6 c4 02 0f 85 d9 03 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00 00 00 00 fc ff df <48> 01 c3 48 c7 03 00 00 00 00 48 c7 43 08 00 00 00 00 48 8b 84 24
RSP: 0018:ffffc90001617728 EFLAGS: 00000206
RAX: dffffc0000000000 RBX: 1ffff920002c2ee7 RCX: 00000000ef938e72
RDX: 1ffff110051221a6 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: ffffffff9022c8a7
R10: fffffbfff2045914 R11: 0000000000000001 R12: 0000000000000002
R13: ffffffff8bf76be0 R14: 0000000000000000 R15: 0000000000000000
 rcu_lock_acquire include/linux/rcupdate.h:267 [inline]
 rcu_read_lock include/linux/rcupdate.h:657 [inline]
 avc_has_perm_noaudit+0xcb/0x390 security/selinux/avc.c:1150
 selinux_inode_permission+0x3ff/0x670 security/selinux/hooks.c:3211
 security_inode_permission+0x92/0xf0 security/security.c:1312
 inode_permission.part.0+0x119/0x440 fs/namei.c:521
 inode_permission fs/namei.c:494 [inline]
 may_lookup fs/namei.c:1659 [inline]
 link_path_walk.part.0+0x800/0xc90 fs/namei.c:2211
 link_path_walk fs/namei.c:2193 [inline]
 path_lookupat+0xb7/0x850 fs/namei.c:2420
 filename_lookup+0x1b1/0x570 fs/namei.c:2454
 user_path_at include/linux/namei.h:60 [inline]
 vfs_statx+0x142/0x390 fs/stat.c:203
 vfs_fstatat fs/stat.c:225 [inline]
 vfs_lstat include/linux/fs.h:3384 [inline]
 __do_sys_newlstat+0x91/0x110 fs/stat.c:380
 do_syscall_64+0x3a/0xb0 arch/x86/entry/common.c:47
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fadf901e335
RSP: 002b:00007ffd3e8886b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000006
RAX: ffffffffffffffda RBX: 000055bf2d911270 RCX: 00007fadf901e335
RDX: 00007ffd3e8886f0 RSI: 00007ffd3e8886f0 RDI: 000055bf2d910270
RBP: 00007ffd3e8887b0 R08: 00007fadf92dd2a8 R09: 0000000000001010
R10: 00007fadf92dcb58 R11: 0000000000000246 R12: 000055bf2d910270
R13: 000055bf2d91027c R14: 000055bf2d8fba15 R15: 000055bf2d8fba1c
rcu: rcu_preempt kthread starved for 10502 jiffies! g211033 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:29032 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 context_switch kernel/sched/core.c:4339 [inline]
 __schedule+0x916/0x23e0 kernel/sched/core.c:5147
 schedule+0xcf/0x270 kernel/sched/core.c:5226
 schedule_timeout+0x14a/0x250 kernel/time/timer.c:1892
 rcu_gp_fqs_loop kernel/rcu/tree.c:2004 [inline]
 rcu_gp_kthread+0xd07/0x2300 kernel/rcu/tree.c:2177
 kthread+0x3b1/0x4a0 kernel/kthread.c:313
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 0
CPU: 0 PID: 9469 Comm: syz-executor.3 Not tainted 5.13.0-rc5-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0x141/0x1d7 lib/dump_stack.c:120
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_check_gp_kthread_starvation.cold+0x1cc/0x1d1 kernel/rcu/tree_stall.h:478
 print_other_cpu_stall kernel/rcu/tree_stall.h:583 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:708 [inline]
 rcu_pending kernel/rcu/tree.c:3911 [inline]
 rcu_sched_clock_irq+0x1d46/0x2080 kernel/rcu/tree.c:2649
 update_process_times+0x16d/0x200 kernel/time/timer.c:1796
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1374
 __run_hrtimer kernel/time/hrtimer.c:1537 [inline]
 __hrtimer_run_queues+0x1c0/0xe40 kernel/time/hrtimer.c:1601
 hrtimer_interrupt+0x330/0xa00 kernel/time/hrtimer.c:1663
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1089 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x540 arch/x86/kernel/apic/apic.c:1106
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1100
 </IRQ>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:647
RIP: 0010:__sanitizer_cov_trace_pc+0x4c/0x60 kernel/kcov.c:202
Code: 0e 85 c9 74 35 8b 82 3c 15 00 00 85 c0 74 2b 8b 82 18 15 00 00 83 f8 02 75 20 48 8b 8a 20 15 00 00 8b 92 1c 15 00 00 48 8b 01 <48> 83 c0 01 48 39 c2 76 07 48 89 34 c1 48 89 01 c3 0f 1f 00 41 55
RSP: 0018:ffffc90001787ca8 EFLAGS: 00000246
RAX: 000000000003ffff RBX: ffff8880b9d39760 RCX: ffffc9000f4d7000
RDX: 0000000000040000 RSI: ffffffff8168d5f0 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8168d616 R11: 0000000000000000 R12: ffffed10173a72ed
R13: 0000000000000001 R14: ffff8880b9d39768 R15: 0000000000000001
 rep_nop arch/x86/include/asm/vdso/processor.h:13 [inline]
 cpu_relax arch/x86/include/asm/vdso/processor.h:18 [inline]
 csd_lock_wait kernel/smp.c:440 [inline]
 smp_call_function_many_cond+0x450/0xc20 kernel/smp.c:967
 on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1133
 on_each_cpu include/linux/smp.h:71 [inline]
 clock_was_set+0x21/0x30 kernel/time/hrtimer.c:889
 do_settimeofday64 kernel/time/timekeeping.c:1327 [inline]
 do_settimeofday64+0x3dd/0x5c0 kernel/time/timekeeping.c:1293
 do_sys_settimeofday64 kernel/time/time.c:195 [inline]
 do_sys_settimeofday64+0x1de/0x260 kernel/time/time.c:169
 __do_sys_clock_settime kernel/time/posix-timers.c:1079 [inline]
 __se_sys_clock_settime kernel/time/posix-timers.c:1067 [inline]
 __x64_sys_clock_settime+0x1a1/0x280 kernel/time/posix-timers.c:1067
 do_syscall_64+0x3a/0xb0 arch/x86/entry/common.c:47
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x4665d9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fb89eca7188 EFLAGS: 00000246 ORIG_RAX: 00000000000000e3
RAX: ffffffffffffffda RBX: 000000000056bf80 RCX: 00000000004665d9
RDX: 0000000000000000 RSI: 0000000020000040 RDI: 0000000000000000
RBP: 00000000004bfcb9 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056bf80
R13: 00007fff27a45c7f R14: 00007fb89eca7300 R15: 0000000000022000

Crashes (13):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/06/13 05:03 upstream 8ecfa36cd4db 1ba81399 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in bpf
2021/04/16 09:16 upstream 7e25f40eab52 c59079a6 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in bpf
2021/06/27 21:41 bpf 3db6735f2ef4 9d2ab5df .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/06/27 04:21 bpf 3db6735f2ef4 9d2ab5df .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/06/23 07:27 bpf 7506d211b932 aba2b2fb .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/06/23 02:40 bpf 5dec6d96d12d aba2b2fb .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/06/18 17:08 bpf 61e8aeda9398 aba2b2fb .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/05/22 20:19 bpf 84316ca4e100 3c7fef33 .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/04/10 10:33 bpf afd0be729953 bfeda1b1 .config console log report info ci-upstream-bpf-kasan-gce INFO: rcu detected stall in bpf
2021/05/13 12:21 bpf-next 0303ce17347a ed7d41c5 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: rcu detected stall in bpf
2021/04/24 08:48 bpf-next 7d3c10770603 17f0b706 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: rcu detected stall in bpf
2021/03/05 15:17 bpf-next bce8623135fb 9d751681 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: rcu detected stall in bpf
2021/03/02 17:13 bpf-next 85e142cb42a1 92ead296 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: rcu detected stall in bpf
* Struck through repros no longer work on HEAD.