syzbot


INFO: rcu detected stall in newfstatat

Status: closed as invalid on 2022/02/08 09:50
Subsystems: fs
[Documentation on labels]
First crash: 850d, last: 834d
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in newfstatat (2) kernfs 2 705d 739d 0/26 auto-closed as invalid on 2022/08/20 09:41
upstream INFO: rcu detected stall in newfstatat (3) usb C done 31 9h11m 321d 0/26 upstream: reported C repro on 2023/06/10 09:07

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=503/1/0x4000000000000000 softirq=96741/96742 fqs=56 
	(detected by 0, t=10502 jiffies, g=143985, q=614)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 2973 Comm: udevd Not tainted 5.16.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:preempt_count_add+0x52/0x140 kernel/sched/core.c:5450
Code: 7c 08 84 d2 0f 85 e1 00 00 00 8b 15 c8 9e 00 0f 65 01 1d 01 95 b2 7e 85 d2 75 11 65 8b 05 f6 94 b2 7e 0f b6 c0 3d f4 00 00 00 <7f> 64 65 8b 05 e5 94 b2 7e 25 ff ff ff 7f 39 c3 74 03 5b 5d c3 48
RSP: 0018:ffffc90000dc0cf8 EFLAGS: 00000097
RAX: 0000000000000003 RBX: 0000000000000001 RCX: 1ffffffff20a0f38
RDX: 0000000000000000 RSI: ffffffff89ae0f60 RDI: 0000000000000001
RBP: ffffffff906d84a0 R08: ffffffff906d84a0 R09: 0000000000000000
R10: ffffffff8167066d R11: 0000000000000000 R12: 0000000000000016
R13: dffffc0000000000 R14: 1ffff920001b81a7 R15: ffffffff89ae0f60
FS:  00007f2a3d12b840(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdd773f7fc8 CR3: 0000000021a0e000 CR4: 0000000000350ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:109 [inline]
 _raw_spin_lock_irqsave+0x1d/0x50 kernel/locking/spinlock.c:162
 debug_object_activate+0x12e/0x3e0 lib/debugobjects.c:661
 debug_hrtimer_activate kernel/time/hrtimer.c:420 [inline]
 debug_activate kernel/time/hrtimer.c:475 [inline]
 enqueue_hrtimer+0x27/0x3e0 kernel/time/hrtimer.c:1084
 __run_hrtimer kernel/time/hrtimer.c:1702 [inline]
 __hrtimer_run_queues+0xb02/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:write_comp_data kernel/kcov.c:221 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp4+0x3/0x70 kernel/kcov.c:287
Code: 83 c2 01 48 c7 44 30 e0 02 00 00 00 48 89 7c 30 e8 48 89 4c 30 f0 4c 89 54 d8 20 48 89 10 5b c3 0f 1f 80 00 00 00 00 41 89 f8 <bf> 03 00 00 00 4c 8b 14 24 89 f1 65 48 8b 34 25 40 70 02 00 e8 14
RSP: 0018:ffffc9000279fcb0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: ffff88807d245700 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff888073e8d780 R08: 0000000000000000 R09: ffffffff8ffb9a0f
R10: ffffffff81d0ef9e R11: 0000000000000001 R12: ffff888073e8d810
R13: ffffffff81d0ed55 R14: ffff888073e8d7c8 R15: 0000000000280040
 retain_dentry fs/dcache.c:646 [inline]
 dput+0x2c3/0xbd0 fs/dcache.c:883
 path_put+0x2d/0x60 fs/namei.c:557
 vfs_statx+0x244/0x390 fs/stat.c:231
 vfs_fstatat fs/stat.c:243 [inline]
 __do_sys_newfstatat+0x96/0x120 fs/stat.c:412
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f2a3d2821da
Code: 48 89 f2 b9 00 01 00 00 48 89 fe bf 9c ff ff ff e9 0b 00 00 00 66 2e 0f 1f 84 00 00 00 00 00 90 41 89 ca b8 06 01 00 00 0f 05 <3d> 00 f0 ff ff 77 07 31 c0 c3 0f 1f 40 00 48 8b 15 69 fc 0c 00 f7
RSP: 002b:00007ffcf3302a98 EFLAGS: 00000246 ORIG_RAX: 0000000000000106
RAX: ffffffffffffffda RBX: 0000556fb9d0d6b8 RCX: 00007f2a3d2821da
RDX: 00007ffcf3302aa8 RSI: 0000556fb9cfd897 RDI: 00000000ffffff9c
RBP: 0000556fbb1de908 R08: 0005d56b698cd371 R09: 00007ffcf336b080
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000001 R14: 0005d56b698cd371 R15: 00007ffcf3302aa8
 </TASK>
rcu: rcu_preempt kthread starved for 10387 jiffies! g143985 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28568 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4986 [inline]
 __schedule+0xab2/0x4f00 kernel/sched/core.c:6296
 schedule+0xd2/0x260 kernel/sched/core.c:6369
 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
 rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1963
 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2136
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 0
CPU: 0 PID: 32096 Comm: syz-executor.0 Not tainted 5.16.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_check_gp_kthread_starvation.cold+0x1fb/0x200 kernel/rcu/tree_stall.h:458
 print_other_cpu_stall kernel/rcu/tree_stall.h:563 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:706 [inline]
 rcu_pending kernel/rcu/tree.c:3919 [inline]
 rcu_sched_clock_irq+0x1f7c/0x2150 kernel/rcu/tree.c:2617
 update_process_times+0x16d/0x200 kernel/time/timer.c:1785
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1428
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_many_cond+0x452/0xc20 kernel/smp.c:969
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 f4 4c 89 f5 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 60 7c 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 33 06 00 00 8b 43 08 31
RSP: 0018:ffffc900032274f0 EFLAGS: 00000246
RAX: 0000000000040000 RBX: ffff8880b9d3e9e0 RCX: ffffc9000c0b2000
RDX: 0000000000040000 RSI: ffffffff816c2410 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff816c2436 R11: 0000000000000000 R12: ffffed10173a7d3d
R13: 0000000000000001 R14: ffff8880b9d3e9e8 R15: 0000000000000001
 on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1135
 __purge_vmap_area_lazy+0x1f6/0x1c50 mm/vmalloc.c:1687
 _vm_unmap_aliases.part.0+0x3f0/0x500 mm/vmalloc.c:2111
 _vm_unmap_aliases mm/vmalloc.c:2085 [inline]
 vm_unmap_aliases+0x45/0x50 mm/vmalloc.c:2134
 change_page_attr_set_clr+0x241/0x500 arch/x86/mm/pat/set_memory.c:1743
 change_page_attr_clear arch/x86/mm/pat/set_memory.c:1800 [inline]
 set_memory_ro+0x78/0xa0 arch/x86/mm/pat/set_memory.c:1946
 bpf_jit_binary_lock_ro include/linux/filter.h:889 [inline]
 bpf_int_jit_compile+0xe92/0x1240 arch/x86/net/bpf_jit_comp.c:2382
 bpf_prog_select_runtime+0x46b/0x780 kernel/bpf/core.c:1931
 bpf_prog_load+0xf89/0x21f0 kernel/bpf/syscall.c:2348
 __sys_bpf+0x68a/0x5970 kernel/bpf/syscall.c:4634
 __do_sys_bpf kernel/bpf/syscall.c:4738 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:4736 [inline]
 __x64_sys_bpf+0x75/0xb0 kernel/bpf/syscall.c:4736
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f37ab037eb9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f37a99ad168 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007f37ab14af60 RCX: 00007f37ab037eb9
RDX: 0000000000000078 RSI: 0000000020000180 RDI: 0000000000000005
RBP: 00007f37ab09208d R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffc359fe20f R14: 00007f37a99ad300 R15: 0000000000022000
 </TASK>
----------------
Code disassembly (best guess):
   0:	7c 08                	jl     0xa
   2:	84 d2                	test   %dl,%dl
   4:	0f 85 e1 00 00 00    	jne    0xeb
   a:	8b 15 c8 9e 00 0f    	mov    0xf009ec8(%rip),%edx        # 0xf009ed8
  10:	65 01 1d 01 95 b2 7e 	add    %ebx,%gs:0x7eb29501(%rip)        # 0x7eb29518
  17:	85 d2                	test   %edx,%edx
  19:	75 11                	jne    0x2c
  1b:	65 8b 05 f6 94 b2 7e 	mov    %gs:0x7eb294f6(%rip),%eax        # 0x7eb29518
  22:	0f b6 c0             	movzbl %al,%eax
  25:	3d f4 00 00 00       	cmp    $0xf4,%eax
* 2a:	7f 64                	jg     0x90 <-- trapping instruction
  2c:	65 8b 05 e5 94 b2 7e 	mov    %gs:0x7eb294e5(%rip),%eax        # 0x7eb29518
  33:	25 ff ff ff 7f       	and    $0x7fffffff,%eax
  38:	39 c3                	cmp    %eax,%ebx
  3a:	74 03                	je     0x3f
  3c:	5b                   	pop    %rbx
  3d:	5d                   	pop    %rbp
  3e:	c3                   	retq
  3f:	48                   	rex.W

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/01/13 00:22 upstream f079ab01b560 44d1319a .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in newfstatat
2021/12/28 05:04 upstream a8ad9a2434dc 6b3c5e64 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in newfstatat
* Struck through repros no longer work on HEAD.