syzbot


INFO: rcu detected stall in sys_openat (2)

Status: closed as invalid on 2022/02/08 09:50
Reported-by: syzbot+@syzkaller.appspotmail.com
First crash: 517d, last: 305d
similar bugs (1):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_openat 5 1422d 1501d 0/24 closed as dup on 2018/10/27 13:02

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=9ef/1/0x4000000000000000 softirq=141449/141451 fqs=1 
	(detected by 1, t=10502 jiffies, g=215657, q=72)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 18339 Comm: udevd Not tainted 5.17.0-rc2-syzkaller-00071-g1f2cfdd349b7 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:check_preemption_disabled+0x3e/0x170 lib/smp_processor_id.c:56
Code: 44 8b 25 7d e2 b7 76 65 8b 1d 6e 38 b8 76 81 e3 ff ff ff 7f 31 ff 89 de 0f 1f 44 00 00 85 db 74 11 0f 1f 44 00 00 44 89 e0 5b <5d> 41 5c 41 5d 41 5e c3 0f 1f 44 00 00 9c 5b 81 e3 00 02 00 00 31
RSP: 0018:ffffc90000007cd0 EFLAGS: 00000002
RAX: 0000000000000000 RBX: ffffffffffffffff RCX: ffffc90000007d48
RDX: 1ffff1100f64b14a RSI: 0000000000010001 RDI: 0000000000000000
RBP: ffffffff8a064920 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
R13: ffffffff89ac6120 R14: ffff88807b258a58 R15: ffff88807b258000
FS:  00007f77ed0d5840(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd2d59f458 CR3: 0000000040206000 CR4: 0000000000350ef0
Call Trace:
 <IRQ>
 lockdep_recursion_finish kernel/locking/lockdep.c:438 [inline]
 lock_release+0x3bb/0x720 kernel/locking/lockdep.c:5661
 __raw_spin_unlock include/linux/spinlock_api_smp.h:141 [inline]
 _raw_spin_unlock+0x12/0x40 kernel/locking/spinlock.c:186
 spin_unlock include/linux/spinlock.h:389 [inline]
 advance_sched+0x435/0x9a0 net/sched/sch_taprio.c:763
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x609/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:should_resched arch/x86/include/asm/preempt.h:103 [inline]
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0x3d/0x70 kernel/locking/spinlock.c:194
Code: 41 10 f8 48 89 ef e8 f2 b6 10 f8 81 e3 00 02 00 00 75 25 9c 58 f6 c4 02 75 2d 48 85 db 74 01 fb bf 01 00 00 00 e8 23 6a 03 f8 <65> 8b 05 9c 5d b5 76 85 c0 74 0a 5b 5d c3 e8 70 ab 31 f8 eb d4 e8
RSP: 0018:ffffc9000292f6c8 EFLAGS: 00000297
RAX: 0000000080000001 RBX: 0000000000000200 RCX: 0000000000000000
RDX: 0000000000000001 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffff888010c678f0 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff817ebdd8 R11: 0000000000000001 R12: 0000000000000246
R13: ffffc9000292f700 R14: ffff888010c67930 R15: ffff888010c678f0
 rwsem_wake.isra.0+0xbb/0x110 kernel/locking/rwsem.c:1179
 __up_read+0x2b5/0x720 kernel/locking/rwsem.c:1298
 kernfs_dop_revalidate+0x500/0x600 fs/kernfs/dir.c:1070
 d_revalidate fs/namei.c:854 [inline]
 d_revalidate fs/namei.c:851 [inline]
 lookup_fast+0x385/0x6d0 fs/namei.c:1668
 walk_component+0xcf/0x6a0 fs/namei.c:2016
 link_path_walk.part.0+0x7ef/0xf70 fs/namei.c:2347
 link_path_walk fs/namei.c:2272 [inline]
 path_openat+0x266/0x2940 fs/namei.c:3605
 do_filp_open+0x1aa/0x400 fs/namei.c:3636
 do_sys_openat2+0x16d/0x4d0 fs/open.c:1214
 do_sys_open fs/open.c:1230 [inline]
 __do_sys_openat fs/open.c:1246 [inline]
 __se_sys_openat fs/open.c:1241 [inline]
 __x64_sys_openat+0x13f/0x1f0 fs/open.c:1241
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f77ed22c7f2
Code: 58 f6 c2 40 75 3a 89 d0 45 31 d2 25 00 00 41 00 3d 00 00 41 00 74 29 64 8b 04 25 18 00 00 00 85 c0 75 45 b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 82 00 00 00 48 83 c4 78 c3 0f 1f 44 00 00
RSP: 002b:00007ffd2d59e590 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 00007ffd2d5a06f8 RCX: 00007f77ed22c7f2
RDX: 0000000000080000 RSI: 00007f77ed61bd78 RDI: 000000000000000b
RBP: 00007f77ed93f780 R08: 00000000ffffffff R09: 00007f77ed907010
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000b
R13: 00007f77ed61bd78 R14: 0000000000080000 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10500 jiffies! g215657 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28736 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4986 [inline]
 __schedule+0xab2/0x4db0 kernel/sched/core.c:6295
 schedule+0xd2/0x260 kernel/sched/core.c:6368
 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
 rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1963
 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2136
 kthread+0x2e9/0x3a0 kernel/kthread.c:377
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 12770 Comm: kworker/u4:9 Not tainted 5.17.0-rc2-syzkaller-00071-g1f2cfdd349b7 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_check_gp_kthread_starvation.cold+0x1fb/0x200 kernel/rcu/tree_stall.h:458
 print_other_cpu_stall kernel/rcu/tree_stall.h:563 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:706 [inline]
 rcu_pending kernel/rcu/tree.c:3919 [inline]
 rcu_sched_clock_irq+0x1f7c/0x2150 kernel/rcu/tree.c:2617
 update_process_times+0x16d/0x200 kernel/time/timer.c:1785
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1428
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_many_cond+0x447/0xc90 kernel/smp.c:969
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 f4 4c 89 f5 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 8b 87 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 95 06 00 00 8b 43 08 31
RSP: 0018:ffffc90002c8fa08 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9c41fa0 RCX: 0000000000000000
RDX: ffff888072423a00 RSI: ffffffff816cf105 RDI: 0000000000000003
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff816cf12b R11: 0000000000000000 R12: ffffed10173883f5
R13: 0000000000000000 R14: ffff8880b9c41fa8 R15: 0000000000000001
 on_each_cpu_cond_mask+0x56/0xa0 kernel/smp.c:1135
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:1112 [inline]
 text_poke_bp_batch+0x1b1/0x510 arch/x86/kernel/alternative.c:1300
 text_poke_flush arch/x86/kernel/alternative.c:1470 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:1467 [inline]
 text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1477
 arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x1da/0x400 kernel/jump_label.c:830
 static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
 static_key_enable+0x16/0x20 kernel/jump_label.c:190
 toggle_allocation_gate mm/kfence/core.c:734 [inline]
 toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:726
 process_one_work+0x9ac/0x1650 kernel/workqueue.c:2307
 worker_thread+0x657/0x1110 kernel/workqueue.c:2454
 kthread+0x2e9/0x3a0 kernel/kthread.c:377
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
----------------
Code disassembly (best guess):
   0:	44 8b 25 7d e2 b7 76 	mov    0x76b7e27d(%rip),%r12d        # 0x76b7e284
   7:	65 8b 1d 6e 38 b8 76 	mov    %gs:0x76b8386e(%rip),%ebx        # 0x76b8387c
   e:	81 e3 ff ff ff 7f    	and    $0x7fffffff,%ebx
  14:	31 ff                	xor    %edi,%edi
  16:	89 de                	mov    %ebx,%esi
  18:	0f 1f 44 00 00       	nopl   0x0(%rax,%rax,1)
  1d:	85 db                	test   %ebx,%ebx
  1f:	74 11                	je     0x32
  21:	0f 1f 44 00 00       	nopl   0x0(%rax,%rax,1)
  26:	44 89 e0             	mov    %r12d,%eax
  29:	5b                   	pop    %rbx
* 2a:	5d                   	pop    %rbp <-- trapping instruction
  2b:	41 5c                	pop    %r12
  2d:	41 5d                	pop    %r13
  2f:	41 5e                	pop    %r14
  31:	c3                   	retq
  32:	0f 1f 44 00 00       	nopl   0x0(%rax,%rax,1)
  37:	9c                   	pushfq
  38:	5b                   	pop    %rbx
  39:	81 e3 00 02 00 00    	and    $0x200,%ebx
  3f:	31                   	.byte 0x31

Crashes (8):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-kasan-gce-root 2022/02/04 01:44 upstream 1f2cfdd349b7 30646bfe .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-kasan-gce-selinux-root 2021/11/09 23:55 upstream d2f38a3c6507 59bcaf9a .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-bpf-kasan-gce 2021/08/02 23:06 bpf b98677d3ea55 6c236867 .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-bpf-next-kasan-gce 2021/09/03 00:45 bpf-next a16ef91aa61a 15cea0a3 .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-bpf-next-kasan-gce 2021/08/19 09:33 bpf-next 374e74de9631 a2fe1cb5 .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-bpf-next-kasan-gce 2021/07/07 02:15 bpf-next dbe69e433722 cca78469 .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-linux-next-kasan-gce-root 2022/01/03 16:37 linux-next ea586a076e8a 4a3f34f2 .config log report info INFO: rcu detected stall in sys_openat
ci-upstream-linux-next-kasan-gce-root 2021/09/20 07:56 linux-next 9004fd387338 70b76c1d .config log report info INFO: rcu detected stall in sys_openat
* Struck through repros no longer work on HEAD.