syzbot


INFO: rcu detected stall in sys_newfstatat

Status: upstream: reported on 2024/04/08 00:26
Reported-by: syzbot+b4da8777e5518ae24a0d@syzkaller.appspotmail.com
First crash: 317d, last: 12d
Similar bugs (5)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_newfstatat (4) mm C error 391 9d05h 848d 0/28 upstream: reported C repro on 2022/10/25 00:38
linux-6.1 INFO: rcu detected stall in sys_newfstatat origin:upstream C done 8 74d 612d 3/3 fixed on 2025/01/19 08:56
upstream INFO: rcu detected stall in sys_newfstatat cgroups mm 1 1905d 1904d 0/28 closed as invalid on 2019/12/04 14:04
upstream INFO: rcu detected stall in sys_newfstatat (2) kernfs 1 1103d 1103d 0/28 auto-closed as invalid on 2022/05/13 02:53
upstream INFO: rcu detected stall in sys_newfstatat (3) tomoyo 1 990d 990d 0/28 auto-closed as invalid on 2022/09/03 09:55

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (10499 ticks this GP) idle=17d/1/0x4000000000000000 softirq=30357/30357 fqs=0 
	(t=10500 jiffies g=35941 q=120)
rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g35941 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=0 timer-softirq=12029
rcu: rcu_preempt kthread starved for 10500 jiffies! g35941 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:27064 pid:   15 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5027 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6373
 schedule+0x11b/0x1f0 kernel/sched/core.c:6456
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1914
 rcu_gp_fqs_loop+0x2bf/0x1080 kernel/rcu/tree.c:1972
 rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2145
 kthread+0x3f6/0x4f0 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 3546 Comm: udevd Not tainted 5.15.178-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
RIP: 0010:advance_sched+0x52/0x940 net/sched/sch_taprio.c:717
Code: c1 e8 03 42 80 3c 30 00 74 05 e8 79 d6 3d f9 4d 8b 65 88 49 8d 6d a8 48 89 ef e8 79 52 bc 01 e8 74 bd b2 01 89 c3 31 ff 89 c6 <e8> 59 f1 f3 f8 85 db 74 28 49 8d 7d c0 be ff ff ff ff e8 07 ba b2
RSP: 0018:ffffc90000007cc0 EFLAGS: 00000046
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff88807d5a9dc0
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffff88807c0dfae8 R08: dffffc0000000000 R09: 0000000000000003
R10: ffffffffffffffff R11: dffffc0000000001 R12: ffff88807c0df800
R13: ffff88807c0dfb40 R14: dffffc0000000000 R15: ffff88807d5aa898
FS:  00007fb71c47bc80(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000200000057000 CR3: 000000002a761000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x598/0xcf0 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1097 [inline]
 __sysvec_apic_timer_interrupt+0x13b/0x4b0 arch/x86/kernel/apic/apic.c:1114
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:check_kcov_mode kernel/kcov.c:172 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x13/0x60 kernel/kcov.c:206
Code: 48 8b 3d 18 04 8a 0c 48 89 de 5b e9 47 11 49 00 cc cc cc cc cc cc cc 48 8b 04 24 65 48 8b 0d 24 ea 81 7e 65 8b 15 25 ea 81 7e <81> e2 00 01 ff 00 74 11 81 fa 00 01 00 00 75 35 83 b9 34 16 00 00
RSP: 0018:ffffc90002e8f678 EFLAGS: 00000282
RAX: ffffffff84398c6a RBX: ffff0a00ffffff04 RCX: ffff88807d5a9dc0
RDX: 0000000080000000 RSI: 0000000000000007 RDI: 00000000ffffffff
RBP: 00ffffffffffffff R08: ffffffff84398c5d R09: ffffffff8438d0c0
R10: 0000000000000012 R11: ffff88807d5a9dc0 R12: 0000000000000007
R13: 00000000fffffff8 R14: ffffc90002e8f881 R15: ffffc90002e8f88c
 widen_string+0x3a/0x310 lib/vsprintf.c:618
 vsnprintf+0x11a4/0x1c70 lib/vsprintf.c:2811
 tomoyo_supervisor+0x145/0x12c0 security/tomoyo/common.c:2069
 tomoyo_audit_path_log security/tomoyo/file.c:168 [inline]
 tomoyo_path_permission+0x243/0x360 security/tomoyo/file.c:587
 tomoyo_path_perm+0x436/0x6b0 security/tomoyo/file.c:838
 security_inode_getattr+0xcf/0x120 security/security.c:1348
 vfs_getattr+0x26/0x360 fs/stat.c:157
 vfs_statx+0x18f/0x3b0 fs/stat.c:225
 vfs_fstatat fs/stat.c:243 [inline]
 __do_sys_newfstatat fs/stat.c:411 [inline]
 __se_sys_newfstatat fs/stat.c:405 [inline]
 __x64_sys_newfstatat+0x12c/0x1b0 fs/stat.c:405
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7fb71c5a65f4
Code: 64 c7 00 09 00 00 00 83 c8 ff c3 48 89 f2 b9 00 01 00 00 48 89 fe bf 9c ff ff ff e9 00 00 00 00 41 89 ca b8 06 01 00 00 0f 05 <45> 31 c0 3d 00 f0 ff ff 76 10 48 8b 15 03 a8 0d 00 f7 d8 41 83 c8
RSP: 002b:00007ffc2e93b3e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000106
RAX: ffffffffffffffda RBX: 0000559e50bb81af RCX: 00007fb71c5a65f4
RDX: 00007ffc2e93b3f8 RSI: 00007ffc2e93b888 RDI: 00000000ffffff9c
RBP: 00007ffc2e93b488 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc2e93b888
R13: 00007ffc2e93b3f8 R14: 0000559e83e62910 R15: 0000000000000000
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 3546 Comm: udevd Not tainted 5.15.178-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
RIP: 0010:lock_acquire+0x292/0x4f0 kernel/locking/lockdep.c:5627
Code: 11 00 00 00 00 66 43 c7 44 25 15 00 00 43 c6 44 25 17 00 65 48 8b 04 25 28 00 00 00 48 3b 84 24 00 01 00 00 0f 85 52 02 00 00 <48> 8d 65 d8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 65 8b 05 38 a9 9f 7e
RSP: 0018:ffffc90000007b60 EFLAGS: 00000046
RAX: 17670fdb6a005100 RBX: 1ffff92000000f78 RCX: ffffffff816389a2
RDX: dffffc0000000000 RSI: ffffffff8aab3e20 RDI: ffffffff8af9f2c0
RBP: ffffc90000007ca8 R08: dffffc0000000000 R09: fffffbfff2131822
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000000f74
R13: dffffc0000000000 R14: ffffc90000007bc0 R15: 0000000000000046
FS:  00007fb71c47bc80(0000) GS:ffff8880b8e00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000200000057000 CR3: 000000002a761000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline]
 _raw_spin_lock_irq+0xcf/0x110 kernel/locking/spinlock.c:170
 __run_hrtimer kernel/time/hrtimer.c:1692 [inline]
 __hrtimer_run_queues+0x662/0xcf0 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1097 [inline]
 __sysvec_apic_timer_interrupt+0x13b/0x4b0 arch/x86/kernel/apic/apic.c:1114
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:check_kcov_mode kernel/kcov.c:172 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x13/0x60 kernel/kcov.c:206
Code: 48 8b 3d 18 04 8a 0c 48 89 de 5b e9 47 11 49 00 cc cc cc cc cc cc cc 48 8b 04 24 65 48 8b 0d 24 ea 81 7e 65 8b 15 25 ea 81 7e <81> e2 00 01 ff 00 74 11 81 fa 00 01 00 00 75 35 83 b9 34 16 00 00
RSP: 0018:ffffc90002e8f678 EFLAGS: 00000282
RAX: ffffffff84398c6a RBX: ffff0a00ffffff04 RCX: ffff88807d5a9dc0
RDX: 0000000080000000 RSI: 0000000000000007 RDI: 00000000ffffffff
RBP: 00ffffffffffffff R08: ffffffff84398c5d R09: ffffffff8438d0c0
R10: 0000000000000012 R11: ffff88807d5a9dc0 R12: 0000000000000007
R13: 00000000fffffff8 R14: ffffc90002e8f881 R15: ffffc90002e8f88c
 widen_string+0x3a/0x310 lib/vsprintf.c:618
 vsnprintf+0x11a4/0x1c70 lib/vsprintf.c:2811
 tomoyo_supervisor+0x145/0x12c0 security/tomoyo/common.c:2069
 tomoyo_audit_path_log security/tomoyo/file.c:168 [inline]
 tomoyo_path_permission+0x243/0x360 security/tomoyo/file.c:587
 tomoyo_path_perm+0x436/0x6b0 security/tomoyo/file.c:838
 security_inode_getattr+0xcf/0x120 security/security.c:1348
 vfs_getattr+0x26/0x360 fs/stat.c:157
 vfs_statx+0x18f/0x3b0 fs/stat.c:225
 vfs_fstatat fs/stat.c:243 [inline]
 __do_sys_newfstatat fs/stat.c:411 [inline]
 __se_sys_newfstatat fs/stat.c:405 [inline]
 __x64_sys_newfstatat+0x12c/0x1b0 fs/stat.c:405
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7fb71c5a65f4
Code: 64 c7 00 09 00 00 00 83 c8 ff c3 48 89 f2 b9 00 01 00 00 48 89 fe bf 9c ff ff ff e9 00 00 00 00 41 89 ca b8 06 01 00 00 0f 05 <45> 31 c0 3d 00 f0 ff ff 76 10 48 8b 15 03 a8 0d 00 f7 d8 41 83 c8
RSP: 002b:00007ffc2e93b3e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000106
RAX: ffffffffffffffda RBX: 0000559e50bb81af RCX: 00007fb71c5a65f4
RDX: 00007ffc2e93b3f8 RSI: 00007ffc2e93b888 RDI: 00000000ffffff9c
RBP: 00007ffc2e93b488 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc2e93b888
R13: 00007ffc2e93b3f8 R14: 0000559e83e62910 R15: 0000000000000000
 </TASK>
NMI backtrace for cpu 1
CPU: 1 PID: 9205 Comm: syz.0.1072 Not tainted 5.15.178-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
 rcu_dump_cpu_stacks+0x223/0x390 kernel/rcu/tree_stall.h:349
 print_cpu_stall+0x320/0x600 kernel/rcu/tree_stall.h:633
 check_cpu_stall kernel/rcu/tree_stall.h:727 [inline]
 rcu_pending kernel/rcu/tree.c:3932 [inline]
 rcu_sched_clock_irq+0x8d9/0x1150 kernel/rcu/tree.c:2619
 update_process_times+0x196/0x200 kernel/time/timer.c:1818
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x386/0x550 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1688 [inline]
 __hrtimer_run_queues+0x55b/0xcf0 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1097 [inline]
 __sysvec_apic_timer_interrupt+0x13b/0x4b0 arch/x86/kernel/apic/apic.c:1114
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_many_cond+0xa93/0xd90 kernel/smp.c:969
Code: 04 03 84 c0 0f 85 84 00 00 00 45 8b 7d 00 44 89 fe 83 e6 01 31 ff e8 3c d6 0b 00 41 83 e7 01 75 07 e8 d1 d2 0b 00 eb 41 f3 90 <48> b8 00 00 00 00 00 fc ff df 0f b6 04 03 84 c0 75 11 41 f7 45 00
RSP: 0018:ffffc900034af880 EFLAGS: 00000246
RAX: ffffffff8174b2e4 RBX: 1ffff110171c8541 RCX: 0000000000080000
RDX: ffffc90014ef4000 RSI: 000000000007ffff RDI: 0000000000080000
RBP: ffffc900034af9b0 R08: ffffffff8174b2b4 R09: ffffed10171e7673
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000000
R13: ffff8880b8e42a08 R14: ffff8880b8f3b380 R15: 0000000000000001
 clock_was_set+0x7b6/0x880 kernel/time/hrtimer.c:974
 timekeeping_inject_offset+0x65f/0x710 kernel/time/timekeeping.c:1378
 do_adjtimex+0x511/0xab0 kernel/time/timekeeping.c:2427
 do_clock_adjtime kernel/time/posix-timers.c:1154 [inline]
 __do_sys_clock_adjtime kernel/time/posix-timers.c:1166 [inline]
 __se_sys_clock_adjtime kernel/time/posix-timers.c:1157 [inline]
 __x64_sys_clock_adjtime+0x1dc/0x290 kernel/time/posix-timers.c:1157
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x66/0xd0
RIP: 0033:0x7f72f9761de9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f72f75cb038 EFLAGS: 00000246 ORIG_RAX: 0000000000000131
RAX: ffffffffffffffda RBX: 00007f72f997afa0 RCX: 00007f72f9761de9
RDX: 0000000000000000 RSI: 0000200000000480 RDI: 0000000000000000
RBP: 00007f72f97e32a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f72f997afa0 R15: 00007fff3c33dbd8
 </TASK>

Crashes (16):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/02/07 09:57 linux-5.15.y c16c81c81336 53657d1b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2025/01/30 19:02 linux-5.15.y 003148680b79 4c6ac32f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/12/19 20:46 linux-5.15.y 91786f140358 5905cb39 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in sys_newfstatat
2024/12/12 19:07 linux-5.15.y 0a51d2d4527b 530e80f8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/11/28 22:29 linux-5.15.y 0a51d2d4527b 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/10/10 01:36 linux-5.15.y 3a5928702e71 56fb2cb7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/08/13 05:05 linux-5.15.y 7e89efd3ae1c 7b0f4b46 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/07/26 07:21 linux-5.15.y 7c6d66f0266f 3f86dfed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/07/13 04:24 linux-5.15.y f45bea23c39c eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/06/24 23:15 linux-5.15.y 4878aadf2d15 215eef4a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/06/23 03:45 linux-5.15.y 4878aadf2d15 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/06/21 18:22 linux-5.15.y 4878aadf2d15 edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/06/17 01:10 linux-5.15.y 4878aadf2d15 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/04/18 04:52 linux-5.15.y c52b9710c83d acc528cb .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/04/08 00:25 linux-5.15.y 9465fef4ae35 ca620dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in sys_newfstatat
2024/09/01 07:17 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 INFO: rcu detected stall in sys_newfstatat
* Struck through repros no longer work on HEAD.