INFO: rcu detected stall in sys

Kernel	Title	Repro	Cause bisect	Fix bisect	Count	Last	Reported	Patched	Status
upstream	INFO: rcu detected stall in sys_open fs				1	1801d	1801d	0/26	auto-closed as invalid on 2019/10/25 09:11
linux-4.19	INFO: rcu detected stall in sys_open				1	1688d	1688d	0/1	auto-closed as invalid on 2020/01/09 20:01
upstream	INFO: rcu detected stall in sys_open (3) fs	C	inconclusive		5	838d	1074d	0/26	closed as invalid on 2022/02/08 10:35
upstream	INFO: rcu detected stall in sys_open (2) crypto	C	done	done	42	1262d	1613d	15/26	fixed on 2020/12/23 11:40

rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 0-...!: (10499 ticks this GP) idle=591/1/0x4000000000000000 softirq=8184/8184 fqs=0 (t=10500 jiffies g=9021 q=3 ncpus=2) rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g9021 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 rcu: Possible timer handling issue on cpu=1 timer-softirq=3163 rcu: rcu_preempt kthread starved for 10500 jiffies! g9021 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:I stack:28680 pid: 16 ppid: 2 flags:0x00004000 Call Trace: <TASK> context_switch kernel/sched/core.c:5106 [inline] __schedule+0xa9a/0x4cc0 kernel/sched/core.c:6421 schedule+0xd2/0x1f0 kernel/sched/core.c:6493 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1907 rcu_gp_fqs_loop+0x1c0/0x840 kernel/rcu/tree.c:2078 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2267 kthread+0x2e9/0x3a0 kernel/kthread.c:376 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298 </TASK> rcu: Stack dump where RCU GP kthread last ran: Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 4473 Comm: syz-executor.3 Not tainted 5.18.0-rc3-next-20220422-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:kvm_wait+0x98/0x100 arch/x86/kernel/kvm.c:1058 Code: fa 83 e2 07 38 d0 7f 04 84 c0 75 63 0f b6 07 40 38 c6 74 35 48 83 c4 10 c3 c3 e8 23 91 4b 00 eb 07 0f 00 2d da b1 94 08 fb f4 <48> 83 c4 10 c3 89 74 24 0c 48 89 3c 24 e8 56 8f 4b 00 8b 74 24 0c RSP: 0018:ffffc90000de0488 EFLAGS: 00000246 RAX: 0000000000000007 RBX: 0000000000000000 RCX: 1ffffffff1b73199 RDX: 0000000000000000 RSI: ffffffff81807171 RDI: ffffffff8134dffd RBP: ffff88802295e1c8 R08: 0000000000000000 R09: 0000000000000000 R10: ffffffff81807158 R11: 0000000000000001 R12: 0000000000000000 R13: ffffed100452bc39 R14: 0000000000000001 R15: ffff8880b9d3ae40 FS: 00007fabee5fe700(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000001b2ca43000 CR3: 0000000051c4c000 CR4: 00000000003506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <IRQ> pv_wait arch/x86/include/asm/paravirt.h:603 [inline] pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline] __pv_queued_spin_lock_slowpath+0x8c7/0xb50 kernel/locking/qspinlock.c:511 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:591 [inline] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline] queued_spin_lock include/asm-generic/qspinlock.h:85 [inline] do_raw_spin_lock+0x200/0x2a0 kernel/locking/spinlock_debug.c:115 spin_lock include/linux/spinlock.h:354 [inline] task_lock include/linux/sched/task.h:170 [inline] __get_task_comm+0x23/0x50 fs/exec.c:1219 __set_page_owner_handle mm/page_owner.c:174 [inline] __set_page_owner+0x253/0x380 mm/page_owner.c:192 prep_new_page mm/page_alloc.c:2394 [inline] get_page_from_freelist+0xba2/0x3e00 mm/page_alloc.c:4135 __alloc_pages+0x1b2/0x500 mm/page_alloc.c:5356 alloc_pages+0x1aa/0x310 mm/mempolicy.c:2273 alloc_slab_page mm/slub.c:1797 [inline] allocate_slab+0x26c/0x3c0 mm/slub.c:1942 new_slab mm/slub.c:2002 [inline] ___slab_alloc+0x985/0xd90 mm/slub.c:3002 __slab_alloc.constprop.0+0x4d/0xa0 mm/slub.c:3089 slab_alloc_node mm/slub.c:3180 [inline] kmem_cache_alloc_node+0x122/0x3f0 mm/slub.c:3264 __alloc_skb+0x215/0x340 net/core/skbuff.c:414 skb_copy+0x139/0x3c0 net/core/skbuff.c:1585 mac80211_hwsim_tx_frame_no_nl.isra.0+0xb7c/0x13b0 drivers/net/wireless/mac80211_hwsim.c:1642 mac80211_hwsim_tx_frame+0x1ee/0x2a0 drivers/net/wireless/mac80211_hwsim.c:1884 mac80211_hwsim_beacon_tx+0x49e/0x920 drivers/net/wireless/mac80211_hwsim.c:1938 __iterate_interfaces+0x1e5/0x560 net/mac80211/util.c:793 ieee80211_iterate_active_interfaces_atomic+0x70/0x180 net/mac80211/util.c:829 mac80211_hwsim_beacon+0xcd/0x1c0 drivers/net/wireless/mac80211_hwsim.c:1961 __run_hrtimer kernel/time/hrtimer.c:1685 [inline] __hrtimer_run_queues+0x609/0xe50 kernel/time/hrtimer.c:1749 hrtimer_run_softirq+0x17b/0x360 kernel/time/hrtimer.c:1766 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558 invoke_softirq kernel/softirq.c:432 [inline] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:637 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1106 </IRQ> <TASK> asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:649 RIP: 0010:memset_erms+0xb/0x10 arch/x86/lib/memset_64.S:65 Code: 03 40 0f b6 f6 48 b8 01 01 01 01 01 01 01 01 48 0f af c6 f3 48 ab 89 d1 f3 aa 4c 89 c8 c3 90 49 89 f9 40 88 f0 48 89 d1 f3 aa <4c> 89 c8 c3 90 49 89 fa 40 0f b6 ce 48 b8 01 01 01 01 01 01 01 01 RSP: 0018:ffffc9000499e890 EFLAGS: 00000202 RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff888012febbb8 RBP: ffff888012febba8 R08: 0000000000000001 R09: ffff888012febbb7 R10: ffffed10025fd776 R11: 0000000000000001 R12: 000000000000000e R13: 0000000000000010 R14: ffff888012febb88 R15: 0000000000000007 strscpy_pad+0x62/0x70 lib/string_helpers.c:792 __get_task_comm+0x35/0x50 fs/exec.c:1221 __set_page_owner_handle mm/page_owner.c:174 [inline] __set_page_owner+0x253/0x380 mm/page_owner.c:192 prep_new_page mm/page_alloc.c:2394 [inline] get_page_from_freelist+0xba2/0x3e00 mm/page_alloc.c:4135 __alloc_pages+0x1b2/0x500 mm/page_alloc.c:5356 alloc_pages+0x1aa/0x310 mm/mempolicy.c:2273 folio_alloc+0x1c/0x70 mm/mempolicy.c:2283 filemap_alloc_folio+0x8e/0xb0 mm/filemap.c:996 page_cache_ra_unbounded+0x1af/0x550 mm/readahead.c:240 do_page_cache_ra mm/readahead.c:291 [inline] page_cache_ra_order+0x67a/0x940 mm/readahead.c:545 ondemand_readahead+0x7d7/0x11c0 mm/readahead.c:665 page_cache_async_ra mm/readahead.c:714 [inline] page_cache_async_ra+0x113/0x140 mm/readahead.c:696 filemap_readahead mm/filemap.c:2568 [inline] filemap_get_pages+0x6c6/0x17c0 mm/filemap.c:2608 filemap_read+0x325/0xc70 mm/filemap.c:2679 generic_file_read_iter+0x3b0/0x5a0 mm/filemap.c:2825 ext4_file_read_iter+0x1d7/0x600 fs/ext4/file.c:132 __kernel_read+0x4e1/0xa30 fs/read_write.c:440 integrity_kernel_read+0x7b/0xb0 security/integrity/iint.c:199 ima_calc_file_hash_tfm+0x2aa/0x3b0 security/integrity/ima/ima_crypto.c:484 ima_calc_file_shash security/integrity/ima/ima_crypto.c:515 [inline] ima_calc_file_hash+0x191/0x4a0 security/integrity/ima/ima_crypto.c:572 ima_collect_measurement+0x4c3/0x570 security/integrity/ima/ima_api.c:252 process_measurement+0xd2b/0x1910 security/integrity/ima/ima_main.c:337 ima_file_check+0xac/0x100 security/integrity/ima/ima_main.c:517 do_open fs/namei.c:3471 [inline] path_openat+0x161a/0x2910 fs/namei.c:3602 do_filp_open+0x1aa/0x400 fs/namei.c:3629 do_sys_openat2+0x16d/0x4c0 fs/open.c:1212 do_sys_open fs/open.c:1228 [inline] __do_sys_open fs/open.c:1236 [inline] __se_sys_open fs/open.c:1232 [inline] __x64_sys_open+0x119/0x1c0 fs/open.c:1232 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7fabef6890e9 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fabee5fe168 EFLAGS: 00000246 ORIG_RAX: 0000000000000002 RAX: ffffffffffffffda RBX: 00007fabef79c030 RCX: 00007fabef6890e9 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000020000400 RBP: 00007fabef6e308d R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffc001f612f R14: 00007fabee5fe300 R15: 0000000000022000 </TASK> NMI backtrace for cpu 0 CPU: 0 PID: 4482 Comm: syz-executor.5 Not tainted 5.18.0-rc3-next-20220422-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: <IRQ> __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111 nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline] rcu_dump_cpu_stacks+0x262/0x3f0 kernel/rcu/tree_stall.h:369 print_cpu_stall kernel/rcu/tree_stall.h:665 [inline] check_cpu_stall kernel/rcu/tree_stall.h:749 [inline] rcu_pending kernel/rcu/tree.c:4068 [inline] rcu_sched_clock_irq.cold+0x144/0x8fc kernel/rcu/tree.c:2755 update_process_times+0x16d/0x200 kernel/time/timer.c:1811 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:243 tick_sched_timer+0xee/0x120 kernel/time/tick-sched.c:1473 __run_hrtimer kernel/time/hrtimer.c:1685 [inline] __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1095 [inline] __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1112 sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1106 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:649 RIP: 0010:kvm_wait+0x98/0x100 arch/x86/kernel/kvm.c:1058 Code: fa 83 e2 07 38 d0 7f 04 84 c0 75 63 0f b6 07 40 38 c6 74 35 48 83 c4 10 c3 c3 e8 23 91 4b 00 eb 07 0f 00 2d da b1 94 08 fb f4 <48> 83 c4 10 c3 89 74 24 0c 48 89 3c 24 e8 56 8f 4b 00 8b 74 24 0c RSP: 0018:ffffc90000007a20 EFLAGS: 00000246 RAX: 0000000000000007 RBX: 0000000000000000 RCX: 1ffffffff1b73199 RDX: 0000000000000000 RSI: ffffffff81807171 RDI: ffffffff8134dffd RBP: ffffffff8cbfc980 R08: 0000000000000000 R09: 0000000000000000 R10: ffffffff81807158 R11: 0000000000000080 R12: 0000000000000000 R13: fffffbfff197f930 R14: 0000000000000001 R15: ffff8880b9c3ae40 pv_wait arch/x86/include/asm/paravirt.h:603 [inline] pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline] __pv_queued_spin_lock_slowpath+0x8c7/0xb50 kernel/locking/qspinlock.c:511 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:591 [inline] queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline] queued_spin_lock include/asm-generic/qspinlock.h:85 [inline] do_raw_spin_lock+0x200/0x2a0 kernel/locking/spinlock_debug.c:115 spin_lock include/linux/spinlock.h:354 [inline] mac80211_hwsim_tx_frame_no_nl.isra.0+0x6f4/0x13b0 drivers/net/wireless/mac80211_hwsim.c:1593 mac80211_hwsim_tx_frame+0x1ee/0x2a0 drivers/net/wireless/mac80211_hwsim.c:1884 mac80211_hwsim_beacon_tx+0x49e/0x920 drivers/net/wireless/mac80211_hwsim.c:1938 __iterate_interfaces+0x1e5/0x560 net/mac80211/util.c:793 ieee80211_iterate_active_interfaces_atomic+0x70/0x180 net/mac80211/util.c:829 mac80211_hwsim_beacon+0xcd/0x1c0 drivers/net/wireless/mac80211_hwsim.c:1961 __run_hrtimer kernel/time/hrtimer.c:1685 [inline] __hrtimer_run_queues+0x609/0xe50 kernel/time/hrtimer.c:1749 hrtimer_run_softirq+0x17b/0x360 kernel/time/hrtimer.c:1766 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558 invoke_softirq kernel/softirq.c:432 [inline] __irq_exit_rcu+0x123/0x180 kernel/softirq.c:637 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1106 </IRQ> <TASK> asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:649 RIP: 0010:pfn_to_bitidx mm/page_alloc.c:469 [inline] RIP: 0010:__get_pfnblock_flags_mask mm/page_alloc.c:482 [inline] RIP: 0010:get_pfnblock_migratetype mm/page_alloc.c:507 [inline] RIP: 0010:free_unref_page_prepare mm/page_alloc.c:3284 [inline] RIP: 0010:free_unref_page_prepare mm/page_alloc.c:3276 [inline] RIP: 0010:free_unref_page+0xe3/0x6a0 mm/page_alloc.c:3376 Code: 05 48 01 c3 48 8d 7b 08 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 c4 04 00 00 4c 89 f0 48 8b 53 08 <48> c1 e8 09 83 e0 3f c1 e0 02 89 c1 48 c1 e8 03 83 e0 18 83 e1 3f RSP: 0018:ffffc900049df920 EFLAGS: 00000246 RAX: 00000000000247a6 RBX: ffff88813fff9080 RCX: 0000000000000000 RDX: ffff88813fff8ba0 RSI: 00000000000000ff RDI: ffff88813fff9088 RBP: ffff88823ffef000 R08: 0000000000000000 R09: ffffed10048f4c00 R10: ffffffff81807158 R11: 000000000000001f R12: ffffea000091e980 R13: 0000000000000000 R14: 00000000000247a6 R15: ffff888022bcc208 __put_single_page mm/swap.c:106 [inline] __put_page+0x145/0x280 mm/swap.c:129 folio_put include/linux/mm.h:1200 [inline] put_page include/linux/mm.h:1233 [inline] generic_pipe_buf_release+0x23a/0x2b0 fs/pipe.c:210 pipe_buf_release include/linux/pipe_fs_i.h:203 [inline] iter_file_splice_write+0x883/0xc70 fs/splice.c:704 do_splice_from fs/splice.c:767 [inline] direct_splice_actor+0x110/0x180 fs/splice.c:936 splice_direct_to_actor+0x34b/0x8c0 fs/splice.c:891 do_splice_direct+0x1a7/0x270 fs/splice.c:979 do_sendfile+0xae0/0x1240 fs/read_write.c:1246 __do_sys_sendfile64 fs/read_write.c:1311 [inline] __se_sys_sendfile64 fs/read_write.c:1297 [inline] __x64_sys_sendfile64+0x1cc/0x210 fs/read_write.c:1297 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x44/0xae RIP: 0033:0x7f8d350890e9 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f8d36106168 EFLAGS: 00000246 ORIG_RAX: 0000000000000028 RAX: ffffffffffffffda RBX: 00007f8d3519bf60 RCX: 00007f8d350890e9 RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000004 RBP: 00007f8d350e308d R08: 0000000000000000 R09: 0000000000000000 R10: 00000000ffffdffa R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffec148342f R14: 00007f8d36106300 R15: 0000000000022000 </TASK> ---------------- Code disassembly (best guess): 0: fa cli 1: 83 e2 07 and $0x7,%edx 4: 38 d0 cmp %dl,%al 6: 7f 04 jg 0xc 8: 84 c0 test %al,%al a: 75 63 jne 0x6f c: 0f b6 07 movzbl (%rdi),%eax f: 40 38 c6 cmp %al,%sil 12: 74 35 je 0x49 14: 48 83 c4 10 add $0x10,%rsp 18: c3 retq 19: c3 retq 1a: e8 23 91 4b 00 callq 0x4b9142 1f: eb 07 jmp 0x28 21: 0f 00 2d da b1 94 08 verw 0x894b1da(%rip) # 0x894b202 28: fb sti 29: f4 hlt * 2a: 48 83 c4 10 add $0x10,%rsp <-- trapping instruction 2e: c3 retq 2f: 89 74 24 0c mov %esi,0xc(%rsp) 33: 48 89 3c 24 mov %rdi,(%rsp) 37: e8 56 8f 4b 00 callq 0x4b8f92 3c: 8b 74 24 0c mov 0xc(%rsp),%esi