rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: 1-...!: (1 GPs behind) idle=7a5c/1/0x4000000000000000 softirq=20758/20759 fqs=2 rcu: (detected by 0, t=10506 jiffies, g=26457, q=380 ncpus=2) Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 29 Comm: kauditd Not tainted 6.10.0-rc1-syzkaller-00296-g89be4025b0db #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024 RIP: 0010:__preempt_count_add kernel/rcu/tree.c:720 [inline] RIP: 0010:rcu_is_watching+0x6/0xb0 kernel/rcu/tree.c:723 Code: 6c 03 e9 2b ff ff ff e8 18 00 0a 0a 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 41 57 <41> 56 53 65 ff 05 58 8e 87 7e e8 4b 1e 0a 0a 89 c3 83 f8 08 73 7a RSP: 0018:ffffc90000a18d18 EFLAGS: 00000002 RAX: 0000000000010002 RBX: 0000000000000001 RCX: ffffffff81839fc6 RDX: ffff888019299e00 RSI: 0000000000000001 RDI: 0000000000000000 RBP: 0000000000000001 R08: ffffffff81839fec R09: 1ffffffff1f5a8ad R10: dffffc0000000000 R11: fffffbfff1f5a8ae R12: dffffc0000000000 R13: ffff8880b952c980 R14: ffff888079642340 R15: ffff8880b952c880 FS: 0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000561cf3155348 CR3: 000000002cc52000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: trace_hrtimer_start include/trace/events/timer.h:222 [inline] debug_activate kernel/time/hrtimer.c:479 [inline] enqueue_hrtimer+0xa5/0x3c0 kernel/time/hrtimer.c:1085 __run_hrtimer kernel/time/hrtimer.c:1704 [inline] __hrtimer_run_queues+0x6cb/0xd50 kernel/time/hrtimer.c:1751 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1813 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline] __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1049 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline] sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1043 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 RIP: 0010:console_flush_all+0xaad/0xfd0 kernel/printk/printk.c:3000 Code: ff ff e8 16 d0 1f 00 90 0f 0b 90 e9 d8 f8 ff ff e8 08 d0 1f 00 e8 43 01 10 0a 4d 85 f6 74 b6 e8 f9 cf 1f 00 fb 48 8b 44 24 70 <42> 0f b6 04 38 84 c0 48 8b 7c 24 30 0f 85 22 02 00 00 0f b6 1f 31 RSP: 0018:ffffc90000a577e0 EFLAGS: 00000293 RAX: 1ffff9200014af48 RBX: 0000000000000000 RCX: ffff888019299e00 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffffc90000a57990 R08: ffffffff81764624 R09: 1ffffffff25f50b0 R10: dffffc0000000000 R11: fffffbfff25f50b1 R12: ffffffff8eb22518 R13: ffffffff8eb224c0 R14: 0000000000000200 R15: dffffc0000000000 console_unlock+0x13b/0x4d0 kernel/printk/printk.c:3063 vprintk_emit+0x5a6/0x770 kernel/printk/printk.c:2345 _printk+0xd5/0x120 kernel/printk/printk.c:2370 kauditd_printk_skb kernel/audit.c:546 [inline] kauditd_hold_skb+0x1bc/0x210 kernel/audit.c:581 kauditd_send_queue+0x2b1/0x310 kernel/audit.c:766 kauditd_thread+0x74a/0x9b0 kernel/audit.c:890 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 rcu: rcu_preempt kthread starved for 10480 jiffies! g26457 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack:25840 pid:17 tgid:17 ppid:2 flags:0x00004000 Call Trace: context_switch kernel/sched/core.c:5408 [inline] __schedule+0x17e8/0x4a20 kernel/sched/core.c:6745 __schedule_loop kernel/sched/core.c:6822 [inline] schedule+0x14b/0x320 kernel/sched/core.c:6837 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2581 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2000 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2202 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 rcu: Stack dump where RCU GP kthread last ran: CPU: 0 PID: 2874 Comm: kworker/u8:10 Not tainted 6.10.0-rc1-syzkaller-00296-g89be4025b0db #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024 Workqueue: events_unbound toggle_allocation_gate RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline] RIP: 0010:smp_call_function_many_cond+0x1860/0x29d0 kernel/smp.c:855 Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 99 09 0c 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 44 05 0c 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 28 05 RSP: 0018:ffffc90009fef700 EFLAGS: 00000293 RAX: ffffffff818a1158 RBX: 1ffff110172a88a9 RCX: ffff88802bd71e00 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 RBP: ffffc90009fef8e0 R08: ffffffff818a1127 R09: 1ffffffff25f50b0 R10: dffffc0000000000 R11: fffffbfff25f50b1 R12: dffffc0000000000 R13: ffff8880b9544548 R14: ffff8880b943f900 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000001b2dd27000 CR3: 000000000e132000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1023 on_each_cpu include/linux/smp.h:71 [inline] text_poke_sync arch/x86/kernel/alternative.c:2069 [inline] text_poke_bp_batch+0x726/0xb30 arch/x86/kernel/alternative.c:2341 text_poke_flush arch/x86/kernel/alternative.c:2470 [inline] text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2477 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205 static_key_enable+0x1a/0x20 kernel/jump_label.c:218 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826 process_one_work kernel/workqueue.c:3231 [inline] process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 watchdog: BUG: soft lockup - CPU#0 stuck for 246s! [kworker/u8:10:2874] CPU#0 Utilization every 22s during lockup: #1: 99% system, 1% softirq, 1% hardirq, 0% idle #2: 99% system, 1% softirq, 1% hardirq, 0% idle #3: 99% system, 1% softirq, 1% hardirq, 0% idle #4: 99% system, 1% softirq, 1% hardirq, 0% idle #5: 99% system, 1% softirq, 1% hardirq, 0% idle Modules linked in: irq event stamp: 4925188 hardirqs last enabled at (4925187): [] irqentry_exit+0x63/0x90 kernel/entry/common.c:357 hardirqs last disabled at (4925188): [] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1043 softirqs last enabled at (4925186): [] __do_softirq kernel/softirq.c:588 [inline] softirqs last enabled at (4925186): [] invoke_softirq kernel/softirq.c:428 [inline] softirqs last enabled at (4925186): [] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637 softirqs last disabled at (4925167): [] __do_softirq kernel/softirq.c:588 [inline] softirqs last disabled at (4925167): [] invoke_softirq kernel/softirq.c:428 [inline] softirqs last disabled at (4925167): [] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637 CPU: 0 PID: 2874 Comm: kworker/u8:10 Not tainted 6.10.0-rc1-syzkaller-00296-g89be4025b0db #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024 Workqueue: events_unbound toggle_allocation_gate RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline] RIP: 0010:smp_call_function_many_cond+0x1860/0x29d0 kernel/smp.c:855 Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 99 09 0c 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 44 05 0c 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 28 05 RSP: 0018:ffffc90009fef700 EFLAGS: 00000293 RAX: ffffffff818a1158 RBX: 1ffff110172a88a9 RCX: ffff88802bd71e00 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 RBP: ffffc90009fef8e0 R08: ffffffff818a1127 R09: 1ffffffff25f50b0 R10: dffffc0000000000 R11: fffffbfff25f50b1 R12: dffffc0000000000 R13: ffff8880b9544548 R14: ffff8880b943f900 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000001b2dd27000 CR3: 000000000e132000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1023 on_each_cpu include/linux/smp.h:71 [inline] text_poke_sync arch/x86/kernel/alternative.c:2069 [inline] text_poke_bp_batch+0x726/0xb30 arch/x86/kernel/alternative.c:2341 text_poke_flush arch/x86/kernel/alternative.c:2470 [inline] text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2477 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205 static_key_enable+0x1a/0x20 kernel/jump_label.c:218 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826 process_one_work kernel/workqueue.c:3231 [inline] process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312 worker_thread+0x86d/0xd70 kernel/workqueue.c:3393 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 29 Comm: kauditd Not tainted 6.10.0-rc1-syzkaller-00296-g89be4025b0db #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024 RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline] RIP: 0010:write_comp_data kernel/kcov.c:236 [inline] RIP: 0010:__sanitizer_cov_trace_const_cmp1+0x18/0x90 kernel/kcov.c:290 Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 4c 8b 04 24 65 48 8b 14 25 00 d5 03 00 65 8b 05 70 bb 6d 7e 00 01 ff 00 74 10 a9 00 01 00 00 74 5b 83 ba 1c 16 00 00 00 74 RSP: 0018:ffffc90000a18d58 EFLAGS: 00000046 RAX: 0000000000010001 RBX: 0000000000000000 RCX: dffffc0000000000 RDX: ffff888019299e00 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffffc90000a18ea8 R08: ffffffff8183729f R09: 1ffffffff1f5a8ad R10: dffffc0000000000 R11: fffffbfff1f5a8ae R12: 1ffff110172a593b R13: ffffffff898fdf00 R14: ffff888079642340 R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000561cf3155348 CR3: 000000002cc52000 CR4: 00000000003506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: __run_hrtimer kernel/time/hrtimer.c:1685 [inline] __hrtimer_run_queues+0x53f/0xd50 kernel/time/hrtimer.c:1751 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1813 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline] __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1049 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline] sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1043 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702 RIP: 0010:console_flush_all+0xaad/0xfd0 kernel/printk/printk.c:3000 Code: ff ff e8 16 d0 1f 00 90 0f 0b 90 e9 d8 f8 ff ff e8 08 d0 1f 00 e8 43 01 10 0a 4d 85 f6 74 b6 e8 f9 cf 1f 00 fb 48 8b 44 24 70 <42> 0f b6 04 38 84 c0 48 8b 7c 24 30 0f 85 22 02 00 00 0f b6 1f 31 RSP: 0018:ffffc90000a577e0 EFLAGS: 00000293 RAX: 1ffff9200014af48 RBX: 0000000000000000 RCX: ffff888019299e00 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffffc90000a57990 R08: ffffffff81764624 R09: 1ffffffff25f50b0 R10: dffffc0000000000 R11: fffffbfff25f50b1 R12: ffffffff8eb22518 R13: ffffffff8eb224c0 R14: 0000000000000200 R15: dffffc0000000000 console_unlock+0x13b/0x4d0 kernel/printk/printk.c:3063 vprintk_emit+0x5a6/0x770 kernel/printk/printk.c:2345 _printk+0xd5/0x120 kernel/printk/printk.c:2370 kauditd_printk_skb kernel/audit.c:546 [inline] kauditd_hold_skb+0x1bc/0x210 kernel/audit.c:581 kauditd_send_queue+0x2b1/0x310 kernel/audit.c:766 kauditd_thread+0x74a/0x9b0 kernel/audit.c:890 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244