sched: RT throttling activated rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P3781/1:b..l P3029/1:b..l (detected by 0, t=10502 jiffies, g=13645, q=529) task:udevd state:R running task stack:24256 pid: 3029 ppid: 1 flags:0x00004002 Call Trace: context_switch kernel/sched/core.c:5027 [inline] __schedule+0x12c4/0x45b0 kernel/sched/core.c:6373 preempt_schedule_common+0x83/0xd0 kernel/sched/core.c:6549 preempt_schedule+0xd9/0xe0 kernel/sched/core.c:6574 preempt_schedule_thunk+0x16/0x18 arch/x86/entry/thunk_64.S:34 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:161 [inline] _raw_spin_unlock_irqrestore+0x128/0x130 kernel/locking/spinlock.c:194 spin_unlock_irqrestore include/linux/spinlock.h:418 [inline] __wake_up_common_lock kernel/sched/wait.c:140 [inline] __wake_up_sync_key+0x121/0x1c0 kernel/sched/wait.c:205 sock_def_readable+0x135/0x240 net/core/sock.c:3092 __netlink_sendskb net/netlink/af_netlink.c:1273 [inline] netlink_sendskb+0x8e/0x120 net/netlink/af_netlink.c:1279 netlink_unicast+0x3a1/0x980 net/netlink/af_netlink.c:1367 netlink_sendmsg+0xa30/0xd60 net/netlink/af_netlink.c:1920 sock_sendmsg_nosec net/socket.c:704 [inline] __sock_sendmsg net/socket.c:716 [inline] ____sys_sendmsg+0x59e/0x8f0 net/socket.c:2431 ___sys_sendmsg+0x252/0x2e0 net/socket.c:2485 __sys_sendmsg net/socket.c:2514 [inline] __do_sys_sendmsg net/socket.c:2523 [inline] __se_sys_sendmsg+0x19a/0x260 net/socket.c:2521 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x66/0xd0 RIP: 0033:0x7fc5e3868a4b RSP: 002b:00007fff5de4dc78 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 000055895c31ddf0 RCX: 00007fc5e3868a4b RDX: 0000000000000000 RSI: 00007fff5de4dc88 RDI: 0000000000000004 RBP: 000055895c3450a0 R08: 0000000000000001 R09: 0000000000000000 R10: 000000000000010f R11: 0000000000000246 R12: 0000000000000000 R13: 00000000000000b4 R14: 0000000000000000 R15: 0000000000000000 task:kworker/0:11 state:R running task stack:25024 pid: 3781 ppid: 2 flags:0x00004000 Workqueue: events_power_efficient gc_worker Call Trace: context_switch kernel/sched/core.c:5027 [inline] __schedule+0x12c4/0x45b0 kernel/sched/core.c:6373 preempt_schedule_irq+0xf7/0x1c0 kernel/sched/core.c:6777 irqentry_exit+0x53/0x80 kernel/entry/common.c:432 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638 RIP: 0010:debug_lockdep_rcu_enabled+0x25/0x30 kernel/rcu/update.c:281 Code: cc cc cc cc cc 31 c0 83 3d 2b 39 b5 03 00 74 1d 83 3d a2 6b b5 03 00 74 14 65 48 8b 0d 94 4c ce 75 31 c0 83 b9 ec 0a 00 00 00 <0f> 94 c0 c3 cc cc cc cc cc cc cc 41 56 53 89 fb e8 d6 0d 00 00 41 RSP: 0018:ffffc90003297af8 EFLAGS: 00000246 RAX: 0000000000000000 RBX: ffff888028bc46b8 RCX: ffff88807e5c9dc0 RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000 RBP: ffffc90003297c50 R08: ffffffff8890d905 R09: fffffbfff20e222b R10: 0000000000000000 R11: dffffc0000000001 R12: 00000000000711af R13: dffffc0000000000 R14: ffff888028a00000 R15: 0000000000004000 rcu_read_unlock include/linux/rcupdate.h:768 [inline] gc_worker+0xccf/0x14b0 net/netfilter/nf_conntrack_core.c:1543 process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310 worker_thread+0xaca/0x1280 kernel/workqueue.c:2457 kthread+0x3f6/0x4f0 kernel/kthread.c:334 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287 rcu: rcu_preempt kthread starved for 9820 jiffies! g13645 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack:27256 pid: 15 ppid: 2 flags:0x00004000 Call Trace: context_switch kernel/sched/core.c:5027 [inline] __schedule+0x12c4/0x45b0 kernel/sched/core.c:6373 schedule+0x11b/0x1f0 kernel/sched/core.c:6456 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1914 rcu_gp_fqs_loop+0x2bf/0x1080 kernel/rcu/tree.c:1972 rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2145 kthread+0x3f6/0x4f0 kernel/kthread.c:334 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287 rcu: Stack dump where RCU GP kthread last ran: Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 skipped: idling at native_safe_halt arch/x86/include/asm/irqflags.h:51 [inline] NMI backtrace for cpu 1 skipped: idling at arch_safe_halt arch/x86/include/asm/irqflags.h:89 [inline] NMI backtrace for cpu 1 skipped: idling at acpi_safe_halt drivers/acpi/processor_idle.c:108 [inline] NMI backtrace for cpu 1 skipped: idling at acpi_idle_do_entry+0x10f/0x340 drivers/acpi/processor_idle.c:562