IPVS: Scheduler module ip_vs_f not found INFO: rcu_sched self-detected stall on CPU 0-...!: (124999 ticks this GP) idle=46a/1/4611686018427387908 softirq=32377/32377 fqs=10 (t=125000 jiffies g=17696 c=17695 q=374137) rcu_sched kthread starved for 124840 jiffies! g17696 c17695 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1 RCU grace-period kthread stack dump: rcu_sched R running task 23592 9 2 0x80000000 Call Trace: context_switch kernel/sched/core.c:2859 [inline] __schedule+0x801/0x1e30 kernel/sched/core.c:3501 schedule+0xef/0x430 kernel/sched/core.c:3545 schedule_timeout+0x138/0x240 kernel/time/timer.c:1801 rcu_gp_kthread+0x6b5/0x1940 kernel/rcu/tree.c:2231 kthread+0x345/0x410 kernel/kthread.c:240 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412 NMI backtrace for cpu 0 CPU: 0 PID: 9469 Comm: udevd Not tainted 4.17.0-rc6+ #70 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 nmi_cpu_backtrace.cold.4+0x19/0xce lib/nmi_backtrace.c:103 nmi_trigger_cpumask_backtrace+0x151/0x192 lib/nmi_backtrace.c:62 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38 trigger_single_cpu_backtrace include/linux/nmi.h:156 [inline] rcu_dump_cpu_stacks+0x175/0x1c2 kernel/rcu/tree.c:1376 print_cpu_stall kernel/rcu/tree.c:1525 [inline] check_cpu_stall.isra.61.cold.80+0x36c/0x59a kernel/rcu/tree.c:1593 __rcu_pending kernel/rcu/tree.c:3356 [inline] rcu_pending kernel/rcu/tree.c:3401 [inline] rcu_check_callbacks+0x21b/0xad0 kernel/rcu/tree.c:2763 update_process_times+0x2d/0x70 kernel/time/timer.c:1636 tick_sched_handle+0x9f/0x180 kernel/time/tick-sched.c:164 tick_sched_timer+0x45/0x130 kernel/time/tick-sched.c:1274 __run_hrtimer kernel/time/hrtimer.c:1398 [inline] __hrtimer_run_queues+0x3e3/0x10a0 kernel/time/hrtimer.c:1460 hrtimer_interrupt+0x2f3/0x750 kernel/time/hrtimer.c:1518 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1025 [inline] smp_apic_timer_interrupt+0x15d/0x710 arch/x86/kernel/apic/apic.c:1050 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863 RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:783 [inline] RIP: 0010:kfree+0x124/0x260 mm/slab.c:3814 RSP: 0018:ffff8801dae06590 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13 RAX: 0000000000000007 RBX: ffff880128e08780 RCX: 1ffff10036a6bdd5 RDX: 0000000000000000 RSI: ffff8801b535ee88 RDI: 0000000000000286 RBP: ffff8801dae065b0 R08: ffff8801b535ee38 R09: 0000000000000002 R10: ffff8801b535e600 R11: 0000000000000000 R12: 0000000000000286 R13: ffff8801da800940 R14: ffffffff85d90bf9 R15: ffff880128e070c0 skb_free_head+0x99/0xc0 net/core/skbuff.c:550 skb_release_data+0x690/0x860 net/core/skbuff.c:570 skb_release_all+0x4a/0x60 net/core/skbuff.c:627 __kfree_skb net/core/skbuff.c:641 [inline] consume_skb+0x18b/0x550 net/core/skbuff.c:701 sctp_chunk_destroy net/sctp/sm_make_chunk.c:1462 [inline] sctp_chunk_put+0x2c0/0x440 net/sctp/sm_make_chunk.c:1489 sctp_chunk_free+0x53/0x60 net/sctp/sm_make_chunk.c:1476 sctp_packet_pack net/sctp/output.c:489 [inline] sctp_packet_transmit+0x142e/0x3ba0 net/sctp/output.c:610 sctp_outq_flush_transports net/sctp/outqueue.c:1164 [inline] sctp_outq_flush+0x5f5/0x3430 net/sctp/outqueue.c:1212 sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776 sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline] sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline] sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191 sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406 call_timer_fn+0x230/0x940 kernel/time/timer.c:1326 expire_timers kernel/time/timer.c:1363 [inline] __run_timers+0x79e/0xc50 kernel/time/timer.c:1666 run_timer_softirq+0x4c/0x70 kernel/time/timer.c:1692 __do_softirq+0x2e0/0xaf5 kernel/softirq.c:285 invoke_softirq kernel/softirq.c:365 [inline] irq_exit+0x1d1/0x200 kernel/softirq.c:405 exiting_irq arch/x86/include/asm/apic.h:525 [inline] smp_apic_timer_interrupt+0x17e/0x710 arch/x86/kernel/apic/apic.c:1052 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863 RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:783 [inline] RIP: 0010:lock_is_held_type+0x18b/0x210 kernel/locking/lockdep.c:3960 RSP: 0018:ffff88019314f558 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13 RAX: dffffc0000000000 RBX: 0000000000000286 RCX: ffffffff813557fd RDX: 1ffffffff11a31ad RSI: ffffffff88d8e620 RDI: 0000000000000286 RBP: ffff88019314f578 R08: ffff8801b535e600 R09: 0000000000000000 R10: ffff8801b535ee38 R11: ffff8801b535e600 R12: ffff8801b535e600 R13: 0000000000000000 R14: 000000000000053e R15: 0000000000000003 lock_is_held include/linux/lockdep.h:344 [inline] ___might_sleep+0x2c0/0x320 kernel/sched/core.c:6149 __might_sleep+0x95/0x190 kernel/sched/core.c:6137 __do_page_fault+0x3ae/0xe40 arch/x86/mm/fault.c:1342 do_page_fault+0xee/0x8a7 arch/x86/mm/fault.c:1474 page_fault+0x1e/0x30 arch/x86/entry/entry_64.S:1160 RIP: 0010:__clear_user+0x42/0x70 arch/x86/lib/usercopy_64.c:23 RSP: 0018:ffff88019314f950 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 0000000000000010 RCX: 0000000000000010 RDX: 0000000000000000 RSI: 0000000000000008 RDI: 00007ffe83a21b40 RBP: ffff88019314f960 R08: 1ffff10032629f06 R09: 0000000000000000 R10: 0000000000000003 R11: 0000000000000000 R12: 00007ffe83a21b40 R13: 0000000000000011 R14: ffff8801b535e600 R15: 0000000000000800 signalfd_copyinfo fs/signalfd.c:91 [inline] signalfd_read+0x41b/0x1050 fs/signalfd.c:226 __vfs_read+0x10f/0xa50 fs/read_write.c:416 vfs_read+0x17f/0x3d0 fs/read_write.c:452 ksys_read+0xf9/0x250 fs/read_write.c:578 __do_sys_read fs/read_write.c:588 [inline] __se_sys_read fs/read_write.c:586 [inline] __x64_sys_read+0x73/0xb0 fs/read_write.c:586 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x7f06e061e310 RSP: 002b:00007ffe83a20238 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: ffffffffffffffda RBX: 20c49ba5e353f7cf RCX: 00007f06e061e310 RDX: 0000000000000080 RSI: 00007ffe83a21b40 RDI: 0000000000000003 RBP: 0000000000000000 R08: 00007ffe83ad501d R09: 0000002199e7c9ac R10: 001038824d800000 R11: 0000000000000246 R12: 0000000001cfa110 R13: 000000000000265a R14: 00007ffe83a202a4 R15: 0000000001cfa250 BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 126s! Showing busy workqueues and worker pools: workqueue events: flags=0x0 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 in-flight: 24:rht_deferred_worker pending: cache_reap pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=8/256 pending: defense_work_handler, defense_work_handler, defense_work_handler, defense_work_handler, defense_work_handler, check_corruption, vmstat_shepherd, cache_reap workqueue events_power_efficient: flags=0x80 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=3/256 pending: check_lifetime, gc_worker, neigh_periodic_work workqueue mm_percpu_wq: flags=0x8 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 pending: vmstat_update pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 pending: vmstat_update workqueue netns: flags=0xe000a pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/1 in-flight: 6732:cleanup_net delayed: cleanup_net workqueue writeback: flags=0x4e pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/256 in-flight: 8277:wb_workfn workqueue ipv6_addrconf: flags=0x40008 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/1 pending: addrconf_verify_work workqueue krdsd: flags=0xe000a pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/1 in-flight: 62:rds_connect_worker pool 2: cpus=1 node=0 flags=0x0 nice=0 hung=0s workers=5 idle: 2106 1979 5266 18 pool 4: cpus=0-1 flags=0x4 nice=0 hung=0s workers=9 idle: 278 6742 6683 22 6 6719