IPVS: stopping backup sync thread 3516 ...
IPVS: sync thread started: state = BACKUP, mcast_ifn = veth1, syncid = 2, id = 0
INFO: task kworker/u4:3:262 blocked for more than 120 seconds.
Not tainted 4.4.131-gaa3863d #39
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u4:3 D ffff8801d97e7908 25648 262 2 0x00000000
Workqueue: netns cleanup_net
ffff8801d97e7908 0000000000000003 ffff8801d90808e0 ffffed003b21011b
ffff8801d9080000 ffff8801db31fdb8 ffff8801db31fde0 ffff8801db31f4d8
ffff8801db31f4c0 ffff8800bb95c800 ffff8801d9080000 0000000000000000
Call Trace:
[<ffffffff838b1e4a>] schedule+0x7a/0x1b0 kernel/sched/core.c:3359
[<ffffffff838b25e3>] schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3392
[<ffffffff838b8727>] __mutex_lock_common kernel/locking/mutex.c:582 [inline]
[<ffffffff838b8727>] mutex_lock_nested+0x307/0x850 kernel/locking/mutex.c:621
[<ffffffff82fb10c7>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:70
[<ffffffff83315452>] ip_fib_net_exit+0x22/0x4b0 net/ipv4/fib_frontend.c:1258
[<ffffffff8331594b>] fib_net_exit+0x6b/0x90 net/ipv4/fib_frontend.c:1318
[<ffffffff82f5bd00>] ops_exit_list.isra.6+0xb0/0x160 net/core/net_namespace.c:134
[<ffffffff82f5ecc1>] cleanup_net+0x321/0x600 net/core/net_namespace.c:452
[<ffffffff81181d5f>] process_one_work+0x7df/0x1600 kernel/workqueue.c:2064
[<ffffffff81182c59>] worker_thread+0xd9/0xfc0 kernel/workqueue.c:2196
[<ffffffff81190768>] kthread+0x268/0x300 kernel/kthread.c:211
[<ffffffff838c0295>] ret_from_fork+0x55/0x80 arch/x86/entry/entry_64.S:510
4 locks held by kworker/u4:3/262:
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] work_static include/linux/workqueue.h:186 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] set_work_data kernel/workqueue.c:625 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] set_work_pool_and_clear_pending kernel/workqueue.c:652 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] process_one_work+0x6c0/0x1600 kernel/workqueue.c:2057
#1: (net_cleanup_work){+.+.+.}, at: [<ffffffff81181c7a>] process_one_work+0x6fa/0x1600 kernel/workqueue.c:2061
#2: (net_mutex){+.+.+.}, at: [<ffffffff82f5eadf>] cleanup_net+0x13f/0x600 net/core/net_namespace.c:418
#3: (rtnl_mutex){+.+.+.}, at: [<ffffffff82fb10c7>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:70
Sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.131-gaa3863d #39
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffffffff84417840 task.stack: ffffffff84400000
RIP: 0010:[<ffffffff810cc306>] [<ffffffff810cc306>] native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:49
RSP: 0018:ffffffff84407dc8 EFLAGS: 00000246
RAX: 0000000000000007 RBX: ffffffff84a05ae8 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff8441811c
RBP: ffffffff84407dc8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: fffffbfff074890c R15: ffffffff84a05ae8
FS: 0000000000000000(0000) GS:ffff8801db200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000001a5b000 CR3: 00000000b3366000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff84407e00 ffffffff81025da5 ffffffff84408000 0000000000000000
0000000000000000 fffffbfff074890c ffffffff84a05ae8 ffffffff84407e10
ffffffff810272f0 ffffffff84407e28 ffffffff8121b9e7 dffffc0000000000
Call Trace:
[<ffffffff81025da5>] arch_safe_halt arch/x86/include/asm/paravirt.h:117 [inline]
[<ffffffff81025da5>] default_idle+0x55/0x3c0 arch/x86/kernel/process.c:290
[<ffffffff810272f0>] arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:281
[<ffffffff8121b9e7>] default_idle_call+0x57/0x70 kernel/sched/idle.c:93
[<ffffffff8121c18f>] cpuidle_idle_call kernel/sched/idle.c:157 [inline]
[<ffffffff8121c18f>] cpu_idle_loop kernel/sched/idle.c:253 [inline]
[<ffffffff8121c18f>] cpu_startup_entry+0x6af/0x780 kernel/sched/idle.c:301
[<ffffffff838acfb1>] rest_init+0x188/0x18e init/main.c:410
[<ffffffff84a4b8a1>] start_kernel+0x6b3/0x6e7 init/main.c:682
[<ffffffff84a4a30f>] x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:196
[<ffffffff84a4a450>] x86_64_start_kernel+0x13f/0x162 arch/x86/kernel/head64.c:185
Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
NMI backtrace for cpu 1
CPU: 1 PID: 492 Comm: khungtaskd Not tainted 4.4.131-gaa3863d #39
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff8800baca4800 task.stack: ffff8800baca8000
RIP: 0010:[<ffffffff810bfda6>] [<ffffffff810bfda6>] native_apic_mem_write arch/x86/include/asm/apic.h:94 [inline]
RIP: 0010:[<ffffffff810bfda6>] [<ffffffff810bfda6>] __default_send_IPI_dest_field arch/x86/include/asm/ipi.h:119 [inline]
RIP: 0010:[<ffffffff810bfda6>] [<ffffffff810bfda6>] _flat_send_IPI_mask arch/x86/kernel/apic/apic_flat_64.c:61 [inline]
RIP: 0010:[<ffffffff810bfda6>] [<ffffffff810bfda6>] flat_send_IPI_mask+0xf6/0x1a0 arch/x86/kernel/apic/apic_flat_64.c:69
RSP: 0018:ffff8800bacafcc8 EFLAGS: 00000046
RAX: 0000000003000000 RBX: 0000000000000c00 RCX: 0000000000000000
RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffffffffff5fb300
RBP: ffff8800bacafcf0 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000246
R13: 0000000000000003 R14: 0000000000000002 R15: ffffffff8446f7a0
FS: 0000000000000000(0000) GS:ffff8801db300000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c420145e80 CR3: 00000000b3366000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff8446f7a0 ffffffff84a18a20 0000000000000007 fffffbfff0942bf4
0000000000000040 ffff8800bacafd10 ffffffff810b5d61 ffffffff83c0b1a0
0000000000000003 ffff8800bacafd68 ffffffff81e19003 ffffffff8140d96f
Call Trace:
[<ffffffff810b5d61>] nmi_raise_cpu_backtrace+0x61/0x80 arch/x86/kernel/apic/hw_nmi.c:33
[<ffffffff81e19003>] nmi_trigger_all_cpu_backtrace.cold.4+0x70/0xad lib/nmi_backtrace.c:85
[<ffffffff810b5e04>] arch_trigger_all_cpu_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
[<ffffffff8141a059>] trigger_all_cpu_backtrace include/linux/nmi.h:44 [inline]
[<ffffffff8141a059>] check_hung_task kernel/hung_task.c:125 [inline]
[<ffffffff8141a059>] check_hung_uninterruptible_tasks kernel/hung_task.c:182 [inline]
[<ffffffff8141a059>] watchdog.cold.1+0xd3/0xee kernel/hung_task.c:238
[<ffffffff81190768>] kthread+0x268/0x300 kernel/kthread.c:211
[<ffffffff838c0295>] ret_from_fork+0x55/0x80 arch/x86/entry/entry_64.S:510
Code: b3 5f ff f6 c4 10 75 e2 44 89 e8 c1 e0 18 89 04 25 10 b3 5f ff 44 89 f2 09 da 80 cf 04 41 83 fe 02 0f 44 d3 89 14 25 00 b3 5f ff <41> f7 c4 00 02 00 00 75 1a 4c 89 e7 57 9d 0f 1f 44 00 00 e8 62