INFO: task kworker/u4:1:19 blocked for more than 120 seconds.
Not tainted 4.4.129-g23eaecc #26
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u4:1 D ffff8801d9b97968 27248 19 2 0x00000000
Workqueue: netns cleanup_net
ffff8801d9b97968 0000000000000003 ffff8801d9aab8e0 ffffed003b35571b
ffff8801d9aab000 ffff8801db31fdb8 ffff8801db31fde0 ffff8801db31f4d8
ffff8801db31f4c0 ffff8801d9a41800 ffff8801d9aab000 0000000000000000
Call Trace:
[<ffffffff838b10ca>] schedule+0x7a/0x1b0 kernel/sched/core.c:3359
[<ffffffff838b1863>] schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3392
[<ffffffff838b79a7>] __mutex_lock_common kernel/locking/mutex.c:582 [inline]
[<ffffffff838b79a7>] mutex_lock_nested+0x307/0x850 kernel/locking/mutex.c:621
[<ffffffff82fb0627>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:70
[<ffffffff833556f0>] ipmr_rules_exit net/ipv4/ipmr.c:310 [inline]
[<ffffffff833556f0>] ipmr_net_exit+0x80/0x120 net/ipv4/ipmr.c:2742
[<ffffffff82f5b3e0>] ops_exit_list.isra.6+0xb0/0x160 net/core/net_namespace.c:134
[<ffffffff82f5e3a1>] cleanup_net+0x321/0x600 net/core/net_namespace.c:452
[<ffffffff81181d5f>] process_one_work+0x7df/0x1600 kernel/workqueue.c:2064
[<ffffffff81182c59>] worker_thread+0xd9/0xfc0 kernel/workqueue.c:2196
[<ffffffff81190768>] kthread+0x268/0x300 kernel/kthread.c:211
[<ffffffff838bf515>] ret_from_fork+0x55/0x80 arch/x86/entry/entry_64.S:510
4 locks held by kworker/u4:1/19:
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] work_static include/linux/workqueue.h:186 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] set_work_data kernel/workqueue.c:625 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] set_work_pool_and_clear_pending kernel/workqueue.c:652 [inline]
#0: ("%s""netns"){.+.+.+}, at: [<ffffffff81181c40>] process_one_work+0x6c0/0x1600 kernel/workqueue.c:2057
#1: (net_cleanup_work){+.+.+.}, at: [<ffffffff81181c7a>] process_one_work+0x6fa/0x1600 kernel/workqueue.c:2061
#2: (net_mutex){+.+.+.}, at: [<ffffffff82f5e1bf>] cleanup_net+0x13f/0x600 net/core/net_namespace.c:418
#3: (rtnl_mutex){+.+.+.}, at: [<ffffffff82fb0627>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:70
Sending NMI to all CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 492 Comm: khungtaskd Not tainted 4.4.129-g23eaecc #26
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff8801d8c51800 task.stack: ffff8800bace8000
RIP: 0010:[<ffffffff810bfd66>] [<ffffffff810bfd66>] native_apic_mem_write arch/x86/include/asm/apic.h:94 [inline]
RIP: 0010:[<ffffffff810bfd66>] [<ffffffff810bfd66>] __default_send_IPI_dest_field arch/x86/include/asm/ipi.h:119 [inline]
RIP: 0010:[<ffffffff810bfd66>] [<ffffffff810bfd66>] _flat_send_IPI_mask arch/x86/kernel/apic/apic_flat_64.c:61 [inline]
RIP: 0010:[<ffffffff810bfd66>] [<ffffffff810bfd66>] flat_send_IPI_mask+0xf6/0x1a0 arch/x86/kernel/apic/apic_flat_64.c:69
RSP: 0018:ffff8800bacefcc8 EFLAGS: 00000046
RAX: 0000000003000000 RBX: 0000000000000c00 RCX: 0000000000000000
RDX: 0000000000000c00 RSI: 0000000000000000 RDI: ffffffffff5fb300
RBP: ffff8800bacefcf0 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000246
R13: 0000000000000003 R14: 0000000000000002 R15: ffffffff8446f7a0
FS: 0000000000000000(0000) GS:ffff8801db200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fac619829b8 CR3: 00000000b5823000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff8446f7a0 ffffffff84a187e0 0000000000000007 fffffbfff0942bac
0000000000000040 ffff8800bacefd10 ffffffff810b5d21 ffffffff83c0b0a0
0000000000000003 ffff8800bacefd68 ffffffff81e18d23 ffffffff8140dabf
Call Trace:
[<ffffffff810b5d21>] nmi_raise_cpu_backtrace+0x61/0x80 arch/x86/kernel/apic/hw_nmi.c:33
[<ffffffff81e18d23>] nmi_trigger_all_cpu_backtrace.cold.4+0x70/0xad lib/nmi_backtrace.c:85
[<ffffffff810b5dc4>] arch_trigger_all_cpu_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
[<ffffffff8141a1a9>] trigger_all_cpu_backtrace include/linux/nmi.h:44 [inline]
[<ffffffff8141a1a9>] check_hung_task kernel/hung_task.c:125 [inline]
[<ffffffff8141a1a9>] check_hung_uninterruptible_tasks kernel/hung_task.c:182 [inline]
[<ffffffff8141a1a9>] watchdog.cold.1+0xd3/0xee kernel/hung_task.c:238
[<ffffffff81190768>] kthread+0x268/0x300 kernel/kthread.c:211
[<ffffffff838bf515>] ret_from_fork+0x55/0x80 arch/x86/entry/entry_64.S:510
Code: b3 5f ff f6 c4 10 75 e2 44 89 e8 c1 e0 18 89 04 25 10 b3 5f ff 44 89 f2 09 da 80 cf 04 41 83 fe 02 0f 44 d3 89 14 25 00 b3 5f ff <41> f7 c4 00 02 00 00 75 1a 4c 89 e7 57 9d 0f 1f 44 00 00 e8 a2
NMI backtrace for cpu 1
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.4.129-g23eaecc #26
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff8801d9a41800 task.stack: ffff8801d9a50000
RIP: 0010:[<ffffffff810cc306>] [<ffffffff810cc306>] native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:49
RSP: 0018:ffff8801d9a57da0 EFLAGS: 00000246
RAX: 0000000000000007 RBX: ffffffff84a058a8 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8801d9a420dc
RBP: ffff8801d9a57da0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000001 R14: fffffbfff074890c R15: ffffffff84a058a8
FS: 0000000000000000(0000) GS:ffff8801db300000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007feebef21000 CR3: 00000001cc179000 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff8801d9a57dd8 ffffffff81025da5 ffff8801d9a58000 0000000000000000
0000000000000000 fffffbfff074890c ffffffff84a058a8 ffff8801d9a57de8
ffffffff810272f0 ffff8801d9a57e00 ffffffff8121b9e7 dffffc0000000000
Call Trace:
[<ffffffff81025da5>] arch_safe_halt arch/x86/include/asm/paravirt.h:117 [inline]
[<ffffffff81025da5>] default_idle+0x55/0x3c0 arch/x86/kernel/process.c:290
[<ffffffff810272f0>] arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:281
[<ffffffff8121b9e7>] default_idle_call+0x57/0x70 kernel/sched/idle.c:93
[<ffffffff8121c18f>] cpuidle_idle_call kernel/sched/idle.c:157 [inline]
[<ffffffff8121c18f>] cpu_idle_loop kernel/sched/idle.c:253 [inline]
[<ffffffff8121c18f>] cpu_startup_entry+0x6af/0x780 kernel/sched/idle.c:301
[<ffffffff810aa174>] start_secondary+0x324/0x400 arch/x86/kernel/smpboot.c:242
Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84