xs_tcp_setup_socket: connect returned unhandled error -113 xs_tcp_setup_socket: connect returned unhandled error -113 xs_tcp_setup_socket: connect returned unhandled error -113 ====================================================== WARNING: possible circular locking dependency detected 4.14.0-rc5+ #137 Not tainted ------------------------------------------------------ kworker/0:0/3 is trying to acquire lock: ("xprtiod"){+.+.}, at: [] start_flush_work kernel/workqueue.c:2857 [inline] ("xprtiod"){+.+.}, at: [] flush_work+0x55a/0x8a0 kernel/workqueue.c:2889 but task is already holding lock: ((&task->u.tk_work)){+.+.}, at: [] process_one_work+0xb32/0x1bc0 kernel/workqueue.c:2094 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 ((&task->u.tk_work)){+.+.}: lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:3994 process_one_work+0xba2/0x1bc0 kernel/workqueue.c:2095 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x35e/0x430 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 -> #0 ("xprtiod"){+.+.}: check_prev_add kernel/locking/lockdep.c:1902 [inline] check_prevs_add kernel/locking/lockdep.c:2019 [inline] validate_chain kernel/locking/lockdep.c:2461 [inline] __lock_acquire+0x2f55/0x3d50 kernel/locking/lockdep.c:3490 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:3994 start_flush_work kernel/workqueue.c:2858 [inline] flush_work+0x57f/0x8a0 kernel/workqueue.c:2889 __cancel_work_timer+0x30a/0x7e0 kernel/workqueue.c:2961 cancel_work_sync+0x17/0x20 kernel/workqueue.c:2997 xprt_destroy+0xa1/0x130 net/sunrpc/xprt.c:1467 xprt_destroy_kref net/sunrpc/xprt.c:1477 [inline] kref_put include/linux/kref.h:70 [inline] xprt_put+0x38/0x40 net/sunrpc/xprt.c:1501 rpc_task_release_client+0x299/0x430 net/sunrpc/clnt.c:986 rpc_release_resources_task+0x7f/0xa0 net/sunrpc/sched.c:1020 rpc_release_task net/sunrpc/sched.c:1059 [inline] __rpc_execute+0x4d9/0xe70 net/sunrpc/sched.c:824 rpc_async_schedule+0x16/0x20 net/sunrpc/sched.c:848 process_one_work+0xbf0/0x1bc0 kernel/workqueue.c:2119 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x35e/0x430 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock((&task->u.tk_work)); lock("xprtiod"); lock((&task->u.tk_work)); lock("xprtiod"); *** DEADLOCK *** 2 locks held by kworker/0:0/3: #0: ("rpciod"){+.+.}, at: [] __write_once_size include/linux/compiler.h:305 [inline] #0: ("rpciod"){+.+.}, at: [] atomic64_set arch/x86/include/asm/atomic64_64.h:33 [inline] #0: ("rpciod"){+.+.}, at: [] atomic_long_set include/asm-generic/atomic-long.h:56 [inline] #0: ("rpciod"){+.+.}, at: [] set_work_data kernel/workqueue.c:617 [inline] #0: ("rpciod"){+.+.}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline] #0: ("rpciod"){+.+.}, at: [] process_one_work+0xadf/0x1bc0 kernel/workqueue.c:2090 #1: ((&task->u.tk_work)){+.+.}, at: [] process_one_work+0xb32/0x1bc0 kernel/workqueue.c:2094 stack backtrace: CPU: 0 PID: 3 Comm: kworker/0:0 Not tainted 4.14.0-rc5+ #137 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: rpciod rpc_async_schedule Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 print_circular_bug.isra.42+0x342/0x36a kernel/locking/lockdep.c:1259 check_prev_add kernel/locking/lockdep.c:1902 [inline] check_prevs_add kernel/locking/lockdep.c:2019 [inline] validate_chain kernel/locking/lockdep.c:2461 [inline] __lock_acquire+0x2f55/0x3d50 kernel/locking/lockdep.c:3490 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:3994 start_flush_work kernel/workqueue.c:2858 [inline] flush_work+0x57f/0x8a0 kernel/workqueue.c:2889 __cancel_work_timer+0x30a/0x7e0 kernel/workqueue.c:2961 cancel_work_sync+0x17/0x20 kernel/workqueue.c:2997 xprt_destroy+0xa1/0x130 net/sunrpc/xprt.c:1467 xprt_destroy_kref net/sunrpc/xprt.c:1477 [inline] kref_put include/linux/kref.h:70 [inline] xprt_put+0x38/0x40 net/sunrpc/xprt.c:1501 rpc_task_release_client+0x299/0x430 net/sunrpc/clnt.c:986 rpc_release_resources_task+0x7f/0xa0 net/sunrpc/sched.c:1020 rpc_release_task net/sunrpc/sched.c:1059 [inline] __rpc_execute+0x4d9/0xe70 net/sunrpc/sched.c:824 rpc_async_schedule+0x16/0x20 net/sunrpc/sched.c:848 process_one_work+0xbf0/0x1bc0 kernel/workqueue.c:2119 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x35e/0x430 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex PF_BRIDGE: RTM_SETLINK with unknown ifindex