xs_tcp_setup_socket: connect returned unhandled error -113 xs_tcp_setup_socket: connect returned unhandled error -113 ====================================================== WARNING: possible circular locking dependency detected 4.14.0-rc3+ #31 Not tainted ------------------------------------------------------ kworker/0:4/3496 is trying to acquire lock: ("xprtiod"){+.+.}, at: [] start_flush_work kernel/workqueue.c:2857 [inline] ("xprtiod"){+.+.}, at: [] flush_work+0x5a2/0x8f0 kernel/workqueue.c:2889 but task is already holding lock: ((&task->u.tk_work)){+.+.}, at: [] process_one_work+0xb2c/0x1bd0 kernel/workqueue.c:2094 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 ((&task->u.tk_work)){+.+.}: check_prevs_add kernel/locking/lockdep.c:2020 [inline] validate_chain kernel/locking/lockdep.c:2469 [inline] __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002 process_one_work+0xba2/0x1bd0 kernel/workqueue.c:2095 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x39c/0x470 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 -> #0 ("xprtiod"){+.+.}: check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894 check_prevs_add kernel/locking/lockdep.c:2020 [inline] validate_chain kernel/locking/lockdep.c:2469 [inline] __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002 start_flush_work kernel/workqueue.c:2858 [inline] flush_work+0x5c7/0x8f0 kernel/workqueue.c:2889 __cancel_work_timer+0x30a/0x7e0 kernel/workqueue.c:2961 cancel_work_sync+0x17/0x20 kernel/workqueue.c:2997 xprt_destroy+0xa1/0x130 net/sunrpc/xprt.c:1467 xprt_destroy_kref net/sunrpc/xprt.c:1477 [inline] kref_put include/linux/kref.h:70 [inline] xprt_put+0x38/0x40 net/sunrpc/xprt.c:1501 rpc_task_release_client+0x299/0x430 net/sunrpc/clnt.c:986 rpc_release_resources_task+0x7f/0xa0 net/sunrpc/sched.c:1020 rpc_release_task net/sunrpc/sched.c:1059 [inline] __rpc_execute+0x4d9/0xe70 net/sunrpc/sched.c:824 rpc_async_schedule+0x16/0x20 net/sunrpc/sched.c:848 process_one_work+0xbfa/0x1bd0 kernel/workqueue.c:2119 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x39c/0x470 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock((&task->u.tk_work)); lock("xprtiod"); lock((&task->u.tk_work)); lock("xprtiod"); *** DEADLOCK *** 2 locks held by kworker/0:4/3496: #0: ("rpciod"){+.+.}, at: [] __write_once_size include/linux/compiler.h:305 [inline] #0: ("rpciod"){+.+.}, at: [] atomic64_set arch/x86/include/asm/atomic64_64.h:33 [inline] #0: ("rpciod"){+.+.}, at: [] atomic_long_set include/asm-generic/atomic-long.h:56 [inline] #0: ("rpciod"){+.+.}, at: [] set_work_data kernel/workqueue.c:617 [inline] #0: ("rpciod"){+.+.}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline] #0: ("rpciod"){+.+.}, at: [] process_one_work+0xad4/0x1bd0 kernel/workqueue.c:2090 #1: ((&task->u.tk_work)){+.+.}, at: [] process_one_work+0xb2c/0x1bd0 kernel/workqueue.c:2094 stack backtrace: CPU: 0 PID: 3496 Comm: kworker/0:4 Not tainted 4.14.0-rc3+ #31 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: rpciod rpc_async_schedule Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 print_circular_bug+0x503/0x710 kernel/locking/lockdep.c:1259 check_prev_add+0x865/0x1520 kernel/locking/lockdep.c:1894 check_prevs_add kernel/locking/lockdep.c:2020 [inline] validate_chain kernel/locking/lockdep.c:2469 [inline] __lock_acquire+0x328f/0x4620 kernel/locking/lockdep.c:3498 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4002 start_flush_work kernel/workqueue.c:2858 [inline] flush_work+0x5c7/0x8f0 kernel/workqueue.c:2889 __cancel_work_timer+0x30a/0x7e0 kernel/workqueue.c:2961 cancel_work_sync+0x17/0x20 kernel/workqueue.c:2997 xprt_destroy+0xa1/0x130 net/sunrpc/xprt.c:1467 xprt_destroy_kref net/sunrpc/xprt.c:1477 [inline] kref_put include/linux/kref.h:70 [inline] xprt_put+0x38/0x40 net/sunrpc/xprt.c:1501 rpc_task_release_client+0x299/0x430 net/sunrpc/clnt.c:986 rpc_release_resources_task+0x7f/0xa0 net/sunrpc/sched.c:1020 rpc_release_task net/sunrpc/sched.c:1059 [inline] __rpc_execute+0x4d9/0xe70 net/sunrpc/sched.c:824 rpc_async_schedule+0x16/0x20 net/sunrpc/sched.c:848 process_one_work+0xbfa/0x1bd0 kernel/workqueue.c:2119 worker_thread+0x223/0x1860 kernel/workqueue.c:2253 kthread+0x39c/0x470 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 netlink: 11 bytes leftover after parsing attributes in process `syz-executor2'. netlink: 14 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 11 bytes leftover after parsing attributes in process `syz-executor2'. netlink: 14 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. netlink: 3 bytes leftover after parsing attributes in process `syz-executor5'. kvm_pmu: event creation failed -2 audit: type=1326 audit(1507484750.181:1449): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=12340 comm="syz-executor7" exe="/root/syz-executor7" sig=31 arch=40000003 syscall=240 compat=1 ip=0xf7fa3c79 code=0x0 audit: type=1326 audit(1507484750.203:1450): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=12340 comm="syz-executor7" exe="/root/syz-executor7" sig=31 arch=40000003 syscall=240 compat=1 ip=0xf7fa3c79 code=0x0 audit: type=1326 audit(1507484750.203:1451): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=12340 comm="syz-executor7" exe="/root/syz-executor7" sig=31 arch=40000003 syscall=162 compat=1 ip=0xf7fa3c79 code=0x0 device gre0 entered promiscuous mode FAULT_INJECTION: forcing a failure. name failslab, interval 1, probability 0, space 0, times 1 CPU: 0 PID: 13412 Comm: syz-executor3 Not tainted 4.14.0-rc3+ #31 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 fail_dump lib/fault-inject.c:51 [inline] should_fail+0x8c0/0xa40 lib/fault-inject.c:149 should_failslab+0xec/0x120 mm/failslab.c:31 slab_pre_alloc_hook mm/slab.h:422 [inline] slab_alloc mm/slab.c:3383 [inline] __do_kmalloc mm/slab.c:3723 [inline] __kmalloc_track_caller+0x5f/0x760 mm/slab.c:3740 memdup_user+0x2c/0x90 mm/util.c:164 map_get_next_key kernel/bpf/syscall.c:670 [inline] SYSC_bpf kernel/bpf/syscall.c:1484 [inline] SyS_bpf+0x1baa/0x46a0 kernel/bpf/syscall.c:1453 do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline] do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391 entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124 RIP: 0023:0xf7fa9c79 RSP: 002b:00000000f77a505c EFLAGS: 00000296 ORIG_RAX: 0000000000000165 RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000020fe1ff6 RDX: 0000000000000018 RSI: 0000000000000000 RDI: 0000000000000000 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 FAULT_INJECTION: forcing a failure. name failslab, interval 1, probability 0, space 0, times 0 CPU: 1 PID: 13591 Comm: syz-executor7 Not tainted 4.14.0-rc3+ #31 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 fail_dump lib/fault-inject.c:51 [inline] should_fail+0x8c0/0xa40 lib/fault-inject.c:149 should_failslab+0xec/0x120 mm/failslab.c:31 slab_pre_alloc_hook mm/slab.h:422 [inline] slab_alloc mm/slab.c:3383 [inline] __do_kmalloc mm/slab.c:3723 [inline] __kmalloc_track_caller+0x5f/0x760 mm/slab.c:3740 memdup_user+0x2c/0x90 mm/util.c:164 map_get_next_key kernel/bpf/syscall.c:670 [inline] SYSC_bpf kernel/bpf/syscall.c:1484 [inline] SyS_bpf+0x1baa/0x46a0 kernel/bpf/syscall.c:1453 do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline] do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391 entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124 RIP: 0023:0xf7fa3c79 RSP: 002b:00000000f779f05c EFLAGS: 00000296 ORIG_RAX: 0000000000000165 RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000020fe1ff6 RDX: 0000000000000018 RSI: 0000000000000000 RDI: 0000000000000000 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 FAULT_INJECTION: forcing a failure. name failslab, interval 1, probability 0, space 0, times 0 nla_parse: 6 callbacks suppressed netlink: 5 bytes leftover after parsing attributes in process `syz-executor6'. CPU: 1 PID: 13776 Comm: syz-executor0 Not tainted 4.14.0-rc3+ #31 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 fail_dump lib/fault-inject.c:51 [inline] should_fail+0x8c0/0xa40 lib/fault-inject.c:149 should_failslab+0xec/0x120 mm/failslab.c:31 slab_pre_alloc_hook mm/slab.h:422 [inline] slab_alloc mm/slab.c:3383 [inline] __do_kmalloc mm/slab.c:3723 [inline] __kmalloc_track_caller+0x5f/0x760 mm/slab.c:3740 memdup_user+0x2c/0x90 mm/util.c:164 map_update_elem kernel/bpf/syscall.c:543 [inline] SYSC_bpf kernel/bpf/syscall.c:1478 [inline] SyS_bpf+0x1db6/0x46a0 kernel/bpf/syscall.c:1453 do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline] do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391 entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124 RIP: 0023:0xf7f06c79 RSP: 002b:00000000f770205c EFLAGS: 00000296 ORIG_RAX: 0000000000000165 RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 000000002053dfe0 RDX: 0000000000000020 RSI: 0000000000000000 RDI: 0000000000000000 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 QAT: Invalid ioctl QAT: Invalid ioctl audit: type=1326 audit(1507484755.883:1452): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=14150 comm="syz-executor7" exe="/root/syz-executor7" sig=31 arch=40000003 syscall=240 compat=1 ip=0xf7fa3c79 code=0xffff0000 QAT: Invalid ioctl audit: type=1326 audit(1507484755.962:1453): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=kernel pid=14150 comm="syz-executor7" exe="/root/syz-executor7" sig=31 arch=40000003 syscall=240 compat=1 ip=0xf7fa3c79 code=0xffff0000 QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl FAULT_INJECTION: forcing a failure. name failslab, interval 1, probability 0, space 0, times 0 CPU: 0 PID: 14367 Comm: syz-executor5 Not tainted 4.14.0-rc3+ #31 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x194/0x257 lib/dump_stack.c:52 fail_dump lib/fault-inject.c:51 [inline] should_fail+0x8c0/0xa40 lib/fault-inject.c:149 should_failslab+0xec/0x120 mm/failslab.c:31 slab_pre_alloc_hook mm/slab.h:422 [inline] slab_alloc_node mm/slab.c:3304 [inline] kmem_cache_alloc_node+0x56/0x760 mm/slab.c:3649 __alloc_skb+0xf1/0x740 net/core/skbuff.c:194 alloc_skb include/linux/skbuff.h:976 [inline] alloc_skb_with_frags+0x10d/0x710 net/core/skbuff.c:5137 sock_alloc_send_pskb+0x787/0x9b0 net/core/sock.c:2078 unix_stream_sendmsg+0x80a/0xbe0 net/unix/af_unix.c:1878 sock_sendmsg_nosec net/socket.c:633 [inline] sock_sendmsg+0xca/0x110 net/socket.c:643 sock_write_iter+0x320/0x5e0 net/socket.c:912 call_write_iter include/linux/fs.h:1770 [inline] do_iter_readv_writev+0x531/0x7f0 fs/read_write.c:673 do_iter_write+0x15a/0x540 fs/read_write.c:952 compat_writev+0x225/0x420 fs/read_write.c:1266 do_compat_writev+0x115/0x220 fs/read_write.c:1287 C_SYSC_writev fs/read_write.c:1298 [inline] compat_SyS_writev+0x26/0x30 fs/read_write.c:1294 do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline] do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391 entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124 RIP: 0023:0xf7f62c79 RSP: 002b:00000000f773d05c EFLAGS: 00000296 ORIG_RAX: 0000000000000092 RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000020051000 RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl QAT: Invalid ioctl