: renamed from bcsh0 IPVS: sync thread started: state = BACKUP, mcast_ifn = sit0, syncid = 0, id = 0 IPVS: stopping backup sync thread 18090 ... kworker/dying (21) used greatest stack depth: 12528 bytes left INFO: task kworker/0:1:24 blocked for more than 120 seconds. Not tainted 4.16.0-rc7+ #3 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:1 D20160 24 2 0x80000000 Workqueue: cgroup_destroy css_release_work_fn Call Trace: context_switch kernel/sched/core.c:2862 [inline] __schedule+0x8fb/0x1ec0 kernel/sched/core.c:3440 schedule+0xf5/0x430 kernel/sched/core.c:3499 schedule_preempt_disabled+0x10/0x20 kernel/sched/core.c:3557 __mutex_lock_common kernel/locking/mutex.c:833 [inline] __mutex_lock+0xaad/0x1a80 kernel/locking/mutex.c:893 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908 css_release_work_fn+0xea/0x940 kernel/cgroup/cgroup.c:4592 process_one_work+0xc47/0x1bb0 kernel/workqueue.c:2113 worker_thread+0x223/0x1990 kernel/workqueue.c:2247 kthread+0x33c/0x400 kernel/kthread.c:238 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406 Showing all locks held in the system: 3 locks held by kworker/0:1/24: #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&css->destroy_work)#2){+.+.}, at: [<0000000081bd1087>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (cgroup_mutex){+.+.}, at: [<000000005ab3a50f>] css_release_work_fn+0xea/0x940 kernel/cgroup/cgroup.c:4592 4 locks held by kworker/u4:4/735: #0: ((wq_completion)"%s""netns"){+.+.}, at: [<0000000057d1748c>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"%s""netns"){+.+.}, at: [<0000000057d1748c>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"%s""netns"){+.+.}, at: [<0000000057d1748c>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"%s""netns"){+.+.}, at: [<0000000057d1748c>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: (net_cleanup_work){+.+.}, at: [<0000000081bd1087>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (net_mutex){+.+.}, at: [<00000000a76af218>] cleanup_net+0x242/0xcb0 net/core/net_namespace.c:484 #3: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 2 locks held by khungtaskd/869: #0: (rcu_read_lock){....}, at: [<0000000078977c4f>] check_hung_uninterruptible_tasks kernel/hung_task.c:175 [inline] #0: (rcu_read_lock){....}, at: [<0000000078977c4f>] watchdog+0x1c5/0xd60 kernel/hung_task.c:249 #1: (tasklist_lock){.+.+}, at: [<000000001d7d6758>] debug_show_all_locks+0xd3/0x3d0 kernel/locking/lockdep.c:4470 2 locks held by rs:main Q:Reg/4294: #0: (&f->f_pos_lock){+.+.}, at: [<00000000fecf39c3>] __fdget_pos+0x12b/0x190 fs/file.c:765 #1: (sb_writers#4){.+.+}, at: [<00000000dff815ce>] file_start_write include/linux/fs.h:2709 [inline] #1: (sb_writers#4){.+.+}, at: [<00000000dff815ce>] vfs_write+0x407/0x510 fs/read_write.c:543 2 locks held by getty/4387: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4388: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4389: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4390: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4391: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4392: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by syz-executor4/4464: #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 2 locks held by syz-executor2/4466: #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 4 locks held by syz-executor6/4468: #0: (sb_writers#11){.+.+}, at: [<00000000953621c5>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#11){.+.+}, at: [<00000000953621c5>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<00000000a6ee945e>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<00000000a6ee945e>] filename_create+0x192/0x520 fs/namei.c:3625 #2: (cgroup_mutex){+.+.}, at: [<00000000e7e34115>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 #3: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 3 locks held by kworker/1:4/5773: #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<0000000057d1748c>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&css->destroy_work)#2){+.+.}, at: [<0000000081bd1087>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (cgroup_mutex){+.+.}, at: [<000000005ab3a50f>] css_release_work_fn+0xea/0x940 kernel/cgroup/cgroup.c:4592 4 locks held by syz-executor3/5901: #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000953621c5>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<0000000006bb9bd1>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 #2: (&type->i_mutex_dir_key#3){++++}, at: [<000000002cd092bd>] inode_lock include/linux/fs.h:713 [inline] #2: (&type->i_mutex_dir_key#3){++++}, at: [<000000002cd092bd>] vfs_rmdir+0xd6/0x410 fs/namei.c:3848 #3: (cgroup_mutex){+.+.}, at: [<00000000e7e34115>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 3 locks held by kworker/1:9/9374: #0: ((wq_completion)"%s"("ipv6_addrconf")){+.+.}, at: [<0000000057d1748c>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"%s"("ipv6_addrconf")){+.+.}, at: [<0000000057d1748c>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"%s"("ipv6_addrconf")){+.+.}, at: [<0000000057d1748c>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"%s"("ipv6_addrconf")){+.+.}, at: [<0000000057d1748c>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((addr_chk_work).work){+.+.}, at: [<0000000081bd1087>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 1 lock held by syz-executor5/16180: #0: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 2 locks held by getty/16483: #0: (&tty->ldisc_sem){++++}, at: [<000000006b51f2e8>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<000000002936c29f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 1 lock held by syz-executor5/18088: #0: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 1 lock held by syz-executor0/18089: #0: (ipvs->sync_mutex){+.+.}, at: [<00000000449bfdc5>] do_ip_vs_set_ctl+0x277/0x1cc0 net/netfilter/ipvs/ip_vs_ctl.c:2393 2 locks held by syz-executor0/18091: #0: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 #1: (ipvs->sync_mutex){+.+.}, at: [<0000000090a4f497>] do_ip_vs_set_ctl+0x10f8/0x1cc0 net/netfilter/ipvs/ip_vs_ctl.c:2388 3 locks held by syz-executor1/18087: #0: (sb_writers#10){.+.+}, at: [<00000000dff815ce>] file_start_write include/linux/fs.h:2709 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000dff815ce>] vfs_write+0x407/0x510 fs/read_write.c:543 #1: (&of->mutex){+.+.}, at: [<000000008500d330>] kernfs_fop_write+0x208/0x440 fs/kernfs/file.c:307 #2: (cgroup_mutex){+.+.}, at: [<00000000e7e34115>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 1 lock held by ipvs-b:13:0/18090: #0: (rtnl_mutex){+.+.}, at: [<00000000c911e2f6>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74 ============================================= NMI backtrace for cpu 0 CPU: 0 PID: 869 Comm: khungtaskd Not tainted 4.16.0-rc7+ #3 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:17 [inline] dump_stack+0x194/0x24d lib/dump_stack.c:53 nmi_cpu_backtrace+0x1d2/0x210 lib/nmi_backtrace.c:103 nmi_trigger_cpumask_backtrace+0x123/0x180 lib/nmi_backtrace.c:62 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38 trigger_all_cpu_backtrace include/linux/nmi.h:138 [inline] check_hung_task kernel/hung_task.c:132 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:190 [inline] watchdog+0x90c/0xd60 kernel/hung_task.c:249 kthread+0x33c/0x400 kernel/kthread.c:238 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 skipped: idling at native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:54