INFO: task kworker/0:4:6722 blocked for more than 120 seconds. Not tainted 4.16.0-rc7+ #368 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:4 D23040 6722 2 0x80000000 Workqueue: events cgwb_release_workfn Call Trace: context_switch kernel/sched/core.c:2862 [inline] __schedule+0x8fb/0x1ec0 kernel/sched/core.c:3440 schedule+0xf5/0x430 kernel/sched/core.c:3499 bit_wait+0x18/0x90 kernel/sched/wait_bit.c:250 __wait_on_bit+0x88/0x130 kernel/sched/wait_bit.c:51 out_of_line_wait_on_bit+0x204/0x3a0 kernel/sched/wait_bit.c:64 wait_on_bit include/linux/wait_bit.h:84 [inline] wb_shutdown+0x335/0x430 mm/backing-dev.c:377 cgwb_release_workfn+0x8b/0x61d mm/backing-dev.c:520 process_one_work+0xc47/0x1bb0 kernel/workqueue.c:2113 worker_thread+0x223/0x1990 kernel/workqueue.c:2247 kthread+0x33c/0x400 kernel/kthread.c:238 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406 Showing all locks held in the system: 5 locks held by kworker/1:0/17: #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000b5914474>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000b5914474>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000b5914474>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000b5914474>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&css->destroy_work)){+.+.}, at: [<0000000083fa8109>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (cgroup_mutex){+.+.}, at: [<00000000b04615d8>] css_killed_work_fn+0x93/0x5c0 kernel/cgroup/cgroup.c:4967 #3: (cpu_hotplug_lock.rw_sem){++++}, at: [<0000000048d2c0f1>] get_online_cpus include/linux/cpu.h:124 [inline] #3: (cpu_hotplug_lock.rw_sem){++++}, at: [<0000000048d2c0f1>] memcg_deactivate_kmem_caches+0x21/0xf0 mm/slab_common.c:747 #4: (slab_mutex){+.+.}, at: [<00000000b79fc52d>] memcg_deactivate_kmem_caches+0x2f/0xf0 mm/slab_common.c:750 2 locks held by khungtaskd/869: #0: (rcu_read_lock){....}, at: [<00000000f11fc31a>] check_hung_uninterruptible_tasks kernel/hung_task.c:175 [inline] #0: (rcu_read_lock){....}, at: [<00000000f11fc31a>] watchdog+0x1c5/0xd60 kernel/hung_task.c:249 #1: (tasklist_lock){.+.+}, at: [<00000000cd9649f6>] debug_show_all_locks+0xd3/0x3d0 kernel/locking/lockdep.c:4470 4 locks held by kworker/0:3/2038: #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&cw->work)){+.+.}, at: [<0000000083fa8109>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000e41f9452>] get_online_cpus include/linux/cpu.h:124 [inline] #2: (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000e41f9452>] memcg_create_kmem_cache+0x16/0x170 mm/slab_common.c:619 #3: (slab_mutex){+.+.}, at: [<00000000714ddc37>] memcg_create_kmem_cache+0x24/0x170 mm/slab_common.c:622 2 locks held by getty/4403: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4404: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4405: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4406: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4407: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4408: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 2 locks held by getty/4409: #0: (&tty->ldisc_sem){++++}, at: [<0000000004053521>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365 #1: (&ldata->atomic_read_lock){+.+.}, at: [<0000000019a96d3a>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131 4 locks held by syz-executor7/4474: #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 #2: (&type->i_mutex_dir_key#3){++++}, at: [<000000007978767d>] inode_lock include/linux/fs.h:713 [inline] #2: (&type->i_mutex_dir_key#3){++++}, at: [<000000007978767d>] vfs_rmdir+0xd6/0x410 fs/namei.c:3848 #3: (cgroup_mutex){+.+.}, at: [<000000006221af8c>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 3 locks held by syz-executor0/4477: #0: (sb_writers#11){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#11){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<000000004294f962>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<000000004294f962>] filename_create+0x192/0x520 fs/namei.c:3625 #2: (cgroup_mutex){+.+.}, at: [<000000006221af8c>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 2 locks held by syz-executor3/4480: #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 2 locks held by syz-executor1/4481: #0: (sb_writers#11){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#11){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<000000004294f962>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#4/1){+.+.}, at: [<000000004294f962>] filename_create+0x192/0x520 fs/namei.c:3625 2 locks held by syz-executor4/4482: #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 2 locks held by syz-executor5/4483: #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] sb_start_write include/linux/fs.h:1548 [inline] #0: (sb_writers#10){.+.+}, at: [<00000000f62ed48f>] mnt_want_write+0x3f/0xb0 fs/namespace.c:386 #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] inode_lock_nested include/linux/fs.h:748 [inline] #1: (&type->i_mutex_dir_key#3/1){+.+.}, at: [<00000000ef3526ed>] do_rmdir+0x380/0x5f0 fs/namei.c:3907 4 locks held by kworker/1:3/5108: #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000b5914474>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&cw->work)){+.+.}, at: [<0000000083fa8109>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 #2: (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000e41f9452>] get_online_cpus include/linux/cpu.h:124 [inline] #2: (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000e41f9452>] memcg_create_kmem_cache+0x16/0x170 mm/slab_common.c:619 #3: (slab_mutex){+.+.}, at: [<00000000714ddc37>] memcg_create_kmem_cache+0x24/0x170 mm/slab_common.c:622 2 locks held by kworker/0:4/6722: #0: ((wq_completion)"events"){+.+.}, at: [<00000000b5914474>] work_static include/linux/workqueue.h:198 [inline] #0: ((wq_completion)"events"){+.+.}, at: [<00000000b5914474>] set_work_data kernel/workqueue.c:619 [inline] #0: ((wq_completion)"events"){+.+.}, at: [<00000000b5914474>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: ((wq_completion)"events"){+.+.}, at: [<00000000b5914474>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084 #1: ((work_completion)(&wb->release_work)){+.+.}, at: [<0000000083fa8109>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088 3 locks held by syz-executor2/31722: #0: (sb_writers#11){.+.+}, at: [<0000000050ec4ea5>] file_start_write include/linux/fs.h:2709 [inline] #0: (sb_writers#11){.+.+}, at: [<0000000050ec4ea5>] vfs_write+0x407/0x510 fs/read_write.c:543 #1: (&of->mutex){+.+.}, at: [<000000009856e8e8>] kernfs_fop_write+0x208/0x440 fs/kernfs/file.c:307 #2: (cgroup_mutex){+.+.}, at: [<000000006221af8c>] cgroup_kn_lock_live+0x29e/0x580 kernel/cgroup/cgroup.c:1535 1 lock held by syz-executor6/31723: #0: (&type->i_mutex_dir_key#4){++++}, at: [<0000000069ddf2f5>] inode_lock_shared include/linux/fs.h:723 [inline] #0: (&type->i_mutex_dir_key#4){++++}, at: [<0000000069ddf2f5>] lookup_slow+0x18e/0x4d0 fs/namei.c:1612 ============================================= NMI backtrace for cpu 0 CPU: 0 PID: 869 Comm: khungtaskd Not tainted 4.16.0-rc7+ #368 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:17 [inline] dump_stack+0x194/0x24d lib/dump_stack.c:53 nmi_cpu_backtrace+0x1d2/0x210 lib/nmi_backtrace.c:103 nmi_trigger_cpumask_backtrace+0x123/0x180 lib/nmi_backtrace.c:62 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38 trigger_all_cpu_backtrace include/linux/nmi.h:138 [inline] check_hung_task kernel/hung_task.c:132 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:190 [inline] watchdog+0x90c/0xd60 kernel/hung_task.c:249 kthread+0x33c/0x400 kernel/kthread.c:238 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 23 Comm: kworker/1:1 Not tainted 4.16.0-rc7+ #368 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: events perf_sched_delayed RIP: 0010:rep_nop arch/x86/include/asm/processor.h:657 [inline] RIP: 0010:cpu_relax arch/x86/include/asm/processor.h:662 [inline] RIP: 0010:csd_lock_wait kernel/smp.c:108 [inline] RIP: 0010:smp_call_function_single+0x364/0x560 kernel/smp.c:302 RSP: 0018:ffff8801d945eee0 EFLAGS: 00000293 RAX: ffff8801d9456600 RBX: 1ffff1003b28bde4 RCX: ffffffff81669da2 RDX: 0000000000000000 RSI: 00000000000000fb RDI: ffff8801d945efd8 RBP: ffff8801d945f030 R08: 1ffff1003b28bdf9 R09: 0000000000000000 R10: ffff8801d945f048 R11: 0000000000000000 R12: ffff8801d945efd8 R13: dffffc0000000000 R14: 0000000000000000 R15: ffffed003b28bdf0 FS: 0000000000000000(0000) GS:ffff8801db300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000001f51978 CR3: 0000000007a22002 CR4: 00000000001606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: smp_call_function_many+0x773/0x930 kernel/smp.c:434 smp_call_function kernel/smp.c:492 [inline] on_each_cpu+0x3d/0x1b0 kernel/smp.c:602 text_poke_bp+0x107/0x170 arch/x86/kernel/alternative.c:814 __jump_label_transform.isra.0+0x6a5/0x8a0 arch/x86/kernel/jump_label.c:102 arch_jump_label_transform+0x2f/0x40 arch/x86/kernel/jump_label.c:110 __jump_label_update+0x20b/0x320 kernel/jump_label.c:375 jump_label_update+0x22c/0x2b0 kernel/jump_label.c:760 static_key_disable_cpuslocked+0x227/0x2a0 kernel/jump_label.c:171 static_key_disable+0x1a/0x30 kernel/jump_label.c:179 perf_sched_delayed+0x31/0x50 kernel/events/core.c:3998 process_one_work+0xc47/0x1bb0 kernel/workqueue.c:2113 worker_thread+0x223/0x1990 kernel/workqueue.c:2247 kthread+0x33c/0x400 kernel/kthread.c:238 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406 Code: 00 00 00 fc ff df 44 89 bd 50 ff ff ff 48 c1 e8 03 4c 01 e8 41 83 e7 01 c6 00 f8 74 4e 49 89 c7 49 83 c4 18 e8 7e 2b 0b 00 f3 90 <4c> 89 e2 41 c6 07 04 48 c1 ea 03 42 0f b6 14 2a 84 d2 74 09 80