syzbot


INFO: task hung in memcg_destroy_kmem_caches

Status: closed as invalid on 2018/03/27 11:14
Subsystems: cgroups mm
[Documentation on labels]
First crash: 2500d, last: 2500d

Sample crash report:
Free swap  = 0kB
Total swap = 0kB
1965969 pages RAM
0 pages HighMem/MovableOnly
329730 pages reserved
INFO: task kworker/0:1:24 blocked for more than 120 seconds.
      Not tainted 4.16.0-rc6+ #282
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/0:1     D19192    24      2 0x80000000
Workqueue: cgroup_destroy css_free_work_fn
Call Trace:
 context_switch kernel/sched/core.c:2862 [inline]
 __schedule+0x8fb/0x1ec0 kernel/sched/core.c:3440
 schedule+0xf5/0x430 kernel/sched/core.c:3499
 schedule_preempt_disabled+0x10/0x20 kernel/sched/core.c:3557
 __mutex_lock_common kernel/locking/mutex.c:833 [inline]
 __mutex_lock+0xaad/0x1a80 kernel/locking/mutex.c:893
 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
 memcg_destroy_kmem_caches+0x24/0x80 mm/slab_common.c:774
 memcg_free_kmem mm/memcontrol.c:2856 [inline]
 mem_cgroup_css_free+0x2a4/0x3f0 mm/memcontrol.c:4298
 css_free_work_fn+0x1c8/0x1420 kernel/cgroup/cgroup.c:4541
 process_one_work+0xc47/0x1bb0 kernel/workqueue.c:2113
 worker_thread+0x223/0x1990 kernel/workqueue.c:2247
 kthread+0x33c/0x400 kernel/kthread.c:238
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406

Showing all locks held in the system:
4 locks held by kworker/0:1/24:
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] work_static include/linux/workqueue.h:198 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] set_work_data kernel/workqueue.c:619 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084
 #1:  ((work_completion)(&css->destroy_work)#3){+.+.}, at: [<00000000a348ebd0>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088
 #2:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000215aab5b>] get_online_cpus include/linux/cpu.h:124 [inline]
 #2:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000215aab5b>] memcg_destroy_kmem_caches+0xf/0x80 mm/slab_common.c:771
 #3:  (slab_mutex){+.+.}, at: [<00000000718ca68d>] memcg_destroy_kmem_caches+0x24/0x80 mm/slab_common.c:774
2 locks held by khungtaskd/767:
 #0:  (rcu_read_lock){....}, at: [<00000000b941e999>] check_hung_uninterruptible_tasks kernel/hung_task.c:175 [inline]
 #0:  (rcu_read_lock){....}, at: [<00000000b941e999>] watchdog+0x1c5/0xd60 kernel/hung_task.c:249
 #1:  (tasklist_lock){.+.+}, at: [<000000002f5af2cb>] debug_show_all_locks+0xd3/0x3d0 kernel/locking/lockdep.c:4470
2 locks held by getty/4148:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4149:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4150:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4151:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4152:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4153:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
2 locks held by getty/4154:
 #0:  (&tty->ldisc_sem){++++}, at: [<0000000093388b9d>] ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:365
 #1:  (&ldata->atomic_read_lock){+.+.}, at: [<000000005aa3151f>] n_tty_read+0x2ef/0x1a40 drivers/tty/n_tty.c:2131
4 locks held by kworker/0:4/6516:
 #0:  ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000f8a2b060>] work_static include/linux/workqueue.h:198 [inline]
 #0:  ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000f8a2b060>] set_work_data kernel/workqueue.c:619 [inline]
 #0:  ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000f8a2b060>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline]
 #0:  ((wq_completion)"memcg_kmem_cache"){+.+.}, at: [<00000000f8a2b060>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084
 #1:  ((work_completion)(&cw->work)){+.+.}, at: [<00000000a348ebd0>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088
 #2:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000400c3c30>] get_online_cpus include/linux/cpu.h:124 [inline]
 #2:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000400c3c30>] memcg_create_kmem_cache+0x16/0x170 mm/slab_common.c:619
 #3:  (slab_mutex){+.+.}, at: [<00000000cea2d92f>] memcg_create_kmem_cache+0x24/0x170 mm/slab_common.c:622
3 locks held by kworker/0:5/6860:
 #0:  ((wq_completion)"events"){+.+.}, at: [<00000000f8a2b060>] work_static include/linux/workqueue.h:198 [inline]
 #0:  ((wq_completion)"events"){+.+.}, at: [<00000000f8a2b060>] set_work_data kernel/workqueue.c:619 [inline]
 #0:  ((wq_completion)"events"){+.+.}, at: [<00000000f8a2b060>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline]
 #0:  ((wq_completion)"events"){+.+.}, at: [<00000000f8a2b060>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084
 #1:  (slab_caches_to_rcu_destroy_work){+.+.}, at: [<00000000a348ebd0>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088
 #2:  (slab_mutex){+.+.}, at: [<0000000062b557b9>] slab_caches_to_rcu_destroy_workfn+0x25/0xc0 mm/slab_common.c:556
5 locks held by kworker/1:7/7292:
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] work_static include/linux/workqueue.h:198 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] set_work_data kernel/workqueue.c:619 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline]
 #0:  ((wq_completion)"cgroup_destroy"){+.+.}, at: [<00000000f8a2b060>] process_one_work+0xb12/0x1bb0 kernel/workqueue.c:2084
 #1:  ((work_completion)(&css->destroy_work)){+.+.}, at: [<00000000a348ebd0>] process_one_work+0xb89/0x1bb0 kernel/workqueue.c:2088
 #2:  (cgroup_mutex){+.+.}, at: [<00000000b4af997b>] css_killed_work_fn+0x93/0x5c0 kernel/cgroup/cgroup.c:4967
 #3:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000458d4b3a>] get_online_cpus include/linux/cpu.h:124 [inline]
 #3:  (cpu_hotplug_lock.rw_sem){++++}, at: [<00000000458d4b3a>] memcg_deactivate_kmem_caches+0x21/0xf0 mm/slab_common.c:747
 #4:  (slab_mutex){+.+.}, at: [<0000000052d19bb1>] memcg_deactivate_kmem_caches+0x2f/0xf0 mm/slab_common.c:750
1 lock held by syz-executor5/20840:
 #0:  (pcpu_drain_mutex){+.+.}, at: [<00000000c6c24bdf>] drain_all_pages+0x94/0x620 mm/page_alloc.c:2528

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 767 Comm: khungtaskd Not tainted 4.16.0-rc6+ #282
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x194/0x24d lib/dump_stack.c:53
 nmi_cpu_backtrace+0x1d2/0x210 lib/nmi_backtrace.c:103
 nmi_trigger_cpumask_backtrace+0x123/0x180 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_all_cpu_backtrace include/linux/nmi.h:138 [inline]
 check_hung_task kernel/hung_task.c:132 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:190 [inline]
 watchdog+0x90c/0xd60 kernel/hung_task.c:249
 kthread+0x33c/0x400 kernel/kthread.c:238
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 20855 Comm: syz-executor5 Not tainted 4.16.0-rc6+ #282
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__debug_check_no_obj_freed lib/debugobjects.c:737 [inline]
RIP: 0010:debug_check_no_obj_freed+0x24e/0xf1f lib/debugobjects.c:774
RSP: 0018:ffff880136257758 EFLAGS: 00000002
RAX: 0000000000000013 RBX: ffff880167bd74b8 RCX: 0000000000000282
RDX: ffff880167950f18 RSI: 1ffff1002ee94da0 RDI: ffff880167950f68
RBP: ffff880136257928 R08: 1ffff10026c4ae29 R09: ffffc90108320000
R10: ffffed0026c4af10 R11: ffffc90208320000 R12: ffffed0026c4af00
R13: ffff880136257900 R14: ffff880167950f50 R15: dffffc0000000000
FS:  00007f271510c700(0000) GS:ffff8801db200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000041a690 CR3: 0000000198a07006 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 __vunmap+0x112/0x380 mm/vmalloc.c:1530
 vfree+0x50/0xe0 mm/vmalloc.c:1606
 __vmalloc_area_node mm/vmalloc.c:1721 [inline]
 __vmalloc_node_range+0x4ad/0x650 mm/vmalloc.c:1759
 __vmalloc_node mm/vmalloc.c:1804 [inline]
 __vmalloc_node_flags_caller+0x50/0x60 mm/vmalloc.c:1826
 kvmalloc_node+0x82/0xd0 mm/util.c:428
 kvmalloc include/linux/mm.h:541 [inline]
 xt_alloc_table_info+0x25/0x80 net/netfilter/x_tables.c:1040
 do_replace net/ipv4/netfilter/arp_tables.c:974 [inline]
 do_arpt_set_ctl+0x29d/0x8d0 net/ipv4/netfilter/arp_tables.c:1462
 nf_sockopt net/netfilter/nf_sockopt.c:106 [inline]
 nf_setsockopt+0x67/0xc0 net/netfilter/nf_sockopt.c:115
 ip_setsockopt+0x97/0xa0 net/ipv4/ip_sockglue.c:1253
 tcp_setsockopt+0x82/0xd0 net/ipv4/tcp.c:2880
 sock_common_setsockopt+0x95/0xd0 net/core/sock.c:3039
 SYSC_setsockopt net/socket.c:1850 [inline]
 SyS_setsockopt+0x189/0x360 net/socket.c:1829
 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x454879
RSP: 002b:00007f271510bc68 EFLAGS: 00000246 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 00007f271510c6d4 RCX: 0000000000454879
RDX: 0000000000000060 RSI: 0000000000000000 RDI: 0000000000000014
RBP: 000000000072c010 R08: 00000000000009cc R09: 0000000000000000
R10: 0000000020000ec0 R11: 0000000000000246 R12: 00000000ffffffff
R13: 000000000000050c R14: 00000000006f99c0 R15: 0000000000000002
Code: 8b 8d 88 fe ff ff 31 c0 49 c1 ea 03 4d 01 fa 4c 89 f2 48 c1 ea 03 42 80 3c 3a 00 0f 85 06 05 00 00 49 8d 7e 18 83 c0 01 49 8b 16 <48> 89 fe 48 c1 ee 03 42 80 3c 3e 00 0f 85 3b 05 00 00 49 8b 5e 

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2018/03/26 17:05 net-next-old 996bfed11874 0ca7878b .config console log report ci-upstream-net-kasan-gce
* Struck through repros no longer work on HEAD.