INFO: task kworker/0:0:27373 blocked for more than 143 seconds. Not tainted 5.0.0+ #98 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:0 D26408 27373 2 0x80000000 Workqueue: events switchdev_deferred_process_work Call Trace: context_switch kernel/sched/core.c:2877 [inline] __schedule+0x817/0x1cc0 kernel/sched/core.c:3518 schedule+0x92/0x180 kernel/sched/core.c:3562 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3620 __mutex_lock_common kernel/locking/mutex.c:1002 [inline] __mutex_lock+0x726/0x1310 kernel/locking/mutex.c:1072 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087 rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 switchdev_deferred_process_work+0xe/0x20 net/switchdev/switchdev.c:78 process_one_work+0x98e/0x1790 kernel/workqueue.c:2269 worker_thread+0x98/0xe40 kernel/workqueue.c:2415 kthread+0x357/0x430 kernel/kthread.c:253 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352 Showing all locks held in the system: 4 locks held by kworker/u4:0/7: #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 00000000edbc0dbe ((wq_completion)netns){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240 #1: 00000000502af2ac (net_cleanup_work){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244 #2: 0000000081db7829 (pernet_ops_rwsem){++++}, at: cleanup_net+0xae/0x960 net/core/net_namespace.c:518 #3: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 4 locks held by ksoftirqd/0/9: 1 lock held by khungtaskd/1040: #0: 000000007c41b7e1 (rcu_read_lock){....}, at: debug_show_all_locks+0x5f/0x27e kernel/locking/lockdep.c:5061 1 lock held by rsyslogd/7597: #0: 00000000454ee2f6 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0xee/0x110 fs/file.c:801 2 locks held by getty/7687: #0: 00000000fb9b4bd6 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 00000000f39badfa (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7688: #0: 000000004e37da09 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 0000000016ec9418 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7689: #0: 00000000f67097db (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 00000000a7061ebe (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7690: #0: 00000000b2e86f9f (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 0000000046c5df1b (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7691: #0: 0000000090f66454 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 0000000019df8a7e (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7692: #0: 00000000ff4e1dfe (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 00000000af475c90 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 2 locks held by getty/7693: #0: 00000000cc98f7f9 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341 #1: 0000000071942cd1 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156 3 locks held by kworker/1:3/7749: #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 00000000794498ec ((wq_completion)ipv6_addrconf){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240 #1: 00000000922c618e ((addr_chk_work).work){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244 #2: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 3 locks held by kworker/0:0/27373: #0: 000000005bae497e ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240 #1: 00000000ae370ffe (deferred_process_work){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244 #2: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 1 lock held by syz-executor.0/26207: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock net/core/rtnetlink.c:76 [inline] #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x40a/0xb00 net/core/rtnetlink.c:5189 1 lock held by syz-executor.0/26225: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock net/core/rtnetlink.c:76 [inline] #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x40a/0xb00 net/core/rtnetlink.c:5189 1 lock held by syz-executor.1/26208: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 1 lock held by syz-executor.1/26227: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 1 lock held by syz-executor.1/26260: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 1 lock held by syz-executor.4/26257: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 1 lock held by syz-executor.4/26293: #0: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 4 locks held by syz-executor.5/26325: #0: 00000000ec51a179 (cb_lock){++++}, at: genl_rcv+0x1a/0x40 net/netlink/genetlink.c:636 #1: 0000000058308107 (nlk_cb_mutex-GENERIC){+.+.}, at: netlink_dump+0xe7/0xfb0 net/netlink/af_netlink.c:2207 #2: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_lock net/netlink/genetlink.c:33 [inline] #2: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_lock_dumpit+0x43/0xa0 net/netlink/genetlink.c:480 #3: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 2 locks held by syz-executor.5/26330: #0: 00000000ec51a179 (cb_lock){++++}, at: genl_rcv+0x1a/0x40 net/netlink/genetlink.c:636 #1: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_lock net/netlink/genetlink.c:33 [inline] #1: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_rcv_msg+0x13e/0x16c net/netlink/genetlink.c:624 2 locks held by syz-executor.3/26668: #0: 00000000ec51a179 (cb_lock){++++}, at: genl_rcv+0x1a/0x40 net/netlink/genetlink.c:636 #1: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_lock net/netlink/genetlink.c:33 [inline] #1: 000000003ecf9b32 (genl_mutex){+.+.}, at: genl_rcv_msg+0x13e/0x16c net/netlink/genetlink.c:624 3 locks held by kworker/0:5/28527: #0: 000000005bae497e ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000005bae497e ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240 #1: 00000000f334bcd2 ((linkwatch_work).work){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244 #2: 0000000038d2a234 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76 2 locks held by net.agent/10532: ============================================= NMI backtrace for cpu 0 CPU: 0 PID: 1040 Comm: khungtaskd Not tainted 5.0.0+ #98 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x172/0x1f0 lib/dump_stack.c:113 nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101 nmi_trigger_cpumask_backtrace+0x1be/0x236 lib/nmi_backtrace.c:62 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline] watchdog+0x9b7/0xec0 kernel/hung_task.c:288 kthread+0x357/0x430 kernel/kthread.c:253 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 CPU: 1 PID: 16 Comm: ksoftirqd/1 Not tainted 5.0.0+ #98 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:rol32 include/linux/bitops.h:83 [inline] RIP: 0010:jhash2 include/linux/jhash.h:128 [inline] RIP: 0010:hash_stack lib/stackdepot.c:161 [inline] RIP: 0010:depot_save_stack+0xa1/0x460 lib/stackdepot.c:230 Code: c1 c1 04 01 c3 45 31 d9 44 29 c8 41 89 c2 44 89 c8 41 01 d9 c1 c0 06 44 31 d0 29 c3 41 89 da 89 c3 44 01 c8 c1 c3 08 44 31 d3 <41> 89 da 41 29 d9 01 c3 41 c1 c2 10 45 31 d1 45 89 ca 44 29 c8 41 RSP: 0018:ffff8880aa2c6fb8 EFLAGS: 00000282 RAX: 0000000046b86701 RBX: 00000000bb93d192 RCX: ffff8880aa2c7018 RDX: ffff8880aa2c706c RSI: 0000000000000a20 RDI: 0000000000000014 RBP: ffff8880aa2c6ff0 R08: 0000000000000013 R09: 0000000000f549f1 R10: 00000000788ec1d7 R11: 00000000abea2ed2 R12: 0000000000000a20 R13: ffff8880aa2c7000 R14: 0000000000000000 R15: ffff8880aa3adb00 FS: 0000000000000000(0000) GS:ffff8880ae900000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f2f8f961e30 CR3: 00000000948d9000 CR4: 00000000001406e0 Call Trace: save_stack+0xa9/0xd0 mm/kasan/common.c:81 set_track mm/kasan/common.c:87 [inline] __kasan_kmalloc mm/kasan/common.c:497 [inline] __kasan_kmalloc.constprop.0+0xcf/0xe0 mm/kasan/common.c:470 kasan_slab_alloc+0xf/0x20 mm/kasan/common.c:505 slab_post_alloc_hook mm/slab.h:436 [inline] slab_alloc mm/slab.c:3392 [inline] kmem_cache_alloc+0x11a/0x6f0 mm/slab.c:3554 skb_clone+0x150/0x3b0 net/core/skbuff.c:1289 deliver_clone+0x46/0xc0 net/bridge/br_forward.c:123 maybe_deliver net/bridge/br_forward.c:184 [inline] maybe_deliver net/bridge/br_forward.c:172 [inline] br_flood+0x4da/0x710 net/bridge/br_forward.c:226 br_handle_frame_finish+0xf4e/0x14c0 net/bridge/br_input.c:169 br_nf_hook_thresh+0x2ec/0x380 net/bridge/br_netfilter_hooks.c:1005 br_nf_pre_routing_finish_ipv6+0x700/0xdc0 net/bridge/br_netfilter_ipv6.c:210 NF_HOOK include/linux/netfilter.h:289 [inline] br_nf_pre_routing_ipv6+0x3c4/0x740 net/bridge/br_netfilter_ipv6.c:238 br_nf_pre_routing+0xe80/0x13a0 net/bridge/br_netfilter_hooks.c:482 nf_hook_entry_hookfn include/linux/netfilter.h:119 [inline] nf_hook_slow+0xbf/0x1f0 net/netfilter/core.c:511 nf_hook include/linux/netfilter.h:244 [inline] NF_HOOK include/linux/netfilter.h:287 [inline] br_handle_frame+0x95b/0x1450 net/bridge/br_input.c:305 __netif_receive_skb_core+0xa96/0x3040 net/core/dev.c:4902 __netif_receive_skb_one_core+0xa8/0x1a0 net/core/dev.c:4971 __netif_receive_skb+0x2c/0x1c0 net/core/dev.c:5083 process_backlog+0x206/0x750 net/core/dev.c:5923 napi_poll net/core/dev.c:6346 [inline] net_rx_action+0x4fa/0x1070 net/core/dev.c:6412 __do_softirq+0x266/0x95a kernel/softirq.c:293 run_ksoftirqd kernel/softirq.c:655 [inline] run_ksoftirqd+0x8e/0x110 kernel/softirq.c:647 smpboot_thread_fn+0x6ab/0xa10 kernel/smpboot.c:164 kthread+0x357/0x430 kernel/kthread.c:253 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352