BUG: workqueue lockup - pool cpus=0 node=0 flags=0x2 nice=0 stuck for 44s! Showing busy workqueues and worker pools: workqueue events: flags=0x0 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=17 refcnt=18 in-flight: 9:rfkill_op_handler ,10:rfkill_global_led_trigger_worker rfkill_global_led_trigger_worker pending: 5*ovs_dp_masks_rebalance, 2*psi_avgs_work, 2*nsim_dev_hwstats_traffic_work, drm_fb_helper_damage_work, vmstat_shepherd, psi_avgs_work, free_obj_work, psi_avgs_work pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=4 refcnt=5 in-flight: 2843:request_firmware_work_func pending: 3*nsim_dev_hwstats_traffic_work workqueue events_long: flags=0x0 pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=6 refcnt=7 pending: 6*defense_work_handler workqueue events_unbound: flags=0x2 pwq 8: cpus=0-1 flags=0x6 nice=0 active=2 refcnt=3 in-flight: 41:linkwatch_event linkwatch_event pwq 8: cpus=0-1 flags=0x6 nice=0 active=10 refcnt=11 in-flight: 2158:toggle_allocation_gate ,8229:nsim_dev_trap_report_work ,1908:cfg80211_wiphy_work cfg80211_wiphy_work ,14:nsim_dev_trap_report_work ,42:cfg80211_wiphy_work cfg80211_wiphy_work ,2082:nsim_dev_trap_report_work pending: 2*nsim_dev_trap_report_work workqueue events_power_efficient: flags=0x82 pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=2 in-flight: 8227:reg_check_chans_work pwq 8: cpus=0-1 flags=0x6 nice=0 active=3 refcnt=4 in-flight: 2211:neigh_periodic_work ,12:gc_worker pending: fb_flashcursor workqueue mm_percpu_wq: flags=0x8 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: vmstat_update pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: vmstat_update workqueue writeback: flags=0x4a pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=2 in-flight: 209:wb_workfn workqueue cfg80211: flags=0x2000a pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=4 in-flight: 2136:cfg80211_dfs_channels_update_work workqueue ipv6_addrconf: flags=0x6000a pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=10 in-flight: 4201(RESCUER):addrconf_verify_work inactive: 5*addrconf_verify_work workqueue bat_events: flags=0x6000a pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=53 in-flight: 4276(RESCUER):batadv_tt_purge inactive: batadv_tt_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, 2*batadv_bla_periodic_work, 2*batadv_dat_purge, 5*batadv_nc_worker, 5*batadv_mcast_mla_update, 4*batadv_iv_send_outstanding_bat_ogm_packet, 5*batadv_purge_orig, 15*batadv_iv_send_outstanding_bat_ogm_packet, 3*batadv_tt_purge workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-crypt-wg2: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=1 refcnt=2 in-flight: 6540:wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=2 refcnt=4 MAYDAY in-flight: 24:wg_packet_encrypt_worker pending: wg_packet_encrypt_worker workqueue wg-kex-wg1: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_decrypt_worker, wg_packet_tx_worker, wg_packet_encrypt_worker workqueue wg-crypt-wg2: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=1 refcnt=2 in-flight: 26:wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_encrypt_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=2 refcnt=3 in-flight: 6577:wg_packet_handshake_receive_worker wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker workqueue wg-kex-wg1: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_tx_worker, wg_packet_decrypt_worker, wg_packet_encrypt_worker workqueue wg-kex-wg1: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-kex-wg1: flags=0x6 pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=2 in-flight: 8233:wg_packet_handshake_send_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg2: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 in-flight: 2323:wg_packet_handshake_receive_worker pending: wg_packet_handshake_receive_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=2 refcnt=3 in-flight: 6640:wg_packet_handshake_receive_worker wg_packet_handshake_receive_worker workqueue wg-crypt-wg2: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_tx_worker, wg_packet_encrypt_worker workqueue wg-kex-wg2: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg2: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 in-flight: 6595:wg_packet_handshake_receive_worker pending: wg_packet_handshake_receive_worker pwq 6: cpus=1 node=0 flags=0x2 nice=0 active=2 refcnt=3 in-flight: 6590:wg_packet_handshake_receive_worker wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_tx_worker, wg_packet_encrypt_worker workqueue wg-crypt-wg1: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=3 refcnt=4 pending: wg_packet_decrypt_worker, wg_packet_tx_worker, wg_packet_encrypt_worker workqueue wg-kex-wg2: flags=0x24 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-kex-wg2: flags=0x6 pwq 8: cpus=0-1 flags=0x6 nice=0 active=1 refcnt=2 in-flight: 8231:wg_packet_handshake_send_worker workqueue wg-crypt-wg2: flags=0x28 pwq 2: cpus=0 node=0 flags=0x2 nice=0 active=2 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker pool 2: cpus=0 node=0 flags=0x2 nice=0 hung=44s workers=8 manager: 6637 idle: 8230 8228 8224 pool 6: cpus=1 node=0 flags=0x2 nice=0 hung=0s workers=9 manager: 6576 idle: 6598 pool 8: cpus=0-1 flags=0x6 nice=0 hung=0s workers=15 manager: 8235 Showing backtraces of running workers in stalled CPU-bound worker pools: pool 2: task:kworker/0:0 state:R running task stack:0 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events rfkill_op_handler Call trace: __switch_to+0x418/0x87c arch/arm64/kernel/process.c:741 (T) context_switch kernel/sched/core.c:5357 [inline] __schedule+0x13b0/0x2864 kernel/sched/core.c:6961 preempt_schedule_common+0xe0/0x194 kernel/sched/core.c:7145 preempt_schedule+0x60/0x80 kernel/sched/core.c:7169 __local_bh_enable_ip+0x20c/0x35c kernel/softirq.c:414 __raw_spin_unlock_bh include/linux/spinlock_api_smp.h:167 [inline] _raw_spin_unlock_bh+0x3c/0x4c kernel/locking/spinlock.c:210 spin_unlock_bh include/linux/spinlock.h:396 [inline] addrconf_ifdown+0x5a0/0x148c net/ipv6/addrconf.c:3907 addrconf_notify+0x2f4/0xcdc net/ipv6/addrconf.c:-1 notifier_call_chain+0x1b8/0x4e4 kernel/notifier.c:85 raw_notifier_call_chain+0x3c/0x50 kernel/notifier.c:453 call_netdevice_notifiers_info net/core/dev.c:2229 [inline] call_netdevice_notifiers_extack net/core/dev.c:2267 [inline] call_netdevice_notifiers net/core/dev.c:2281 [inline] netif_close_many+0x2d4/0x448 net/core/dev.c:1784 netif_close+0x148/0x1f8 net/core/dev.c:1797 dev_close+0xf8/0x1e4 net/core/dev_api.c:220 cfg80211_shutdown_all_interfaces+0xd4/0x210 net/wireless/core.c:277 cfg80211_rfkill_set_block+0x30/0x50 net/wireless/core.c:307 rfkill_set_block+0x18c/0x374 net/rfkill/core.c:346 rfkill_epo+0x8c/0x158 net/rfkill/core.c:466 __rfkill_handle_global_op net/rfkill/input.c:60 [inline] rfkill_op_handler+0x164/0x25c net/rfkill/input.c:108 process_one_work+0x7e8/0x155c kernel/workqueue.c:3236 process_scheduled_works kernel/workqueue.c:3319 [inline] worker_thread+0x958/0xed8 kernel/workqueue.c:3400 kthread+0x5fc/0x75c kernel/kthread.c:463 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:844 pool 2: task:kworker/0:3 state:R running task stack:0 pid:6595 tgid:6595 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: wg-kex-wg0 wg_packet_handshake_receive_worker Call trace: __switch_to+0x418/0x87c arch/arm64/kernel/process.c:741 (T) context_switch kernel/sched/core.c:5357 [inline] __schedule+0x13b0/0x2864 kernel/sched/core.c:6961 preempt_schedule_common+0xe0/0x194 kernel/sched/core.c:7145 preempt_schedule+0x60/0x80 kernel/sched/core.c:7169 __local_bh_enable_ip+0x20c/0x35c kernel/softirq.c:414 local_bh_enable+0x28/0x34 include/linux/bottom_half.h:33 rcu_read_unlock_bh include/linux/rcupdate.h:910 [inline] wg_packet_create_data drivers/net/wireguard/send.c:325 [inline] wg_packet_send_staged_packets+0xdac/0x1288 drivers/net/wireguard/send.c:388 wg_packet_send_keepalive+0x140/0x240 drivers/net/wireguard/send.c:239 wg_receive_handshake_packet drivers/net/wireguard/receive.c:186 [inline] wg_packet_handshake_receive_worker+0x550/0xc20 drivers/net/wireguard/receive.c:213 process_one_work+0x7e8/0x155c kernel/workqueue.c:3236 process_scheduled_works kernel/workqueue.c:3319 [inline] worker_thread+0x958/0xed8 kernel/workqueue.c:3400 kthread+0x5fc/0x75c kernel/kthread.c:463 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:844 pool 2: task:kworker/0:2 state:R running task stack:0 pid:2323 tgid:2323 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: wg-kex-wg2 wg_packet_handshake_receive_worker Call trace: __switch_to+0x418/0x87c arch/arm64/kernel/process.c:741 (T) context_switch kernel/sched/core.c:5357 [inline] __schedule+0x13b0/0x2864 kernel/sched/core.c:6961 preempt_schedule_common+0xe0/0x194 kernel/sched/core.c:7145 preempt_schedule+0x60/0x80 kernel/sched/core.c:7169 __local_bh_enable_ip+0x20c/0x35c kernel/softirq.c:414 local_bh_enable+0x28/0x34 include/linux/bottom_half.h:33 rcu_read_unlock_bh include/linux/rcupdate.h:910 [inline] mod_peer_timer+0x1e0/0x220 drivers/net/wireguard/timers.c:38 wg_timers_any_authenticated_packet_traversal+0x98/0xc0 drivers/net/wireguard/timers.c:218 wg_packet_send_handshake_response+0x114/0x24c drivers/net/wireguard/send.c:99 wg_receive_handshake_packet drivers/net/wireguard/receive.c:154 [inline] wg_packet_handshake_receive_worker+0x4fc/0xc20 drivers/net/wireguard/receive.c:213 process_one_work+0x7e8/0x155c kernel/workqueue.c:3236 process_scheduled_works kernel/workqueue.c:3319 [inline] worker_thread+0x958/0xed8 kernel/workqueue.c:3400 kthread+0x5fc/0x75c kernel/kthread.c:463 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:844 bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0) bridge0: received packet on veth0_to_bridge with own address as source address (addr:2e:9e:d6:17:13:74, vlan:0) bridge0: received packet on veth0_to_bridge with own address as source address (addr:2e:9e:d6:17:13:74, vlan:0) bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0) bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)