BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 142s!
BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=-20 stuck for 140s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x100
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=15 refcnt=16
in-flight: 14890:drain_vmap_area_work drain_vmap_area_work
pending: psi_avgs_work, l2cap_info_timeout, destroy_super_work, 5*nsim_dev_hwstats_traffic_work, rfkill_sync_work, rfkill_global_led_trigger_worker, vmstat_shepherd, 2*ovs_dp_masks_rebalance
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=15 refcnt=16
pending: 5*nsim_dev_hwstats_traffic_work, psi_avgs_work, 4*ovs_dp_masks_rebalance, psi_avgs_work, free_obj_work, 3*psi_avgs_work
workqueue events_long: flags=0x100
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=6 refcnt=7
pending: 5*defense_work_handler, br_fdb_cleanup
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=5 refcnt=6
pending: 2*defense_work_handler, br_multicast_gc_work, 2*br_fdb_cleanup
workqueue events_unbound: flags=0x2
pwq 8: cpus=0-1 flags=0x4 nice=0 active=20 refcnt=21
in-flight: 13736:nsim_dev_trap_report_work ,13742:cfg80211_wiphy_work cfg80211_wiphy_work ,6422:cfg80211_wiphy_work cfg80211_wiphy_work ,13747:nsim_dev_trap_report_work ,13734:toggle_allocation_gate
pending: 3*nsim_dev_trap_report_work, cfg80211_wiphy_work, 5*nsim_dev_trap_report_work, flush_memcg_stats_dwork, cfg80211_wiphy_work, idle_cull_fn, macvlan_process_broadcast
pwq 8: cpus=0-1 flags=0x4 nice=0 active=7 refcnt=8
in-flight: 49:cfg80211_wiphy_work cfg80211_wiphy_work ,13749:cfg80211_wiphy_work
pending: 4*cfg80211_wiphy_work
workqueue events_freezable: flags=0x104
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: update_balloon_stats_func
workqueue events_power_efficient: flags=0x180
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
in-flight: 14889:wg_ratelimiter_gc_entries
pending: check_lifetime
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=12 refcnt=13
pending: 2*nf_flow_offload_work_gc, 2*neigh_managed_work, 2*neigh_periodic_work, do_cache_clean, check_lifetime, gc_worker, 3*check_lifetime
workqueue mm_percpu_wq: flags=0x108
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: vmstat_update
workqueue writeback: flags=0x4a
pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
pending: wb_update_bandwidth_workfn
workqueue kblockd: flags=0x18
pwq 3: cpus=0 node=0 flags=0x0 nice=-20 active=1 refcnt=2
pending: blk_mq_timeout_work
pwq 7: cpus=1 node=0 flags=0x0 nice=-20 active=1 refcnt=2
pending: blk_mq_requeue_work
workqueue bat_events: flags=0x6000a
pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=26
pending: batadv_mcast_mla_update
inactive: 2*batadv_mcast_mla_update, 3*batadv_purge_orig, 9*batadv_iv_send_outstanding_bat_ogm_packet, 3*batadv_tt_purge, batadv_dat_purge, 2*batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work
workqueue wg-kex-wg0: flags=0x6
pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_send_worker
workqueue wg-kex-wg1: flags=0x6
pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
in-flight: 6409:wg_packet_handshake_send_worker
workqueue wg-crypt-wg1: flags=0x128
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x124
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_tx_worker
workqueue wg-crypt-wg2: flags=0x128
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_encrypt_worker
workqueue wg-crypt-wg1: flags=0x128
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_encrypt_worker
workqueue wg-crypt-wg1: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_encrypt_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=3 refcnt=4
pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x124
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_encrypt_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
in-flight: 7729:wg_packet_encrypt_worker
workqueue wg-crypt-wg1: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
pending: wg_packet_encrypt_worker, wg_packet_tx_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x124
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x124
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x124
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
pending: wg_packet_encrypt_worker, wg_packet_decrypt_worker, wg_packet_tx_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_encrypt_worker
workqueue wg-kex-wg2: flags=0x124
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x128
pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue hci5: flags=0x20012
workqueue hci5: flags=0x20012
pwq 9: cpus=0-1 node=0 flags=0x4 nice=-20 active=1 refcnt=4
in-flight: 11295:hci_power_on
pool 2: cpus=0 node=0 flags=0x0 nice=0 hung=13s workers=15 idle: 14894 14891 14892 5878 9 87 5961 12106 5962 10 5822 14897 14895
pool 6: cpus=1 node=0 flags=0x0 nice=0 hung=0s workers=6 idle: 7727 11502 5947
pool 8: cpus=0-1 flags=0x4 nice=0 hung=0s workers=33 idle: 13747 13752 13742 13744 13734 13732 13726 49 13749 1150 13733 6430 6411 6433 6431 3568 6419 153 12 13731 61 13737 35 13727 13753 6424 13754 6436
pool 9: cpus=0-1 node=0 flags=0x4 nice=-20 hung=0s workers=8 idle: 5829 11293 5142 11296 5838 5824 11294
Showing backtraces of running workers in stalled CPU-bound worker pools:
pool 6:
task:kworker/1:8 state:R running task stack:24408 pid:7723 tgid:7723 ppid:2 task_flags:0x4208060 flags:0x00080000
Workqueue: events_power_efficient neigh_periodic_work
Call Trace:
context_switch kernel/sched/core.c:5295 [inline]
__schedule+0x157c/0x52b0 kernel/sched/core.c:6907
preempt_schedule_common+0x82/0xd0 kernel/sched/core.c:7091
preempt_schedule_thunk+0x16/0x30 arch/x86/entry/thunk.S:12
__local_bh_enable_ip+0xe1/0x130 kernel/softirq.c:457
spin_unlock_bh include/linux/spinlock.h:395 [inline]
neigh_periodic_work+0xc28/0xe50 net/core/neighbour.c:1038
process_one_work kernel/workqueue.c:3275 [inline]
process_scheduled_works+0xaec/0x17a0 kernel/workqueue.c:3358
worker_thread+0xa50/0xfc0 kernel/workqueue.c:3439
kthread+0x388/0x470 kernel/kthread.c:467
ret_from_fork+0x51e/0xb90 arch/x86/kernel/process.c:158
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
pool 6:
task:kworker/1:13 state:R running task stack:23560 pid:7729 tgid:7729 ppid:2 task_flags:0x4208060 flags:0x00080000
Workqueue: events_power_efficient gc_worker
Call Trace: