BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 33s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=8 refcnt=9
    in-flight: 25800:nsim_dev_hwstats_traffic_work
    pending: nsim_dev_hwstats_traffic_work, 2*nsim_dev_trap_report_work, nsim_dev_hwstats_traffic_work, 2*psi_avgs_work, kfree_rcu_monitor
workqueue events_power_efficient: flags=0x82
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 27820:wg_ratelimiter_gc_entries
workqueue mm_percpu_wq: flags=0x8
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: vmstat_update
workqueue kblockd: flags=0x18
  pwq 7: cpus=1 node=0 flags=0x0 nice=-20 active=1 refcnt=2
    in-flight: 24:blk_mq_timeout_work
workqueue dm_bufio_cache: flags=0x8
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: work_fn
workqueue wg-crypt-wg0: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
pool 6: cpus=1 node=0 flags=0x0 nice=0 hung=33s workers=6 idle: 27754 27749 27879 26193 27597
pool 7: cpus=1 node=0 flags=0x0 nice=-20 hung=0s workers=2 idle: 52
pool 8: cpus=0-1 flags=0x4 nice=0 hung=0s workers=7 idle: 958 23675 21041 25206 26733 25108
Showing backtraces of running workers in stalled CPU-bound worker pools:
pool 6:
task:kworker/1:0     state:R  running task     stack:0     pid:25800 tgid:25800 ppid:2      flags:0x00000008
Workqueue: events nsim_dev_hwstats_traffic_work
Call trace:
 __switch_to+0x1c8/0x2a0 arch/arm64/kernel/process.c:553
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0xad8/0x2b14 kernel/sched/core.c:6746
 preempt_schedule_irq+0x80/0x1a8 kernel/sched/core.c:7068
 arm64_preempt_schedule_irq+0x3c/0x54 arch/arm64/kernel/entry-common.c:301
 __el1_irq arch/arm64/kernel/entry-common.c:539 [inline]
 el1_interrupt+0x3c/0x54 arch/arm64/kernel/entry-common.c:551
 el1h_64_irq_handler+0x18/0x24 arch/arm64/kernel/entry-common.c:556
 el1h_64_irq+0x64/0x68 arch/arm64/kernel/entry.S:594
 __daif_local_irq_restore arch/arm64/include/asm/irqflags.h:176 [inline]
 arch_local_irq_restore arch/arm64/include/asm/irqflags.h:196 [inline]
 queue_delayed_work_on+0x6c/0xe4 kernel/workqueue.c:2599
 queue_delayed_work include/linux/workqueue.h:620 [inline]
 schedule_delayed_work include/linux/workqueue.h:734 [inline]
 nsim_dev_hwstats_traffic_work+0x144/0x1ac drivers/net/netdevsim/hwstats.c:51
 process_one_work+0x78c/0x1898 kernel/workqueue.c:3254
 process_scheduled_works kernel/workqueue.c:3335 [inline]
 worker_thread+0x764/0xc24 kernel/workqueue.c:3416
 kthread+0x27c/0x300 kernel/kthread.c:388
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:860