sched: DL replenish lagged too much BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 13s! Showing busy workqueues and worker pools: workqueue events: flags=0x100 pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3 pending: virtio_gpu_dequeue_ctrl_func, ovs_dp_masks_rebalance pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: nsim_dev_hwstats_traffic_work workqueue events_unbound: flags=0x2 pwq 33: cpus=0-7 node=0 flags=0x4 nice=0 active=2 refcnt=3 in-flight: 74:nsim_dev_trap_report_work for 0s pending: toggle_allocation_gate workqueue events_power_efficient: flags=0x180 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=4 refcnt=5 in-flight: 1518:gc_worker for 13s pending: 2*neigh_managed_work, neigh_periodic_work workqueue events_dfl_long: flags=0x2 pwq 33: cpus=0-7 node=0 flags=0x4 nice=0 active=1 refcnt=2 pending: defense_work_handler workqueue mm_percpu_wq: flags=0x108 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: vmstat_update workqueue bat_events: flags=0x6000a pwq 32: cpus=0-7 flags=0x4 nice=0 active=1 refcnt=189 in-flight: 5766:batadv_tt_purge for 19s inactive: batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_tt_purge, batadv_iv_send_outstanding_bat_ogm_packet, 6*batadv_tt_purge, 2*batadv_purge_orig, 4*batadv_tt_purge, batadv_iv_send_outstanding_bat_ogm_packet, 4*batadv_tt_purge, batadv_purge_orig, 4*batadv_tt_purge, batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 4*batadv_tt_purge, batadv_iv_send_outstanding_bat_ogm_packet, 4*batadv_tt_purge, batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, batadv_purge_orig, 4*batadv_iv_send_outstanding_bat_ogm_packet, 28*batadv_mcast_mla_update, 6*batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, 2*batadv_mcast_mla_update, batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, batadv_purge_orig, 4*batadv_iv_send_outstanding_bat_ogm_packet, 6*batadv_purge_orig , 2*batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 3*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, 2*batadv_iv_send_outstanding_bat_ogm_packet, batadv_purge_orig, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work , batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_tt_purge pool 6: cpus=1 node=0 flags=0x0 nice=0 hung=13s workers=4 idle: 63 5824 30 pool 32: cpus=0-7 flags=0x4 nice=0 hung=0s workers=5 idle: 12 5794 5826 13 pool 33: cpus=0-7 node=0 flags=0x4 nice=0 hung=0s workers=14 idle: 37 1162 38 62 1164 1163 221 1161 1165 1066 137 46 1166 Showing backtraces of busy workers in stalled worker pools: pool 6: task:kworker/1:2 state:R running task stack:24424 pid:1518 tgid:1518 ppid:2 task_flags:0x4208060 flags:0x00080000 Workqueue: events_power_efficient gc_worker Call Trace: context_switch kernel/sched/core.c:5526 [inline] __schedule+0x1259/0x6920 kernel/sched/core.c:7277 preempt_schedule_irq+0x4e/0x90 kernel/sched/core.c:7599 irqentry_exit_to_kernel_mode_preempt include/linux/irq-entry-common.h:468 [inline] irqentry_exit_to_kernel_mode include/linux/irq-entry-common.h:539 [inline] irqentry_exit+0x205/0xa10 kernel/entry/common.c:167 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:674 RIP: 0010:__seqprop_spinlock_sequence include/linux/seqlock.h:228 [inline] RIP: 0010:nf_conntrack_get_ht include/net/netfilter/nf_conntrack.h:343 [inline] RIP: 0010:gc_worker+0x2bf/0x1a10 net/netfilter/nf_conntrack_core.c:1534 Code: 00 00 48 c7 c7 28 db 45 91 e8 2d d4 f9 f7 9c 5b 81 e3 00 02 00 00 31 ff 48 89 de e8 8b 96 1e f8 58 48 85 db 0f 85 32 13 00 00 dc 9b 1e f8 8b 1d e6 59 58 07 89 dd 31 ff 83 e5 01 89 ee e8 38 RSP: 0018:ffffc9000668fb28 EFLAGS: 00000293 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 RDX: ffff88802bf48000 RSI: ffffffff89ed8f10 RDI: ffff88802bf48000 RBP: ffff888036200000 R08: 0000000000000007 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000001 R12: dffffc0000000000 R13: 0000000000000000 R14: 0000000000010000 R15: 000000000000b919 process_one_work+0xac7/0x1b10 kernel/workqueue.c:3396 process_scheduled_works kernel/workqueue.c:3479 [inline] worker_thread+0x5ef/0xe50 kernel/workqueue.c:3560 kthread+0x373/0x450 kernel/kthread.c:436 ret_from_fork+0x730/0xd60 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 ---------------- Code disassembly (best guess): 0: 00 00 add %al,(%rax) 2: 48 c7 c7 28 db 45 91 mov $0xffffffff9145db28,%rdi 9: e8 2d d4 f9 f7 call 0xf7f9d43b e: 9c pushf f: 5b pop %rbx 10: 81 e3 00 02 00 00 and $0x200,%ebx 16: 31 ff xor %edi,%edi 18: 48 89 de mov %rbx,%rsi 1b: e8 8b 96 1e f8 call 0xf81e96ab 20: 58 pop %rax 21: 48 85 db test %rbx,%rbx 24: 0f 85 32 13 00 00 jne 0x135c * 2a: e8 dc 9b 1e f8 call 0xf81e9c0b <-- trapping instruction 2f: 8b 1d e6 59 58 07 mov 0x75859e6(%rip),%ebx # 0x7585a1b 35: 89 dd mov %ebx,%ebp 37: 31 ff xor %edi,%edi 39: 83 e5 01 and $0x1,%ebp 3c: 89 ee mov %ebp,%esi 3e: e8 .byte 0xe8 3f: 38 .byte 0x38