BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 209s! Showing busy workqueues and worker pools: workqueue events: flags=0x0 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=4/256 in-flight: 7253:free_obj_work pending: free_obj_work, key_garbage_collector, free_ioctx pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 in-flight: 22:bpf_map_free_deferred, 7566:bpf_map_free_deferred workqueue mm_percpu_wq: flags=0x8 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 pending: lru_add_drain_per_cpu BAR(11501), vmstat_update workqueue netns: flags=0xe000a pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/1 in-flight: 11201:cleanup_net ====================================================== WARNING: possible circular locking dependency detected 4.14.71+ #8 Not tainted ------------------------------------------------------ syz-executor4/10207 is trying to acquire lock: (&port_lock_key){-.-.}, at: [] serial8250_console_write+0x6af/0x800 drivers/tty/serial/8250/8250_port.c:3232 but task is already holding lock: (&pool->lock/1){..-.}, at: [] show_workqueue_state.cold.23+0x278/0x1053 kernel/workqueue.c:4495 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (&pool->lock/1){..-.}: __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:152 spin_lock include/linux/spinlock.h:317 [inline] __queue_work+0x184/0xe40 kernel/workqueue.c:1419 queue_work_on+0x8f/0xa0 kernel/workqueue.c:1488 pty_write+0x172/0x1c0 drivers/tty/pty.c:124 n_tty_write+0x969/0xda0 drivers/tty/n_tty.c:2342 do_tty_write drivers/tty/tty_io.c:957 [inline] tty_write+0x397/0x810 drivers/tty/tty_io.c:1041 __vfs_write+0xf4/0x5c0 fs/read_write.c:482 vfs_write+0x17f/0x4d0 fs/read_write.c:546 SYSC_write fs/read_write.c:593 [inline] SyS_write+0xc2/0x1a0 fs/read_write.c:585 do_syscall_64+0x19b/0x4b0 arch/x86/entry/common.c:289 entry_SYSCALL_64_after_hwframe+0x42/0xb7 -> #1 (&(&port->lock)->rlock){-.-.}: __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x41/0x50 kernel/locking/spinlock.c:160 tty_port_tty_get+0x1b/0x80 drivers/tty/tty_port.c:287 tty_port_default_wakeup+0x11/0x40 drivers/tty/tty_port.c:46 serial8250_tx_chars+0x417/0xa70 drivers/tty/serial/8250/8250_port.c:1810 serial8250_handle_irq.part.15+0x1b1/0x1f0 drivers/tty/serial/8250/8250_port.c:1883 serial8250_handle_irq drivers/tty/serial/8250/8250_port.c:1869 [inline] serial8250_default_handle_irq+0x9b/0x110 drivers/tty/serial/8250/8250_port.c:1899 serial8250_interrupt+0xea/0x1a0 drivers/tty/serial/8250/8250_core.c:129 __handle_irq_event_percpu+0xf8/0x770 kernel/irq/handle.c:147 handle_irq_event_percpu+0x76/0x150 kernel/irq/handle.c:187 handle_irq_event+0xa1/0x12d kernel/irq/handle.c:204 handle_edge_irq+0x1f0/0x7f0 kernel/irq/chip.c:770 generic_handle_irq_desc include/linux/irqdesc.h:158 [inline] handle_irq+0x242/0x328 arch/x86/kernel/irq_64.c:78 do_IRQ+0x7d/0x1b0 arch/x86/kernel/irq.c:230 ret_from_intr+0x0/0x22 native_safe_halt+0x2/0x10 arch/x86/include/asm/irqflags.h:57 arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline] default_idle+0x50/0x380 arch/x86/kernel/process.c:501 cpuidle_idle_call kernel/sched/idle.c:159 [inline] do_idle+0x2da/0x3b0 kernel/sched/idle.c:268 cpu_startup_entry+0xc9/0xe0 kernel/sched/idle.c:374 start_kernel+0x6fb/0x739 init/main.c:710 secondary_startup_64+0xa5/0xb0 arch/x86/kernel/head_64.S:239 -> #0 (&port_lock_key){-.-.}: lock_acquire+0x10f/0x380 kernel/locking/lockdep.c:3991 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x41/0x50 kernel/locking/spinlock.c:160 serial8250_console_write+0x6af/0x800 drivers/tty/serial/8250/8250_port.c:3232 call_console_drivers kernel/printk/printk.c:1574 [inline] console_unlock+0x5b9/0xb50 kernel/printk/printk.c:2242 vprintk_emit+0x112/0x150 kernel/printk/printk.c:1766 vprintk_func+0x58/0x159 kernel/printk/printk_safe.c:401 printk+0xa7/0xcf kernel/printk/printk.c:1838 show_pwq kernel/workqueue.c:4402 [inline] show_workqueue_state.cold.23+0x3d5/0x1053 kernel/workqueue.c:4497 wq_watchdog_timer_fn+0x436/0x4d0 kernel/workqueue.c:5481 call_timer_fn+0x163/0x6a0 kernel/time/timer.c:1279 expire_timers+0x1f3/0x4a0 kernel/time/timer.c:1318 __run_timers kernel/time/timer.c:1634 [inline] run_timer_softirq+0x3a5/0x560 kernel/time/timer.c:1649 __do_softirq+0x215/0x997 kernel/softirq.c:288 invoke_softirq kernel/softirq.c:368 [inline] irq_exit+0x10f/0x150 kernel/softirq.c:409 exiting_irq arch/x86/include/asm/apic.h:648 [inline] smp_apic_timer_interrupt+0x188/0x5f0 arch/x86/kernel/apic/apic.c:1064 apic_timer_interrupt+0x84/0x90 arch/x86/entry/entry_64.S:787 other info that might help us debug this: Chain exists of: &port_lock_key --> &(&port->lock)->rlock --> &pool->lock/1 Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&pool->lock/1); lock(&(&port->lock)->rlock); lock(&pool->lock/1); lock(&port_lock_key); *** DEADLOCK *** 4 locks held by syz-executor4/10207: #0: (kernel/workqueue.c:5419){+.-.}, at: [] lockdep_copy_map include/linux/lockdep.h:174 [inline] #0: (kernel/workqueue.c:5419){+.-.}, at: [] call_timer_fn+0xd7/0x6a0 kernel/time/timer.c:1269 #1: (rcu_read_lock_sched){....}, at: [] show_workqueue_state+0x0/0x100 kernel/workqueue.c:4367 #2: (&pool->lock/1){..-.}, at: [] show_workqueue_state.cold.23+0x278/0x1053 kernel/workqueue.c:4495 #3: (console_lock){+.+.}, at: [] vprintk_emit+0x104/0x150 kernel/printk/printk.c:1765 stack backtrace: CPU: 0 PID: 10207 Comm: syz-executor4 Not tainted 4.14.71+ #8 Call Trace: __dump_stack lib/dump_stack.c:17 [inline] dump_stack+0xb9/0x11b lib/dump_stack.c:53 print_circular_bug.isra.18.cold.43+0x2d3/0x40c kernel/locking/lockdep.c:1258 check_prev_add kernel/locking/lockdep.c:1901 [inline] check_prevs_add kernel/locking/lockdep.c:2018 [inline] validate_chain kernel/locking/lockdep.c:2460 [inline] __lock_acquire+0x2ff9/0x4320 kernel/locking/lockdep.c:3487 lock_acquire+0x10f/0x380 kernel/locking/lockdep.c:3991 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x41/0x50 kernel/locking/spinlock.c:160 serial8250_console_write+0x6af/0x800 drivers/tty/serial/8250/8250_port.c:3232 call_console_drivers kernel/printk/printk.c:1574 [inline] console_unlock+0x5b9/0xb50 kernel/printk/printk.c:2242 vprintk_emit+0x112/0x150 kernel/printk/printk.c:1766 vprintk_func+0x58/0x159 kernel/printk/printk_safe.c:401 printk+0xa7/0xcf kernel/printk/printk.c:1838 show_pwq kernel/workqueue.c:4402 [inline] show_workqueue_state.cold.23+0x3d5/0x1053 kernel/workqueue.c:4497 wq_watchdog_timer_fn+0x436/0x4d0 kernel/workqueue.c:5481 call_timer_fn+0x163/0x6a0 kernel/time/timer.c:1279 expire_timers+0x1f3/0x4a0 kernel/time/timer.c:1318 __run_timers kernel/time/timer.c:1634 [inline] run_timer_softirq+0x3a5/0x560 kernel/time/timer.c:1649 __do_softirq+0x215/0x997 kernel/softirq.c:288 invoke_softirq kernel/softirq.c:368 [inline] irq_exit+0x10f/0x150 kernel/softirq.c:409 exiting_irq arch/x86/include/asm/apic.h:648 [inline] smp_apic_timer_interrupt+0x188/0x5f0 arch/x86/kernel/apic/apic.c:1064 apic_timer_interrupt+0x84/0x90 arch/x86/entry/entry_64.S:787 RIP: 0033:0x401574 RSP: 002b:00007f4e6bd19690 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff10 RAX: 000000003e2d2675 RBX: 000000000000000b RCX: 0000000000457679 RDX: 0000000000000000 RSI: 00007f4e6bd196c0 RDI: 000000000000000b RBP: 000000000072bf00 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff R13: 00000000004d7cf0 R14: 00000000004c49b7 R15: 0000000000000000 workqueue kblockd: flags=0x18 pwq 3: cpus=1 node=0 flags=0x0 nice=-20 active=1/256 pending: blk_mq_requeue_work workqueue ipv6_addrconf: flags=0x40008 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/1 pending: addrconf_verify_work pool 0: cpus=0 node=0 flags=0x0 nice=0 hung=0s workers=15 idle: 11586 11584 11583 11582 11581 11580 7564 8576 5424 3 72 11587 11585 pool 2: cpus=1 node=0 flags=0x0 nice=0 hung=210s workers=6 idle: 5360 68 10860 18 4710 pool 4: cpus=0-1 flags=0x4 nice=0 hung=5s workers=38 idle: 9358 11199 11212 11195 9235 9366 11198 11205 9233 11193 11188 64 11185 9239 11175 11183 11179 11167 11189 11169 11172 11177 9354 2210 373 9370 11163 9363 2492 11155 11171 2112 9357 4098 5 11215 11208