====================================================== WARNING: possible circular locking dependency detected syzkaller #0 Tainted: G L ------------------------------------------------------ kworker/u35:2/8856 is trying to acquire lock: ffff888013c93660 (sk_lock-AF_INET-NBD){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1713 [inline] ffff888013c93660 (sk_lock-AF_INET-NBD){+.+.}-{0:0}, at: inet_shutdown+0x67/0x410 net/ipv4/af_inet.c:913 but task is already holding lock: ffff888072426a68 (&nsock->tx_lock){+.+.}-{4:4}, at: recv_work+0x5e7/0x8c0 drivers/block/nbd.c:1020 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #6 (&nsock->tx_lock){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 nbd_handle_cmd drivers/block/nbd.c:1143 [inline] nbd_queue_rq+0x428/0x1080 drivers/block/nbd.c:1207 blk_mq_dispatch_rq_list+0x422/0x1e70 block/blk-mq.c:2117 __blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline] blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline] __blk_mq_sched_dispatch_requests+0xcea/0x1620 block/blk-mq-sched.c:307 blk_mq_sched_dispatch_requests+0xd7/0x1c0 block/blk-mq-sched.c:329 blk_mq_run_work_fn+0x135/0x380 block/blk-mq.c:2531 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322 process_scheduled_works kernel/workqueue.c:3405 [inline] worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486 kthread+0x370/0x450 kernel/kthread.c:436 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #5 (&cmd->lock){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 nbd_queue_rq+0xba/0x1080 drivers/block/nbd.c:1199 blk_mq_dispatch_rq_list+0x422/0x1e70 block/blk-mq.c:2117 __blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline] blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline] __blk_mq_sched_dispatch_requests+0xcea/0x1620 block/blk-mq-sched.c:307 blk_mq_sched_dispatch_requests+0xd7/0x1c0 block/blk-mq-sched.c:329 blk_mq_run_work_fn+0x135/0x380 block/blk-mq.c:2531 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322 process_scheduled_works kernel/workqueue.c:3405 [inline] worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486 kthread+0x370/0x450 kernel/kthread.c:436 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #4 (set->srcu){.+.+}-{0:0}: lock_sync kernel/locking/lockdep.c:5916 [inline] lock_sync+0x9d/0x110 kernel/locking/lockdep.c:5904 srcu_lock_sync include/linux/srcu.h:199 [inline] __synchronize_srcu+0xa2/0x300 kernel/rcu/srcutree.c:1481 blk_mq_wait_quiesce_done block/blk-mq.c:284 [inline] blk_mq_wait_quiesce_done block/blk-mq.c:281 [inline] blk_mq_quiesce_queue block/blk-mq.c:304 [inline] blk_mq_quiesce_queue+0x149/0x1c0 block/blk-mq.c:299 elevator_switch+0x17b/0x7e0 block/elevator.c:576 elevator_change+0x352/0x530 block/elevator.c:681 elevator_set_default+0x29e/0x360 block/elevator.c:754 blk_register_queue+0x48e/0x630 block/blk-sysfs.c:992 __add_disk+0x73f/0xe40 block/genhd.c:524 add_disk_fwnode+0x118/0x480 block/genhd.c:593 add_disk include/linux/blkdev.h:800 [inline] nbd_dev_add+0x77a/0xb10 drivers/block/nbd.c:2021 nbd_init+0x291/0x2b0 drivers/block/nbd.c:2729 do_one_initcall+0x11d/0x700 init/main.c:1347 do_initcall_level init/main.c:1409 [inline] do_initcalls init/main.c:1425 [inline] do_basic_setup init/main.c:1445 [inline] kernel_init_freeable+0x6ea/0x7b0 init/main.c:1658 kernel_init+0x1f/0x1e0 init/main.c:1548 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #3 (&q->elevator_lock){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x1a4/0x1bd0 kernel/locking/mutex.c:821 elevator_change+0x1bc/0x530 block/elevator.c:679 elevator_set_none+0x92/0xf0 block/elevator.c:769 blk_mq_elv_switch_none block/blk-mq.c:5101 [inline] __blk_mq_update_nr_hw_queues block/blk-mq.c:5146 [inline] blk_mq_update_nr_hw_queues+0x4c1/0x15f0 block/blk-mq.c:5211 nbd_start_device+0x1a6/0xbd0 drivers/block/nbd.c:1526 nbd_genl_connect+0xff2/0x1a40 drivers/block/nbd.c:2276 genl_family_rcv_msg_doit+0x214/0x300 net/netlink/genetlink.c:1114 genl_family_rcv_msg net/netlink/genetlink.c:1194 [inline] genl_rcv_msg+0x560/0x800 net/netlink/genetlink.c:1209 netlink_rcv_skb+0x159/0x420 net/netlink/af_netlink.c:2556 genl_rcv+0x28/0x40 net/netlink/genetlink.c:1218 netlink_unicast_kernel net/netlink/af_netlink.c:1319 [inline] netlink_unicast+0x585/0x850 net/netlink/af_netlink.c:1345 netlink_sendmsg+0x8b0/0xda0 net/netlink/af_netlink.c:1900 sock_sendmsg_nosec net/socket.c:775 [inline] __sock_sendmsg net/socket.c:790 [inline] ____sys_sendmsg+0xa4d/0xbe0 net/socket.c:2684 ___sys_sendmsg+0x190/0x1e0 net/socket.c:2738 __sys_sendmsg+0x160/0x210 net/socket.c:2770 do_syscall_32_irqs_on arch/x86/entry/syscall_32.c:83 [inline] __do_fast_syscall_32+0xe7/0x970 arch/x86/entry/syscall_32.c:307 do_fast_syscall_32+0x32/0x70 arch/x86/entry/syscall_32.c:332 entry_SYSENTER_compat_after_hwframe+0x84/0x8e -> #2 (&q->q_usage_counter(io)#49){++++}-{0:0}: lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 blk_alloc_queue+0x610/0x790 block/blk-core.c:504 blk_mq_alloc_queue+0x174/0x290 block/blk-mq.c:4420 __blk_mq_alloc_disk+0x29/0x120 block/blk-mq.c:4467 nbd_dev_add+0x492/0xb10 drivers/block/nbd.c:1991 nbd_init+0x291/0x2b0 drivers/block/nbd.c:2729 do_one_initcall+0x11d/0x700 init/main.c:1347 do_initcall_level init/main.c:1409 [inline] do_initcalls init/main.c:1425 [inline] do_basic_setup init/main.c:1445 [inline] kernel_init_freeable+0x6ea/0x7b0 init/main.c:1658 kernel_init+0x1f/0x1e0 init/main.c:1548 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #1 (fs_reclaim){+.+.}-{0:0}: lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 __fs_reclaim_acquire mm/page_alloc.c:4329 [inline] fs_reclaim_acquire+0xc4/0x100 mm/page_alloc.c:4343 might_alloc include/linux/sched/mm.h:317 [inline] slab_pre_alloc_hook mm/slub.c:4565 [inline] slab_alloc_node mm/slub.c:4925 [inline] __kmalloc_cache_noprof+0x63/0x6c0 mm/slub.c:5511 _kmalloc_noprof include/linux/slab.h:969 [inline] _kzalloc_noprof include/linux/slab.h:1290 [inline] tcp_sendmsg_fastopen+0x24d/0x750 net/ipv4/tcp.c:1067 tcp_sendmsg_locked+0x276d/0x44d0 net/ipv4/tcp.c:1181 tcp_sendmsg+0x2e/0x50 net/ipv4/tcp.c:1451 inet_sendmsg+0xb9/0x140 net/ipv4/af_inet.c:865 sock_sendmsg_nosec net/socket.c:775 [inline] __sock_sendmsg net/socket.c:790 [inline] __sys_sendto+0x453/0x4e0 net/socket.c:2252 __do_sys_sendto net/socket.c:2259 [inline] __se_sys_sendto net/socket.c:2255 [inline] __ia32_sys_sendto+0xdd/0x1b0 net/socket.c:2255 do_syscall_32_irqs_on arch/x86/entry/syscall_32.c:83 [inline] __do_fast_syscall_32+0xe7/0x970 arch/x86/entry/syscall_32.c:307 do_fast_syscall_32+0x32/0x70 arch/x86/entry/syscall_32.c:332 entry_SYSENTER_compat_after_hwframe+0x84/0x8e -> #0 (sk_lock-AF_INET-NBD){+.+.}-{0:0}: check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3165 check_prevs_add kernel/locking/lockdep.c:3284 [inline] validate_chain kernel/locking/lockdep.c:3908 [inline] __lock_acquire+0x136c/0x1a40 kernel/locking/lockdep.c:5237 lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 lock_sock_nested+0x41/0xf0 net/core/sock.c:3825 lock_sock include/net/sock.h:1713 [inline] inet_shutdown+0x67/0x410 net/ipv4/af_inet.c:913 nbd_mark_nsock_dead+0xae/0x5c0 drivers/block/nbd.c:318 recv_work+0x5fb/0x8c0 drivers/block/nbd.c:1021 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322 process_scheduled_works kernel/workqueue.c:3405 [inline] worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486 kthread+0x370/0x450 kernel/kthread.c:436 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 other info that might help us debug this: Chain exists of: sk_lock-AF_INET-NBD --> &cmd->lock --> &nsock->tx_lock Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&nsock->tx_lock); lock(&cmd->lock); lock(&nsock->tx_lock); lock(sk_lock-AF_INET-NBD); *** DEADLOCK *** 3 locks held by kworker/u35:2/8856: #0: ffff88802503c940 ((wq_completion)nbd2-recv){+.+.}-{0:0}, at: process_one_work+0x12b1/0x1940 kernel/workqueue.c:3297 #1: ffffc9002b0b7d08 ((work_completion)(&args->work)){+.+.}-{0:0}, at: process_one_work+0x988/0x1940 kernel/workqueue.c:3298 #2: ffff888072426a68 (&nsock->tx_lock){+.+.}-{4:4}, at: recv_work+0x5e7/0x8c0 drivers/block/nbd.c:1020 stack backtrace: CPU: 3 UID: 0 PID: 8856 Comm: kworker/u35:2 Tainted: G L syzkaller #0 PREEMPT(full) Tainted: [L]=SOFTLOCKUP Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 Workqueue: nbd2-recv recv_work Call Trace: __dump_stack lib/dump_stack.c:94 [inline] dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120 print_circular_bug.cold+0x178/0x1c7 kernel/locking/lockdep.c:2043 check_noncircular+0x146/0x160 kernel/locking/lockdep.c:2175 check_prev_add+0xeb/0xe60 kernel/locking/lockdep.c:3165 check_prevs_add kernel/locking/lockdep.c:3284 [inline] validate_chain kernel/locking/lockdep.c:3908 [inline] __lock_acquire+0x136c/0x1a40 kernel/locking/lockdep.c:5237 lock_acquire kernel/locking/lockdep.c:5868 [inline] lock_acquire+0x1b9/0x370 kernel/locking/lockdep.c:5825 lock_sock_nested+0x41/0xf0 net/core/sock.c:3825 lock_sock include/net/sock.h:1713 [inline] inet_shutdown+0x67/0x410 net/ipv4/af_inet.c:913 nbd_mark_nsock_dead+0xae/0x5c0 drivers/block/nbd.c:318 recv_work+0x5fb/0x8c0 drivers/block/nbd.c:1021 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322 process_scheduled_works kernel/workqueue.c:3405 [inline] worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486 kthread+0x370/0x450 kernel/kthread.c:436 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245