====================================================== WARNING: possible circular locking dependency detected 6.7.0-rc4-syzkaller-00858-g70028b2e51c6 #0 Not tainted ------------------------------------------------------ kworker/u4:0/11 is trying to acquire lock: ffff888021182530 (&rs->rs_recv_lock){....}-{2:2}, at: rds_wake_sk_sleep+0x23/0xe0 net/rds/af_rds.c:109 but task is already holding lock: ffff8880287fd900 (&rm->m_rs_lock){....}-{2:2}, at: rds_send_remove_from_sock+0x155/0xa50 net/rds/send.c:628 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&rm->m_rs_lock){....}-{2:2}: __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x3a/0x50 kernel/locking/spinlock.c:162 rds_message_purge net/rds/message.c:138 [inline] rds_message_put+0x1dd/0xc50 net/rds/message.c:180 rds_inc_put+0x13c/0x1a0 net/rds/recv.c:83 rds_clear_recv_queue+0x14c/0x350 net/rds/recv.c:768 rds_release+0xdb/0x3c0 net/rds/af_rds.c:73 __sock_release+0xae/0x260 net/socket.c:659 sock_close+0x1c/0x20 net/socket.c:1419 __fput+0x270/0xbb0 fs/file_table.c:394 __fput_sync+0x47/0x50 fs/file_table.c:475 __do_sys_close fs/open.c:1590 [inline] __se_sys_close fs/open.c:1575 [inline] __x64_sys_close+0x87/0xf0 fs/open.c:1575 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0x40/0x110 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x63/0x6b -> #0 (&rs->rs_recv_lock){....}-{2:2}: check_prev_add kernel/locking/lockdep.c:3134 [inline] check_prevs_add kernel/locking/lockdep.c:3253 [inline] validate_chain kernel/locking/lockdep.c:3869 [inline] __lock_acquire+0x2433/0x3b20 kernel/locking/lockdep.c:5137 lock_acquire kernel/locking/lockdep.c:5754 [inline] lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719 __raw_read_lock_irqsave include/linux/rwlock_api_smp.h:160 [inline] _raw_read_lock_irqsave+0x46/0x90 kernel/locking/spinlock.c:236 rds_wake_sk_sleep+0x23/0xe0 net/rds/af_rds.c:109 rds_send_remove_from_sock+0x1e9/0xa50 net/rds/send.c:634 rds_send_path_drop_acked+0x2f4/0x3c0 net/rds/send.c:710 rds_tcp_write_space+0x1b5/0x6d0 net/rds/tcp_send.c:199 tcp_new_space net/ipv4/tcp_input.c:5608 [inline] tcp_check_space+0x178/0x8d0 net/ipv4/tcp_input.c:5627 tcp_data_snd_check net/ipv4/tcp_input.c:5636 [inline] tcp_rcv_established+0x959/0x20e0 net/ipv4/tcp_input.c:6154 tcp_v4_do_rcv+0x68c/0xa10 net/ipv4/tcp_ipv4.c:1912 sk_backlog_rcv include/net/sock.h:1121 [inline] __release_sock+0x132/0x3b0 net/core/sock.c:2968 release_sock+0x5a/0x1f0 net/core/sock.c:3534 rds_send_xmit+0x164f/0x2490 net/rds/send.c:422 rds_send_worker+0x8f/0x2e0 net/rds/threads.c:200 process_one_work+0x886/0x15d0 kernel/workqueue.c:2627 process_scheduled_works kernel/workqueue.c:2700 [inline] worker_thread+0x8b9/0x1290 kernel/workqueue.c:2781 kthread+0x2c6/0x3a0 kernel/kthread.c:388 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242 other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&rm->m_rs_lock); lock(&rs->rs_recv_lock); lock(&rm->m_rs_lock); rlock(&rs->rs_recv_lock); *** DEADLOCK *** 5 locks held by kworker/u4:0/11: #0: ffff88814bfaa138 ((wq_completion)krdsd){+.+.}-{0:0}, at: process_one_work+0x789/0x15d0 kernel/workqueue.c:2602 #1: ffffc90000107d80 ((work_completion)(&(&cp->cp_send_w)->work)){+.+.}-{0:0}, at: process_one_work+0x7eb/0x15d0 kernel/workqueue.c:2603 #2: ffff8880263558b0 (k-sk_lock-AF_INET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1720 [inline] #2: ffff8880263558b0 (k-sk_lock-AF_INET){+.+.}-{0:0}, at: tcp_sock_set_cork+0x1c/0x90 net/ipv4/tcp.c:3230 #3: ffff888026355b38 (k-clock-AF_INET){++.-}-{2:2}, at: rds_tcp_write_space+0x29/0x6d0 net/rds/tcp_send.c:185 #4: ffff8880287fd900 (&rm->m_rs_lock){....}-{2:2}, at: rds_send_remove_from_sock+0x155/0xa50 net/rds/send.c:628 stack backtrace: CPU: 1 PID: 11 Comm: kworker/u4:0 Not tainted 6.7.0-rc4-syzkaller-00858-g70028b2e51c6 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/10/2023 Workqueue: krdsd rds_send_worker Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106 check_noncircular+0x317/0x400 kernel/locking/lockdep.c:2187 check_prev_add kernel/locking/lockdep.c:3134 [inline] check_prevs_add kernel/locking/lockdep.c:3253 [inline] validate_chain kernel/locking/lockdep.c:3869 [inline] __lock_acquire+0x2433/0x3b20 kernel/locking/lockdep.c:5137 lock_acquire kernel/locking/lockdep.c:5754 [inline] lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719 __raw_read_lock_irqsave include/linux/rwlock_api_smp.h:160 [inline] _raw_read_lock_irqsave+0x46/0x90 kernel/locking/spinlock.c:236 rds_wake_sk_sleep+0x23/0xe0 net/rds/af_rds.c:109 rds_send_remove_from_sock+0x1e9/0xa50 net/rds/send.c:634 rds_send_path_drop_acked+0x2f4/0x3c0 net/rds/send.c:710 rds_tcp_write_space+0x1b5/0x6d0 net/rds/tcp_send.c:199 tcp_new_space net/ipv4/tcp_input.c:5608 [inline] tcp_check_space+0x178/0x8d0 net/ipv4/tcp_input.c:5627 tcp_data_snd_check net/ipv4/tcp_input.c:5636 [inline] tcp_rcv_established+0x959/0x20e0 net/ipv4/tcp_input.c:6154 tcp_v4_do_rcv+0x68c/0xa10 net/ipv4/tcp_ipv4.c:1912 sk_backlog_rcv include/net/sock.h:1121 [inline] __release_sock+0x132/0x3b0 net/core/sock.c:2968 release_sock+0x5a/0x1f0 net/core/sock.c:3534 rds_send_xmit+0x164f/0x2490 net/rds/send.c:422 rds_send_worker+0x8f/0x2e0 net/rds/threads.c:200 process_one_work+0x886/0x15d0 kernel/workqueue.c:2627 process_scheduled_works kernel/workqueue.c:2700 [inline] worker_thread+0x8b9/0x1290 kernel/workqueue.c:2781 kthread+0x2c6/0x3a0 kernel/kthread.c:388 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242