ci2 starts bisection 2023-07-10 09:38:43.383584698 +0000 UTC m=+236116.964135089 bisecting fixing commit since 0ab06468cbd149aac0d7f216ec00452ff8c74e0b building syzkaller on a4ae4f428721da42ac15f07d6f3b54584dedee27 ensuring issue is reproducible on original commit 0ab06468cbd149aac0d7f216ec00452ff8c74e0b testing commit 0ab06468cbd149aac0d7f216ec00452ff8c74e0b gcc compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1b6714e850ff280c3ec949638a26588260c78cba7111b6989b6cd7df353a8c77 all runs: crashed: possible deadlock in rds_wake_sk_sleep representative crash: possible deadlock in rds_wake_sk_sleep, types: [LOCKDEP] check whether we can drop unnecessary instrumentation disabling configs for [UBSAN BUG KASAN ATOMIC_SLEEP HANG LEAK], they are not needed testing commit 0ab06468cbd149aac0d7f216ec00452ff8c74e0b gcc compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 78732522cb3e84e485fb11651f156fd9fad0a25d1d2f8ecaa7111fb3b4812f2a all runs: crashed: possible deadlock in rds_wake_sk_sleep representative crash: possible deadlock in rds_wake_sk_sleep, types: [LOCKDEP] the bug reproduces without the instrumentation disabling configs for [LEAK UBSAN BUG KASAN ATOMIC_SLEEP HANG], they are not needed testing current HEAD d54cfc420586425d418a53871290cc4a59d33501 testing commit d54cfc420586425d418a53871290cc4a59d33501 gcc compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a8b8821eabbbd792d1d42c512b6843934b402308de3088fd2682d3942aae4a46 all runs: crashed: possible deadlock in rds_wake_sk_sleep representative crash: possible deadlock in rds_wake_sk_sleep, types: [LOCKDEP] crash still not fixed/happens on the oldest tested release revisions tested: 3, total time: 1h39m36.197863279s (build: 59m10.24493958s, test: 28m57.320170698s) crash still not fixed on HEAD or HEAD had kernel test errors commit msg: Linux 5.15.120 crash: possible deadlock in rds_wake_sk_sleep ====================================================== WARNING: possible circular locking dependency detected 5.15.120-syzkaller #0 Not tainted ------------------------------------------------------ syz-executor.0/5493 is trying to acquire lock: ffff88810f60ebb8 (&rs->rs_recv_lock){....}-{2:2}, at: rds_wake_sk_sleep+0x16/0x60 net/rds/af_rds.c:109 but task is already holding lock: ffff888113c77500 (&rm->m_rs_lock){....}-{2:2}, at: rds_send_remove_from_sock+0xd8/0x340 net/rds/send.c:628 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&rm->m_rs_lock){....}-{2:2}: __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:162 rds_message_purge net/rds/message.c:138 [inline] rds_message_put net/rds/message.c:180 [inline] rds_message_put+0x82/0x3e0 net/rds/message.c:173 rds_clear_recv_queue+0x9b/0xd0 net/rds/recv.c:767 rds_release+0x56/0x140 net/rds/af_rds.c:73 __sock_release+0x32/0xa0 net/socket.c:649 sock_close+0xf/0x20 net/socket.c:1317 __fput+0x95/0x260 fs/file_table.c:280 task_work_run+0x54/0x80 kernel/task_work.c:164 tracehook_notify_resume include/linux/tracehook.h:189 [inline] exit_to_user_mode_loop kernel/entry/common.c:175 [inline] exit_to_user_mode_prepare+0x214/0x220 kernel/entry/common.c:208 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline] syscall_exit_to_user_mode+0x19/0x50 kernel/entry/common.c:301 do_syscall_64+0x42/0xb0 arch/x86/entry/common.c:86 entry_SYSCALL_64_after_hwframe+0x61/0xcb -> #0 (&rs->rs_recv_lock){....}-{2:2}: check_prev_add kernel/locking/lockdep.c:3053 [inline] check_prevs_add kernel/locking/lockdep.c:3172 [inline] validate_chain kernel/locking/lockdep.c:3787 [inline] __lock_acquire+0x1103/0x1a90 kernel/locking/lockdep.c:5011 lock_acquire kernel/locking/lockdep.c:5622 [inline] lock_acquire+0xc9/0x2c0 kernel/locking/lockdep.c:5587 __raw_read_lock_irqsave include/linux/rwlock_api_smp.h:159 [inline] _raw_read_lock_irqsave+0x45/0x90 kernel/locking/spinlock.c:236 rds_wake_sk_sleep+0x16/0x60 net/rds/af_rds.c:109 rds_send_remove_from_sock+0x103/0x340 net/rds/send.c:634 rds_send_path_drop_acked+0x121/0x150 net/rds/send.c:710 rds_tcp_write_space+0xb7/0x240 net/rds/tcp_send.c:198 tcp_new_space net/ipv4/tcp_input.c:5443 [inline] tcp_check_space net/ipv4/tcp_input.c:5462 [inline] tcp_check_space+0x31/0x110 net/ipv4/tcp_input.c:5456 tcp_data_snd_check net/ipv4/tcp_input.c:5471 [inline] tcp_rcv_established+0x27e/0x730 net/ipv4/tcp_input.c:5966 tcp_v4_do_rcv+0x17c/0x2f0 net/ipv4/tcp_ipv4.c:1727 sk_backlog_rcv include/net/sock.h:1059 [inline] __release_sock+0x54/0xd0 net/core/sock.c:2690 release_sock+0x26/0xa0 net/core/sock.c:3231 rds_send_xmit+0x5e7/0xa60 net/rds/send.c:422 rds_sendmsg+0xf10/0x1150 net/rds/send.c:1382 sock_sendmsg_nosec net/socket.c:704 [inline] sock_sendmsg+0x2e/0x40 net/socket.c:724 ____sys_sendmsg+0x234/0x250 net/socket.c:2412 ___sys_sendmsg+0x6e/0xb0 net/socket.c:2466 __sys_sendmsg+0x54/0xa0 net/socket.c:2495 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x61/0xcb other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&rm->m_rs_lock); lock(&rs->rs_recv_lock); lock(&rm->m_rs_lock); lock(&rs->rs_recv_lock); *** DEADLOCK *** 3 locks held by syz-executor.0/5493: #0: ffff88810ec623a0 (k-sk_lock-AF_INET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1661 [inline] #0: ffff88810ec623a0 (k-sk_lock-AF_INET){+.+.}-{0:0}, at: tcp_sock_set_cork+0xe/0x40 net/ipv4/tcp.c:3238 #1: ffff88810ec62658 (k-clock-AF_INET){++.-}-{2:2}, at: rds_tcp_write_space+0x1c/0x240 net/rds/tcp_send.c:184 #2: ffff888113c77500 (&rm->m_rs_lock){....}-{2:2}, at: rds_send_remove_from_sock+0xd8/0x340 net/rds/send.c:628 stack backtrace: CPU: 1 PID: 5493 Comm: syz-executor.0 Not tainted 5.15.120-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/03/2023 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x57/0x7d lib/dump_stack.c:106 check_noncircular+0xcc/0xe0 kernel/locking/lockdep.c:2133 check_prev_add kernel/locking/lockdep.c:3053 [inline] check_prevs_add kernel/locking/lockdep.c:3172 [inline] validate_chain kernel/locking/lockdep.c:3787 [inline] __lock_acquire+0x1103/0x1a90 kernel/locking/lockdep.c:5011 lock_acquire kernel/locking/lockdep.c:5622 [inline] lock_acquire+0xc9/0x2c0 kernel/locking/lockdep.c:5587 __raw_read_lock_irqsave include/linux/rwlock_api_smp.h:159 [inline] _raw_read_lock_irqsave+0x45/0x90 kernel/locking/spinlock.c:236 rds_wake_sk_sleep+0x16/0x60 net/rds/af_rds.c:109 rds_send_remove_from_sock+0x103/0x340 net/rds/send.c:634 rds_send_path_drop_acked+0x121/0x150 net/rds/send.c:710 rds_tcp_write_space+0xb7/0x240 net/rds/tcp_send.c:198 tcp_new_space net/ipv4/tcp_input.c:5443 [inline] tcp_check_space net/ipv4/tcp_input.c:5462 [inline] tcp_check_space+0x31/0x110 net/ipv4/tcp_input.c:5456 tcp_data_snd_check net/ipv4/tcp_input.c:5471 [inline] tcp_rcv_established+0x27e/0x730 net/ipv4/tcp_input.c:5966 tcp_v4_do_rcv+0x17c/0x2f0 net/ipv4/tcp_ipv4.c:1727 sk_backlog_rcv include/net/sock.h:1059 [inline] __release_sock+0x54/0xd0 net/core/sock.c:2690 release_sock+0x26/0xa0 net/core/sock.c:3231 rds_send_xmit+0x5e7/0xa60 net/rds/send.c:422 rds_sendmsg+0xf10/0x1150 net/rds/send.c:1382 sock_sendmsg_nosec net/socket.c:704 [inline] sock_sendmsg+0x2e/0x40 net/socket.c:724 ____sys_sendmsg+0x234/0x250 net/socket.c:2412 ___sys_sendmsg+0x6e/0xb0 net/socket.c:2466 __sys_sendmsg+0x54/0xa0 net/socket.c:2495 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x61/0xcb RIP: 0033:0x7f91b3d57169 Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 f1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f91b30a8168 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 00007f91b3e77050 RCX: 00007f91b3d57169 RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000004 RBP: 00007f91b3db2ca1 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007fff7d59268f R14: 00007f91b30a8300 R15: 0000000000022000