syzbot


possible deadlock in tcp_diag_get_aux

Status: upstream: reported C repro on 2024/03/01 14:44
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+bc70c3c417805c7b8ea4@syzkaller.appspotmail.com
First crash: 62d, last: 58d
Bug presence (2)
Date Name Commit Repro Result
2024/03/02 linux-5.15.y (ToT) 80efc6265290 C [report] possible deadlock in tcp_diag_get_aux
2024/03/02 upstream (ToT) d17468c6f1f4 C Didn't crash
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in tcp_diag_get_aux net C 117 55d 73d 26/26 fixed on 2024/04/02 11:36
linux-6.1 possible deadlock in tcp_diag_get_aux origin:lts-only C inconclusive 4 58d 62d 0/3 upstream: reported C repro on 2024/03/01 15:08
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/03/26 01:45 1m fix candidate upstream error job log (0)

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
5.15.150-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor234/3960 is trying to acquire lock:
ffff0000d2a49aa0 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: tcp_diag_put_ulp net/ipv4/tcp_diag.c:100 [inline]
ffff0000d2a49aa0 (k-sk_lock-AF_INET6){+.+.}-{0:0}, at: tcp_diag_get_aux+0x680/0x750 net/ipv4/tcp_diag.c:137

but task is already holding lock:
ffff0000c5728bc0 (&h->lhash2[i].lock){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff0000c5728bc0 (&h->lhash2[i].lock){+.+.}-{2:2}, at: inet_diag_dump_icsk+0xee4/0x1210 net/ipv4/inet_diag.c:1038

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&h->lhash2[i].lock){+.+.}-{2:2}:
       __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
       _raw_spin_lock+0xb0/0x10c kernel/locking/spinlock.c:154
       spin_lock include/linux/spinlock.h:363 [inline]
       __inet_hash+0xd8/0x754 net/ipv4/inet_hashtables.c:606
       inet6_hash+0x74/0x9c net/ipv6/inet6_hashtables.c:336
       inet_csk_listen_start+0x1e8/0x2cc net/ipv4/inet_connection_sock.c:1084
       inet_listen+0x258/0x6d4 net/ipv4/af_inet.c:231
       rds_tcp_listen_init+0x378/0x504 net/rds/tcp_listen.c:311
       rds_tcp_init_net+0x128/0x2e4 net/rds/tcp.c:559
       ops_init+0x2e8/0x548 net/core/net_namespace.c:135
       __register_pernet_operations net/core/net_namespace.c:1147 [inline]
       register_pernet_operations+0x268/0x700 net/core/net_namespace.c:1216
       register_pernet_device+0x3c/0x9c net/core/net_namespace.c:1303
       rds_tcp_init+0x74/0xe0 net/rds/tcp.c:717
       do_one_initcall+0x234/0x990 init/main.c:1299
       do_initcall_level+0x154/0x214 init/main.c:1372
       do_initcalls+0x58/0xac init/main.c:1388
       do_basic_setup+0x8c/0xa0 init/main.c:1407
       kernel_init_freeable+0x460/0x640 init/main.c:1612
       kernel_init+0x24/0x294 init/main.c:1503
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870

-> #0 (k-sk_lock-AF_INET6){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3053 [inline]
       check_prevs_add kernel/locking/lockdep.c:3172 [inline]
       validate_chain kernel/locking/lockdep.c:3788 [inline]
       __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
       lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
       lock_sock_fast include/net/sock.h:1700 [inline]
       subflow_get_info+0x1e8/0xd10 net/mptcp/diag.c:28
       tcp_diag_put_ulp net/ipv4/tcp_diag.c:100 [inline]
       tcp_diag_get_aux+0x680/0x750 net/ipv4/tcp_diag.c:137
       inet_sk_diag_fill+0xcfc/0x17b4 net/ipv4/inet_diag.c:345
       inet_diag_dump_icsk+0x104c/0x1210 net/ipv4/inet_diag.c:1061
       tcp_diag_dump+0x3c/0x50 net/ipv4/tcp_diag.c:184
       __inet_diag_dump+0x1e8/0x33c net/ipv4/inet_diag.c:1179
       inet_diag_dump+0x4c/0x5c net/ipv4/inet_diag.c:1198
       netlink_dump+0x470/0xa88 net/netlink/af_netlink.c:2279
       __netlink_dump_start+0x488/0x6ec net/netlink/af_netlink.c:2384
       netlink_dump_start include/linux/netlink.h:258 [inline]
       inet_diag_handler_cmd+0x1a8/0x274 net/ipv4/inet_diag.c:1342
       sock_diag_rcv_msg+0x174/0x39c
       netlink_rcv_skb+0x20c/0x3b8 net/netlink/af_netlink.c:2505
       sock_diag_rcv+0x3c/0x54 net/core/sock_diag.c:276
       netlink_unicast_kernel net/netlink/af_netlink.c:1330 [inline]
       netlink_unicast+0x664/0x938 net/netlink/af_netlink.c:1356
       netlink_sendmsg+0x844/0xb38 net/netlink/af_netlink.c:1924
       sock_sendmsg_nosec net/socket.c:704 [inline]
       __sock_sendmsg net/socket.c:716 [inline]
       sock_write_iter+0x2b0/0x3f8 net/socket.c:1079
       do_iter_readv_writev+0x420/0x5f8
       do_iter_write+0x1b8/0x664 fs/read_write.c:855
       vfs_writev fs/read_write.c:928 [inline]
       do_writev+0x220/0x3ec fs/read_write.c:971
       __do_sys_writev fs/read_write.c:1044 [inline]
       __se_sys_writev fs/read_write.c:1041 [inline]
       __arm64_sys_writev+0x80/0x94 fs/read_write.c:1041
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&h->lhash2[i].lock);
                               lock(k-sk_lock-AF_INET6);
                               lock(&h->lhash2[i].lock);
  lock(k-sk_lock-AF_INET6);

 *** DEADLOCK ***

5 locks held by syz-executor234/3960:
 #0: ffff800016a04148 (sock_diag_mutex){+.+.}-{3:3}, at: sock_diag_rcv+0x2c/0x54 net/core/sock_diag.c:275
 #1: ffff800016a03fa8 (sock_diag_table_mutex){+.+.}-{3:3}, at: __sock_diag_cmd net/core/sock_diag.c:229 [inline]
 #1: ffff800016a03fa8 (sock_diag_table_mutex){+.+.}-{3:3}, at: sock_diag_rcv_msg+0x220/0x39c net/core/sock_diag.c:265
 #2: ffff0000d8776690 (nlk_cb_mutex-SOCK_DIAG){+.+.}-{3:3}, at: netlink_dump+0xbc/0xa88 net/netlink/af_netlink.c:2227
 #3: ffff800016add3e8 (inet_diag_table_mutex){+.+.}-{3:3}, at: inet_diag_lock_handler net/ipv4/inet_diag.c:63 [inline]
 #3: ffff800016add3e8 (inet_diag_table_mutex){+.+.}-{3:3}, at: __inet_diag_dump+0x17c/0x33c net/ipv4/inet_diag.c:1177
 #4: ffff0000c5728bc0 (&h->lhash2[i].lock){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #4: ffff0000c5728bc0 (&h->lhash2[i].lock){+.+.}-{2:2}, at: inet_diag_dump_icsk+0xee4/0x1210 net/ipv4/inet_diag.c:1038

stack backtrace:
CPU: 0 PID: 3960 Comm: syz-executor234 Not tainted 5.15.150-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
 check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
 check_prev_add kernel/locking/lockdep.c:3053 [inline]
 check_prevs_add kernel/locking/lockdep.c:3172 [inline]
 validate_chain kernel/locking/lockdep.c:3788 [inline]
 __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 lock_sock_fast include/net/sock.h:1700 [inline]
 subflow_get_info+0x1e8/0xd10 net/mptcp/diag.c:28
 tcp_diag_put_ulp net/ipv4/tcp_diag.c:100 [inline]
 tcp_diag_get_aux+0x680/0x750 net/ipv4/tcp_diag.c:137
 inet_sk_diag_fill+0xcfc/0x17b4 net/ipv4/inet_diag.c:345
 inet_diag_dump_icsk+0x104c/0x1210 net/ipv4/inet_diag.c:1061
 tcp_diag_dump+0x3c/0x50 net/ipv4/tcp_diag.c:184
 __inet_diag_dump+0x1e8/0x33c net/ipv4/inet_diag.c:1179
 inet_diag_dump+0x4c/0x5c net/ipv4/inet_diag.c:1198
 netlink_dump+0x470/0xa88 net/netlink/af_netlink.c:2279
 __netlink_dump_start+0x488/0x6ec net/netlink/af_netlink.c:2384
 netlink_dump_start include/linux/netlink.h:258 [inline]
 inet_diag_handler_cmd+0x1a8/0x274 net/ipv4/inet_diag.c:1342
 sock_diag_rcv_msg+0x174/0x39c
 netlink_rcv_skb+0x20c/0x3b8 net/netlink/af_netlink.c:2505
 sock_diag_rcv+0x3c/0x54 net/core/sock_diag.c:276
 netlink_unicast_kernel net/netlink/af_netlink.c:1330 [inline]
 netlink_unicast+0x664/0x938 net/netlink/af_netlink.c:1356
 netlink_sendmsg+0x844/0xb38 net/netlink/af_netlink.c:1924
 sock_sendmsg_nosec net/socket.c:704 [inline]
 __sock_sendmsg net/socket.c:716 [inline]
 sock_write_iter+0x2b0/0x3f8 net/socket.c:1079
 do_iter_readv_writev+0x420/0x5f8
 do_iter_write+0x1b8/0x664 fs/read_write.c:855
 vfs_writev fs/read_write.c:928 [inline]
 do_writev+0x220/0x3ec fs/read_write.c:971
 __do_sys_writev fs/read_write.c:1044 [inline]
 __se_sys_writev fs/read_write.c:1041 [inline]
 __arm64_sys_writev+0x80/0x94 fs/read_write.c:1041
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584
BUG: sleeping function called from invalid context at net/core/sock.c:3271
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 3960, name: syz-executor234
INFO: lockdep is turned off.
Preemption disabled at:
[<ffff800010781714>] spin_lock include/linux/spinlock.h:363 [inline]
[<ffff800010781714>] inet_diag_dump_icsk+0xee4/0x1210 net/ipv4/inet_diag.c:1038
CPU: 0 PID: 3960 Comm: syz-executor234 Not tainted 5.15.150-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 ___might_sleep+0x380/0x4dc kernel/sched/core.c:9626
 __might_sleep+0x98/0xf0 kernel/sched/core.c:9580
 __lock_sock_fast+0x3c/0xf0 net/core/sock.c:3271
 lock_sock_fast include/net/sock.h:1702 [inline]
 subflow_get_info+0x1f0/0xd10 net/mptcp/diag.c:28
 tcp_diag_put_ulp net/ipv4/tcp_diag.c:100 [inline]
 tcp_diag_get_aux+0x680/0x750 net/ipv4/tcp_diag.c:137
 inet_sk_diag_fill+0xcfc/0x17b4 net/ipv4/inet_diag.c:345
 inet_diag_dump_icsk+0x104c/0x1210 net/ipv4/inet_diag.c:1061
 tcp_diag_dump+0x3c/0x50 net/ipv4/tcp_diag.c:184
 __inet_diag_dump+0x1e8/0x33c net/ipv4/inet_diag.c:1179
 inet_diag_dump+0x4c/0x5c net/ipv4/inet_diag.c:1198
 netlink_dump+0x470/0xa8 net/netlink/af_netlink.c:2279

Crashes (7):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/03/01 22:25 linux-5.15.y 80efc6265290 83acf9e0 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
2024/03/04 15:43 linux-5.15.y 80efc6265290 3717835d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in tcp_diag_get_aux
2024/03/04 18:15 linux-5.15.y 80efc6265290 3717835d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
2024/03/04 13:28 linux-5.15.y 80efc6265290 3717835d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
2024/03/02 06:04 linux-5.15.y 80efc6265290 25905f5d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
2024/03/01 19:04 linux-5.15.y 80efc6265290 83acf9e0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
2024/03/01 14:44 linux-5.15.y 80efc6265290 83acf9e0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in tcp_diag_get_aux
* Struck through repros no longer work on HEAD.