syzbot


possible deadlock in smc_release

Status: upstream: reported C repro on 2024/02/02 13:26
Subsystems: net s390
[Documentation on labels]
Reported-by: syzbot+621fd56ba002faba6392@syzkaller.appspotmail.com
First crash: 80d, last: 25d
Cause bisection: failed (error log, bisect log)
  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [net?] [s390?] possible deadlock in smc_release 0 (1) 2024/02/02 13:26
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in smc_release origin:upstream C 4 9d07h 78d 0/3 upstream: reported C repro on 2024/01/29 16:17
Last patch testing requests (8)
Created Duration User Patch Repo Result
2024/03/22 15:56 19m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/03/22 15:56 20m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/03/22 15:56 18m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/03/18 07:55 24m retest repro net OK log
2024/03/18 07:55 22m retest repro upstream error OK
2024/03/18 07:55 23m retest repro upstream error OK
2024/03/18 07:55 23m retest repro upstream error OK
2024/03/18 07:55 32m retest repro net-next OK log

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.8.0-rc2-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor225/5062 is trying to acquire lock:
ffff8880218893f8 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}, at: __flush_work+0xfa/0xa10 kernel/workqueue.c:3406

but task is already holding lock:
ffff888021888130 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x3a3/0x640 net/smc/af_smc.c:336

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_SMC/1){+.+.}-{0:0}:
       lock_sock_nested+0x3a/0xf0 net/core/sock.c:3524
       smc_listen_out+0x1e7/0x4b0 net/smc/af_smc.c:1914
       smc_listen_out_connected net/smc/af_smc.c:1934 [inline]
       smc_listen_work+0x56e/0x5190 net/smc/af_smc.c:2448
       process_one_work+0x886/0x15d0 kernel/workqueue.c:2633
       process_scheduled_works kernel/workqueue.c:2706 [inline]
       worker_thread+0x8b9/0x1290 kernel/workqueue.c:2787
       kthread+0x2c6/0x3a0 kernel/kthread.c:388
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242

-> #0 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x2445/0x3b30 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719
       __flush_work+0x103/0xa10 kernel/workqueue.c:3406
       __cancel_work_timer+0x3ef/0x590 kernel/workqueue.c:3497
       smc_clcsock_release+0x5f/0xe0 net/smc/smc_close.c:29
       __smc_release+0x5b9/0x890 net/smc/af_smc.c:301
       smc_close_non_accepted+0xda/0x230 net/smc/af_smc.c:1846
       smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
       smc_close_active+0xc2d/0x1070 net/smc/smc_close.c:225
       __smc_release+0x62b/0x890 net/smc/af_smc.c:277
       smc_release+0x209/0x640 net/smc/af_smc.c:344
       __sock_release+0xae/0x260 net/socket.c:659
       sock_close+0x1c/0x20 net/socket.c:1421
       __fput+0x270/0xb70 fs/file_table.c:376
       task_work_run+0x14d/0x240 kernel/task_work.c:180
       exit_task_work include/linux/task_work.h:38 [inline]
       do_exit+0xa8a/0x2ad0 kernel/exit.c:871
       do_group_exit+0xd4/0x2a0 kernel/exit.c:1020
       __do_sys_exit_group kernel/exit.c:1031 [inline]
       __se_sys_exit_group kernel/exit.c:1029 [inline]
       __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:1029
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xd3/0x250 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x63/0x6b

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(sk_lock-AF_SMC/1);
                               lock((work_completion)(&new_smc->smc_listen_work));
                               lock(sk_lock-AF_SMC/1);
  lock((work_completion)(&new_smc->smc_listen_work));

 *** DEADLOCK ***

2 locks held by syz-executor225/5062:
 #0: ffff8880791f6210 (&sb->s_type->i_mutex_key#10){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:802 [inline]
 #0: ffff8880791f6210 (&sb->s_type->i_mutex_key#10){+.+.}-{3:3}, at: __sock_release+0x86/0x260 net/socket.c:658
 #1: ffff888021888130 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x3a3/0x640 net/smc/af_smc.c:336

stack backtrace:
CPU: 1 PID: 5062 Comm: syz-executor225 Not tainted 6.8.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
 check_noncircular+0x317/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x2445/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719
 __flush_work+0x103/0xa10 kernel/workqueue.c:3406
 __cancel_work_timer+0x3ef/0x590 kernel/workqueue.c:3497
 smc_clcsock_release+0x5f/0xe0 net/smc/smc_close.c:29
 __smc_release+0x5b9/0x890 net/smc/af_smc.c:301
 smc_close_non_accepted+0xda/0x230 net/smc/af_smc.c:1846
 smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
 smc_close_active+0xc2d/0x1070 net/smc/smc_close.c:225
 __smc_release+0x62b/0x890 net/smc/af_smc.c:277
 smc_release+0x209/0x640 net/smc/af_smc.c:344
 __sock_release+0xae/0x260 net/socket.c:659
 sock_close+0x1c/0x20 net/socket.c:1421
 __fput+0x270/0xb70 fs/file_table.c:376
 task_work_run+0x14d/0x240 kernel/task_work.c:180
 exit_task_work include/linux/task_work.h:38 [inline]
 do_exit+0xa8a/0x2ad0 kernel/exit.c:871
 do_group_exit+0xd4/0x2a0 kernel/exit.c:1020
 __do_sys_exit_group kernel/exit.c:1031 [inline]
 __se_sys_exit_group kernel/exit.c:1029 [inline]
 __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:1029
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xd3/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7f8804a8bc09
Code: Unable to access opcode bytes at 0x7f8804a8bbdf.
RSP: 002b:00007ffcbc267b78 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f8804a8bc09
RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
RBP: 00007f8804b062d0 R08: ffffffffffffffb8 R09: 0000000000000006
R10: 0000000000000006 R11: 0000000000000246 R12: 00007f8804b062d0
R13: 0000000000000000 R14: 00007f8804b06d20 R15: 00007f8804a5ce60
 </TASK>

Crashes (26):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/01/30 05:21 upstream 41bccc98fb79 991a98f4 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in smc_release
2024/01/30 02:42 upstream 41bccc98fb79 991a98f4 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in smc_release
2024/01/27 04:09 upstream 168174d78157 cc4a4020 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in smc_release
2024/01/30 00:59 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 0802e17d9aca 991a98f4 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in smc_release
2024/01/30 00:38 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 0802e17d9aca 991a98f4 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in smc_release
2024/01/27 03:19 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 0802e17d9aca cc4a4020 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in smc_release
2024/01/27 02:42 net fcf67d82b8b8 cc4a4020 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in smc_release
2024/01/27 03:15 net-next 5f76499fb541 cc4a4020 .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce possible deadlock in smc_release
2024/01/29 16:20 upstream 41bccc98fb79 991a98f4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root possible deadlock in smc_release
2024/01/29 16:12 upstream 41bccc98fb79 991a98f4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root possible deadlock in smc_release
2024/01/28 20:15 upstream 8a696a29c690 cc4a4020 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in smc_release
2024/01/29 16:15 upstream 41bccc98fb79 991a98f4 .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream possible deadlock in smc_release
2024/01/30 17:24 net aa2b2eb39348 7f400fcb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in smc_release
2024/01/28 20:17 net 62b424810535 cc4a4020 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in smc_release
2024/01/27 02:23 net fcf67d82b8b8 cc4a4020 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce possible deadlock in smc_release
2024/03/04 04:16 net-next 4b2765ae410a 25905f5d .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/03/03 14:56 net-next 4b2765ae410a 25905f5d .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/25 10:41 net-next d662c5b3ce6d 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/17 12:09 net-next 71b605d32017 578f7538 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/17 08:54 net-next 71b605d32017 578f7538 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/13 23:29 net-next 86fe596b588f e66542d7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/09 16:48 net-next e7689879d14e 7f07e9b0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/03 02:56 net-next d81c0792e640 60bf9982 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/01/28 20:16 net-next beb53f32698f cc4a4020 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-net-next-test-gce possible deadlock in smc_release
2024/02/09 01:33 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 41bccc98fb79 7f07e9b0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in smc_release
2024/01/28 20:15 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 0802e17d9aca cc4a4020 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 possible deadlock in smc_release
* Struck through repros no longer work on HEAD.