syzbot


possible deadlock in smc_release

Status: upstream: reported C repro on 2024/09/24 05:56
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+793d678d565a6f4300f4@syzkaller.appspotmail.com
First crash: 28d, last: 2d13h
Bug presence (1)
Date Name Commit Repro Result
2024/09/24 upstream (ToT) abf2050f51fd C [report] possible deadlock in smc_release
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in smc_release net s390 C error done 27 3d18h 263d 0/28 upstream: reported C repro on 2024/02/02 13:26
linux-6.1 possible deadlock in smc_release origin:upstream missing-backport C 10 2d20h 267d 0/3 upstream: reported C repro on 2024/01/29 16:17

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
5.15.167-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor946/4016 is trying to acquire lock:
ffff0000ca799e90 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}, at: __flush_work+0xd0/0x1c0 kernel/workqueue.c:3090

but task is already holding lock:
ffff0000ca798120 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x1f4/0x574

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (sk_lock-AF_SMC/1){+.+.}-{0:0}:
       lock_sock_nested+0xec/0x1ec net/core/sock.c:3250
       smc_listen_out+0x8c/0x324 net/smc/af_smc.c:1480
       smc_listen_out_connected net/smc/af_smc.c:1500 [inline]
       smc_listen_decline net/smc/af_smc.c:1535 [inline]
       smc_listen_work+0x47c/0xe60 net/smc/af_smc.c:1988
       process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
       worker_thread+0x910/0x1034 kernel/workqueue.c:2457
       kthread+0x37c/0x45c kernel/kthread.c:334
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870

-> #0 ((work_completion)(&new_smc->smc_listen_work)){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3053 [inline]
       check_prevs_add kernel/locking/lockdep.c:3172 [inline]
       validate_chain kernel/locking/lockdep.c:3788 [inline]
       __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
       lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
       __flush_work+0xf8/0x1c0 kernel/workqueue.c:3090
       __cancel_work_timer+0x3ec/0x548 kernel/workqueue.c:3181
       cancel_work_sync+0x24/0x38 kernel/workqueue.c:3217
       smc_clcsock_release+0x64/0xec net/smc/smc_close.c:29
       __smc_release+0x388/0x540 net/smc/af_smc.c:171
       smc_close_non_accepted+0xdc/0x298 net/smc/af_smc.c:1426
       smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
       smc_close_active+0x908/0xc8c net/smc/smc_close.c:219
       __smc_release+0x9c/0x540 net/smc/af_smc.c:147
       smc_release+0x26c/0x574 net/smc/af_smc.c:214
       __sock_release net/socket.c:649 [inline]
       sock_close+0xb8/0x1fc net/socket.c:1336
       __fput+0x1c4/0x800 fs/file_table.c:280
       ____fput+0x20/0x30 fs/file_table.c:308
       task_work_run+0x130/0x1e4 kernel/task_work.c:188
       exit_task_work include/linux/task_work.h:33 [inline]
       do_exit+0x670/0x20bc kernel/exit.c:874
       do_group_exit+0x110/0x268 kernel/exit.c:996
       __do_sys_exit_group kernel/exit.c:1007 [inline]
       __se_sys_exit_group kernel/exit.c:1005 [inline]
       __wake_up_parent+0x0/0x60 kernel/exit.c:1005
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
       el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
       el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
       el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
       el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(sk_lock-AF_SMC/1);
                               lock((work_completion)(&new_smc->smc_listen_work));
                               lock(sk_lock-AF_SMC/1);
  lock((work_completion)(&new_smc->smc_listen_work));

 *** DEADLOCK ***

2 locks held by syz-executor946/4016:
 #0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:789 [inline]
 #0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
 #0: ffff0000e1c8d010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x80/0x1fc net/socket.c:1336
 #1: ffff0000ca798120 (sk_lock-AF_SMC/1){+.+.}-{0:0}, at: smc_release+0x1f4/0x574

stack backtrace:
CPU: 1 PID: 4016 Comm: syz-executor946 Not tainted 5.15.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x150/0x1b8 kernel/locking/lockdep.c:2011
 check_noncircular+0x2cc/0x378 kernel/locking/lockdep.c:2133
 check_prev_add kernel/locking/lockdep.c:3053 [inline]
 check_prevs_add kernel/locking/lockdep.c:3172 [inline]
 validate_chain kernel/locking/lockdep.c:3788 [inline]
 __lock_acquire+0x32d4/0x7638 kernel/locking/lockdep.c:5012
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 __flush_work+0xf8/0x1c0 kernel/workqueue.c:3090
 __cancel_work_timer+0x3ec/0x548 kernel/workqueue.c:3181
 cancel_work_sync+0x24/0x38 kernel/workqueue.c:3217
 smc_clcsock_release+0x64/0xec net/smc/smc_close.c:29
 __smc_release+0x388/0x540 net/smc/af_smc.c:171
 smc_close_non_accepted+0xdc/0x298 net/smc/af_smc.c:1426
 smc_close_cleanup_listen net/smc/smc_close.c:45 [inline]
 smc_close_active+0x908/0xc8c net/smc/smc_close.c:219
 __smc_release+0x9c/0x540 net/smc/af_smc.c:147
 smc_release+0x26c/0x574 net/smc/af_smc.c:214
 __sock_release net/socket.c:649 [inline]
 sock_close+0xb8/0x1fc net/socket.c:1336
 __fput+0x1c4/0x800 fs/file_table.c:280
 ____fput+0x20/0x30 fs/file_table.c:308
 task_work_run+0x130/0x1e4 kernel/task_work.c:188
 exit_task_work include/linux/task_work.h:33 [inline]
 do_exit+0x670/0x20bc kernel/exit.c:874
 do_group_exit+0x110/0x268 kernel/exit.c:996
 __do_sys_exit_group kernel/exit.c:1007 [inline]
 __se_sys_exit_group kernel/exit.c:1005 [inline]
 __wake_up_parent+0x0/0x60 kernel/exit.c:1005
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (11):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/24 05:55 linux-5.15.y 3a5928702e71 89298aad .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in smc_release
2024/10/20 07:04 linux-5.15.y 584a40a22cb9 cd6fc0a3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/10/07 14:16 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/10/07 14:16 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/10/05 04:52 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/09/29 14:30 linux-5.15.y 3a5928702e71 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/09/29 14:28 linux-5.15.y 3a5928702e71 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in smc_release
2024/10/07 01:09 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in smc_release
2024/10/07 01:08 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in smc_release
2024/10/07 01:07 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in smc_release
2024/09/28 08:54 linux-5.15.y 3a5928702e71 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in smc_release
* Struck through repros no longer work on HEAD.