syzbot


possible deadlock in __sock_release

Status: fixed on 2021/08/30 09:32
Reported-by: syzbot+76d5d0cbd38a47a2db43@syzkaller.appspotmail.com
Fix commit: 7d2c0c0516e6 PCI: Leave Apple Thunderbolt controllers on for s2idle or standby
First crash: 1472d, last: 1209d
Fix bisection: fixed by (bisect log) :
commit 7d2c0c0516e63b71a3476d3b4b6d38406d237486
Author: Konstantin Kharlamov <Hi-Angel@yandex.ru>
Date: Thu May 20 23:55:01 2021 +0000

  PCI: Leave Apple Thunderbolt controllers on for s2idle or standby

  
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in __sock_release tipc C done done 2728 1234d 1542d 0/28 auto-obsoleted due to no activity on 2023/04/15 01:06
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2021/08/29 16:42 4h00m bisect fix linux-4.19.y OK (1) job log
2021/07/30 16:12 30m bisect fix linux-4.19.y OK (0) job log log

Sample crash report:
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
======================================================
WARNING: possible circular locking dependency detected
4.19.156-syzkaller #0 Not tainted
------------------------------------------------------
kworker/1:2/4382 is trying to acquire lock:
000000009041c4f3 (&sb->s_type->i_mutex_key#13){+.+.}, at: inode_lock include/linux/fs.h:748 [inline]
000000009041c4f3 (&sb->s_type->i_mutex_key#13){+.+.}, at: __sock_release+0x86/0x2a0 net/socket.c:578

but task is already holding lock:
00000000caacd63b ((delayed_fput_work).work){+.+.}, at: process_one_work+0x79c/0x1570 kernel/workqueue.c:2130

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #3 ((delayed_fput_work).work){+.+.}:
       worker_thread+0x64c/0x1130 kernel/workqueue.c:2298
       kthread+0x33f/0x460 kernel/kthread.c:259
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

-> #2 ((wq_completion)"events"){+.+.}:
       flush_scheduled_work include/linux/workqueue.h:599 [inline]
       tipc_exit_net+0x38/0x60 net/tipc/core.c:100
       ops_exit_list+0xa5/0x150 net/core/net_namespace.c:153
       cleanup_net+0x3b4/0x8b0 net/core/net_namespace.c:553
       process_one_work+0x864/0x1570 kernel/workqueue.c:2155
       worker_thread+0x64c/0x1130 kernel/workqueue.c:2298
       kthread+0x33f/0x460 kernel/kthread.c:259
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

-> #1 (pernet_ops_rwsem){++++}:
       unregister_netdevice_notifier+0x7b/0x330 net/core/dev.c:1708
       raw_release+0x58/0x820 net/can/raw.c:358
       __sock_release+0xcd/0x2a0 net/socket.c:579
       sock_close+0x15/0x20 net/socket.c:1140
       __fput+0x2ce/0x890 fs/file_table.c:278
       task_work_run+0x148/0x1c0 kernel/task_work.c:113
       tracehook_notify_resume include/linux/tracehook.h:193 [inline]
       exit_to_usermode_loop+0x251/0x2a0 arch/x86/entry/common.c:167
       prepare_exit_to_usermode arch/x86/entry/common.c:198 [inline]
       syscall_return_slowpath arch/x86/entry/common.c:271 [inline]
       do_syscall_64+0x538/0x620 arch/x86/entry/common.c:296
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (&sb->s_type->i_mutex_key#13){+.+.}:
       down_write+0x34/0x90 kernel/locking/rwsem.c:70
       inode_lock include/linux/fs.h:748 [inline]
       __sock_release+0x86/0x2a0 net/socket.c:578
       sock_close+0x15/0x20 net/socket.c:1140
       __fput+0x2ce/0x890 fs/file_table.c:278
       delayed_fput+0x56/0x70 fs/file_table.c:304
       process_one_work+0x864/0x1570 kernel/workqueue.c:2155
       worker_thread+0x64c/0x1130 kernel/workqueue.c:2298
       kthread+0x33f/0x460 kernel/kthread.c:259
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

other info that might help us debug this:

Chain exists of:
  &sb->s_type->i_mutex_key#13 --> (wq_completion)"events" --> (delayed_fput_work).work

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock((delayed_fput_work).work);
                               lock((wq_completion)"events");
                               lock((delayed_fput_work).work);
  lock(&sb->s_type->i_mutex_key#13);

 *** DEADLOCK ***

2 locks held by kworker/1:2/4382:
 #0: 00000000138532ef ((wq_completion)"events"){+.+.}, at: process_one_work+0x767/0x1570 kernel/workqueue.c:2126
 #1: 00000000caacd63b ((delayed_fput_work).work){+.+.}, at: process_one_work+0x79c/0x1570 kernel/workqueue.c:2130

stack backtrace:
CPU: 1 PID: 4382 Comm: kworker/1:2 Not tainted 4.19.156-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events delayed_fput
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
 print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1221
 check_prev_add kernel/locking/lockdep.c:1865 [inline]
 check_prevs_add kernel/locking/lockdep.c:1978 [inline]
 validate_chain kernel/locking/lockdep.c:2419 [inline]
 __lock_acquire+0x30c9/0x3ff0 kernel/locking/lockdep.c:3415
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
 down_write+0x34/0x90 kernel/locking/rwsem.c:70
 inode_lock include/linux/fs.h:748 [inline]
 __sock_release+0x86/0x2a0 net/socket.c:578
 sock_close+0x15/0x20 net/socket.c:1140
 __fput+0x2ce/0x890 fs/file_table.c:278
 delayed_fput+

Crashes (57446):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/11/10 21:18 linux-4.19.y 53fff24aaf01 cca87986 .config console log report syz C ci2-linux-4-19
2021/06/30 12:16 linux-4.19.y eb575cd5d7f6 84fd4c77 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 10:29 linux-4.19.y eb575cd5d7f6 84fd4c77 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 09:38 linux-4.19.y eb575cd5d7f6 84fd4c77 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 08:37 linux-4.19.y eb575cd5d7f6 84fd4c77 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 08:31 linux-4.19.y eb575cd5d7f6 84fd4c77 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 07:23 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 06:15 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 05:49 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 04:49 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 04:33 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 03:26 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 03:23 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 02:07 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 01:04 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/30 00:49 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 23:48 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 23:02 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 21:55 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 21:52 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 20:44 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 20:13 linux-4.19.y eb575cd5d7f6 a4fccb01 .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 19:09 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 18:10 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 17:09 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 16:14 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 15:37 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 14:33 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 13:31 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 12:09 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 10:57 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 09:55 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 08:54 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 07:49 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 06:39 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 05:36 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 05:12 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 04:10 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 04:04 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 03:03 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 02:02 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 01:03 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/29 00:01 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/06/28 23:46 linux-4.19.y eb575cd5d7f6 9d2ab5df .config console log report info ci2-linux-4-19 possible deadlock in __sock_release
2021/01/17 13:30 linux-4.19.y 675cc038067f 813be542 .config console log report info ci2-linux-4-19
2020/11/10 12:28 linux-4.19.y 53fff24aaf01 cca87986 .config console log report info ci2-linux-4-19
* Struck through repros no longer work on HEAD.