syzbot


possible deadlock in io_disarm_next

Status: auto-obsoleted due to no activity on 2022/11/16 18:39
Subsystems: fs io-uring
[Documentation on labels]
Reported-by: syzbot+57e67273f92d7f5f1931@syzkaller.appspotmail.com
First crash: 745d, last: 741d
Cause bisection: introduced by (bisect log) :
commit 78bfbdd1a4977df1dded20f9783a6ec174e67ef8
Author: Pavel Begunkov <asml.silence@gmail.com>
Date: Fri Apr 15 21:08:23 2022 +0000

  io_uring: kill io_put_req_deferred()

Crash: possible deadlock in io_disarm_next (log)
Repro: C syz .config
  
Discussions (2)
Title Replies (including bot) Last reply
[PATCH for-next 0/3] timeout fixes & improvements 5 (5) 2022/04/20 22:24
[syzbot] possible deadlock in io_disarm_next 1 (3) 2022/04/20 13:18
Last patch testing requests (2)
Created Duration User Patch Repo Result
2022/11/16 07:30 22m retest repro linux-next OK log
2022/04/20 12:38 16m asml.silence@gmail.com https://github.com/isilence/linux.git syz_timeout_deadlock OK

Sample crash report:
============================================
WARNING: possible recursive locking detected
5.18.0-rc3-next-20220419-syzkaller #0 Not tainted
--------------------------------------------
syz-executor162/3588 is trying to acquire lock:
ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:379 [inline]
ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: io_disarm_next+0x545/0xaa0 fs/io_uring.c:2452

but task is already holding lock:
ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:379 [inline]
ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: io_kill_timeouts+0x4c/0x227 fs/io_uring.c:10432

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&ctx->timeout_lock);
  lock(&ctx->timeout_lock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

2 locks held by syz-executor162/3588:
 #0: ffff888011a45398 (&ctx->completion_lock){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:354 [inline]
 #0: ffff888011a45398 (&ctx->completion_lock){+.+.}-{2:2}, at: io_kill_timeouts+0x38/0x227 fs/io_uring.c:10431
 #1: ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: spin_lock_irq include/linux/spinlock.h:379 [inline]
 #1: ffff888011a453d8 (&ctx->timeout_lock){....}-{2:2}, at: io_kill_timeouts+0x4c/0x227 fs/io_uring.c:10432

stack backtrace:
CPU: 1 PID: 3588 Comm: syz-executor162 Not tainted 5.18.0-rc3-next-20220419-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2988 [inline]
 check_deadlock kernel/locking/lockdep.c:3031 [inline]
 validate_chain kernel/locking/lockdep.c:3816 [inline]
 __lock_acquire.cold+0x1f5/0x3b4 kernel/locking/lockdep.c:5053
 lock_acquire kernel/locking/lockdep.c:5665 [inline]
 lock_acquire+0x1ab/0x570 kernel/locking/lockdep.c:5630
 __raw_spin_lock_irq include/linux/spinlock_api_smp.h:119 [inline]
 _raw_spin_lock_irq+0x32/0x50 kernel/locking/spinlock.c:170
 spin_lock_irq include/linux/spinlock.h:379 [inline]
 io_disarm_next+0x545/0xaa0 fs/io_uring.c:2452
 __io_req_complete_post+0x794/0xd90 fs/io_uring.c:2200
 io_kill_timeout fs/io_uring.c:1815 [inline]
 io_kill_timeout+0x210/0x21d fs/io_uring.c:1803
 io_kill_timeouts+0xe2/0x227 fs/io_uring.c:10435
 io_ring_ctx_wait_and_kill+0x1eb/0x360 fs/io_uring.c:10462
 io_uring_release+0x42/0x46 fs/io_uring.c:10483
 __fput+0x277/0x9d0 fs/file_table.c:317
 task_work_run+0xdd/0x1a0 kernel/task_work.c:164
 exit_task_work include/linux/task_work.h:37 [inline]
 do_exit+0xaff/0x2a00 kernel/exit.c:796
 do_group_exit+0xd2/0x2f0 kernel/exit.c:926
 __do_sys_exit_group kernel/exit.c:937 [inline]
 __se_sys_exit_group kernel/exit.c:935 [inline]
 __x64_sys_exit_group+0x3a/0x50 kernel/exit.c:935
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f786cb8ccb9
Code: Unable to access opcode bytes at RIP 0x7f786cb8cc8f.
RSP: 002b:00007ffcf6b5b088 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 00007f786cc01350 RCX: 00007f786cb8ccb9
RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffffffffffffffc0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f786cc01350
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
 </TASK>

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/04/19 15:20 linux-next 634de1db0e9b c334415e .config console log report syz C ci-upstream-linux-next-kasan-gce-root possible deadlock in io_disarm_next
2022/04/23 04:09 linux-next f1244c81da13 131df97d .config console log report info ci-upstream-linux-next-kasan-gce-root possible deadlock in io_disarm_next
2022/04/19 12:33 linux-next 634de1db0e9b c334415e .config console log report info ci-upstream-linux-next-kasan-gce-root possible deadlock in io_disarm_next
2022/04/19 10:56 linux-next 634de1db0e9b c334415e .config console log report info ci-upstream-linux-next-kasan-gce-root possible deadlock in io_disarm_next
2022/04/19 09:01 linux-next 634de1db0e9b 8bcc32a6 .config console log report info ci-upstream-linux-next-kasan-gce-root possible deadlock in io_disarm_next
* Struck through repros no longer work on HEAD.