syzbot


INFO: task hung in io_sq_thread_park

Status: fixed on 2021/04/09 19:46
Subsystems: fs io-uring
[Documentation on labels]
Reported-by: syzbot+fb5458330b4442f2090d@syzkaller.appspotmail.com
Fix commit: 3ebba796fa25 io_uring: ensure that SQPOLL thread is started for exit
First crash: 1128d, last: 1119d
Cause bisection: introduced by (bisect log) [merge commit]:
commit 8075fc3b113dee1531106aaec3dfa19c8158374d
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date: Fri Sep 4 20:04:51 2020 +0000

  Merge tag 'block-5.9-2020-09-04' of git://git.kernel.dk/linux-block

Crash: SYZFAIL: wrong response packet (log)
Repro: C syz .config
  
Discussions (4)
Title Replies (including bot) Last reply
[PATCH 5.11 000/120] 5.11.9-rc1 review 132 (132) 2021/04/02 09:27
[PATCH 5.10 000/157] 5.10.26-rc1 review 167 (167) 2021/03/24 14:28
[PATCHSET 0/33] Fixes queued up for 5.12 36 (36) 2021/03/04 14:05
INFO: task hung in io_sq_thread_park 1 (2) 2021/02/28 22:20
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in io_sq_thread_park (3) fs 1 847d 847d 0/26 auto-closed as invalid on 2022/04/01 18:56
upstream INFO: task hung in io_sq_thread_park (2) fs io-uring syz done 2 992d 988d 20/26 fixed on 2021/11/10 00:50

Sample crash report:
INFO: task syz-executor458:8401 blocked for more than 143 seconds.
      Not tainted 5.11.0-next-20210226-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor458 state:D stack:27536 pid: 8401 ppid:  8400 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4324 [inline]
 __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
 schedule+0xcf/0x270 kernel/sched/core.c:5154
 schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
 io_sq_thread_park fs/io_uring.c:7115 [inline]
 io_sq_thread_park+0xd5/0x130 fs/io_uring.c:7103
 io_uring_cancel_task_requests+0x24c/0xd90 fs/io_uring.c:8745
 __io_uring_files_cancel+0x110/0x230 fs/io_uring.c:8840
 io_uring_files_cancel include/linux/io_uring.h:47 [inline]
 do_exit+0x299/0x2a60 kernel/exit.c:780
 do_group_exit+0x125/0x310 kernel/exit.c:922
 __do_sys_exit_group kernel/exit.c:933 [inline]
 __se_sys_exit_group kernel/exit.c:931 [inline]
 __x64_sys_exit_group+0x3a/0x50 kernel/exit.c:931
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x43e899
RSP: 002b:00007ffe89376d48 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 00000000004af2f0 RCX: 000000000043e899
RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffffffffffffffc0 R09: 0000000010000000
R10: 0000000000008011 R11: 0000000000000246 R12: 00000000004af2f0
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
INFO: task iou-sqp-8401:8402 can't die for more than 143 seconds.
task:iou-sqp-8401    state:D stack:30272 pid: 8402 ppid:  8400 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4324 [inline]
 __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
 schedule+0xcf/0x270 kernel/sched/core.c:5154
 schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
 io_sq_thread+0x27d/0x1ae0 fs/io_uring.c:6717
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
INFO: task iou-sqp-8401:8402 blocked for more than 143 seconds.
      Not tainted 5.11.0-next-20210226-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:iou-sqp-8401    state:D stack:30272 pid: 8402 ppid:  8400 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4324 [inline]
 __schedule+0x90c/0x21a0 kernel/sched/core.c:5075
 schedule+0xcf/0x270 kernel/sched/core.c:5154
 schedule_timeout+0x1db/0x250 kernel/time/timer.c:1868
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x168/0x270 kernel/sched/completion.c:138
 io_sq_thread+0x27d/0x1ae0 fs/io_uring.c:6717
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294

Showing all locks held in the system:
1 lock held by khungtaskd/1666:
 #0: ffffffff8bf741e0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6327
1 lock held by syz-executor458/8401:
 #0: ffff88801cafe870 (&sqd->lock){+.+.}-{3:3}, at: io_sq_thread_park+0x5d/0x130 fs/io_uring.c:7108

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1666 Comm: khungtaskd Not tainted 5.11.0-next-20210226-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0xfa/0x151 lib/dump_stack.c:120
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:147 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:253 [inline]
 watchdog+0xd8e/0xf40 kernel/hung_task.c:338
 kthread+0x3b1/0x4a0 kernel/kthread.c:292
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1 skipped: idling at native_safe_halt arch/x86/include/asm/irqflags.h:51 [inline]
NMI backtrace for cpu 1 skipped: idling at arch_safe_halt arch/x86/include/asm/irqflags.h:89 [inline]
NMI backtrace for cpu 1 skipped: idling at acpi_safe_halt drivers/acpi/processor_idle.c:110 [inline]
NMI backtrace for cpu 1 skipped: idling at acpi_idle_do_entry+0x1c9/0x250 drivers/acpi/processor_idle.c:516

Crashes (263):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/02/27 18:18 linux-next d01f2f7e3557 4c37c133 .config console log report syz C ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/02/26 22:21 linux-next d01f2f7e3557 4c37c133 .config console log report syz C ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/05 18:46 upstream 280d542f6ffa 9d751681 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/05 16:45 upstream 280d542f6ffa 9d751681 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/05 11:42 upstream cee407c5cc42 9d751681 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/05 09:18 upstream cee407c5cc42 9d751681 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/05 08:02 upstream cee407c5cc42 9d751681 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/05 06:58 upstream cee407c5cc42 9d751681 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/05 02:55 upstream cee407c5cc42 9d751681 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 18:57 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/04 17:02 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/04 16:45 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 15:42 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/04 14:14 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 13:14 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 12:13 upstream f69d02e37a85 d7e4e604 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 04:12 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/04 02:39 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/04 01:38 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/04 00:46 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/03 23:35 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/03 21:49 upstream f69d02e37a85 06ed56cd .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/03 09:25 upstream 7a7fd0de4a98 e5b64d68 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/03 07:09 upstream 7a7fd0de4a98 e5b64d68 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/03 04:43 upstream 7a7fd0de4a98 e5b64d68 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/02 21:33 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 20:07 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/02 19:09 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 17:11 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 16:07 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 15:05 upstream 7a7fd0de4a98 92ead296 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/02 07:01 upstream fe07bfda2fb9 183afb6c .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 04:01 upstream fe07bfda2fb9 183afb6c .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 03:34 upstream 7a7fd0de4a98 183afb6c .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/02 02:01 upstream 7a7fd0de4a98 183afb6c .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 23:41 upstream fe07bfda2fb9 183afb6c .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/01 13:23 upstream cd278456d4ca 4c37c133 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/01 10:51 upstream cd278456d4ca 4c37c133 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 09:59 upstream cd278456d4ca 4c37c133 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/01 08:31 upstream cd278456d4ca 4c37c133 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 06:11 upstream cd278456d4ca 4c37c133 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in io_sq_thread_park
2021/03/02 00:43 linux-next 26ada2d1f143 183afb6c .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 22:37 linux-next 26ada2d1f143 183afb6c .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 22:24 linux-next 26ada2d1f143 183afb6c .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 17:55 linux-next 26ada2d1f143 4c37c133 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 15:21 linux-next 26ada2d1f143 4c37c133 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 12:19 linux-next 26ada2d1f143 4c37c133 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/03/01 07:13 linux-next 26ada2d1f143 4c37c133 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/02/24 20:42 linux-next 577c283560fe fcc6d71b .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in io_sq_thread_park
2021/02/26 17:15 linux-next d01f2f7e3557 4c37c133 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task can't die in io_sq_thread_park
* Struck through repros no longer work on HEAD.