syzbot


INFO: task hung in io_wq_put_and_exit

Status: fixed on 2021/11/10 00:50
Reported-by: syzbot+f62d3e0a4ea4f38f5326@syzkaller.appspotmail.com
Fix commit: 3b33e3f4a6c0 io-wq: fix silly logic error in io_task_work_match()
First crash: 344d, last: 275d

Cause bisection: introduced by (bisect log) [merge commit]:
commit b91db6a0b52e019b6bdabea3f1dbe36d85c7e52c
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date: Tue Aug 31 02:39:59 2021 +0000

  Merge tag 'for-5.15/io_uring-vfs-2021-08-30' of git://git.kernel.dk/linux-block

Crash: KASAN: use-after-free Read in __d_alloc (log)
Repro: C syz .config
similar bugs (2):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in io_wq_put_and_exit (3) C error unreliable 57 79d 140d 0/23 upstream: reported C repro on 2022/03/23 17:52
upstream INFO: task hung in io_wq_put_and_exit (2) 22 244d 273d 0/23 closed as invalid on 2022/02/08 09:40
Patch testing requests:
Created Duration User Patch Repo Result
2021/09/09 02:01 12m axboe@kernel.dk git://git.kernel.dk/linux-block io_uring-5.15 OK
2021/09/09 00:48 18m asml.silence@gmail.com https://github.com/isilence/linux.git syztest_iowq_workers OK

Sample crash report:
INFO: task syz-executor687:8514 blocked for more than 143 seconds.
      Not tainted 5.14.0-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor687 state:D stack:27296 pid: 8514 ppid:  8479 flags:0x00024004
Call Trace:
 context_switch kernel/sched/core.c:4940 [inline]
 __schedule+0x940/0x26f0 kernel/sched/core.c:6287
 schedule+0xd3/0x270 kernel/sched/core.c:6366
 schedule_timeout+0x1db/0x2a0 kernel/time/timer.c:1857
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x176/0x280 kernel/sched/completion.c:138
 io_wq_exit_workers fs/io-wq.c:1162 [inline]
 io_wq_put_and_exit+0x40c/0xc70 fs/io-wq.c:1197
 io_uring_clean_tctx fs/io_uring.c:9607 [inline]
 io_uring_cancel_generic+0x5fe/0x740 fs/io_uring.c:9687
 io_uring_files_cancel include/linux/io_uring.h:16 [inline]
 do_exit+0x265/0x2a30 kernel/exit.c:780
 do_group_exit+0x125/0x310 kernel/exit.c:922
 get_signal+0x47f/0x2160 kernel/signal.c:2868
 arch_do_signal_or_restart+0x2a9/0x1c40 arch/x86/kernel/signal.c:865
 handle_signal_work kernel/entry/common.c:148 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:172 [inline]
 exit_to_user_mode_prepare+0x17d/0x290 kernel/entry/common.c:209
 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
 syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:302
 do_syscall_64+0x42/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x445cd9
RSP: 002b:00007fc657f4b308 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: 0000000000000001 RBX: 00000000004cb448 RCX: 0000000000445cd9
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00000000004cb44c
RBP: 00000000004cb440 R08: 000000000000000e R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000049b154
R13: 0000000000000003 R14: 00007fc657f4b400 R15: 0000000000022000

Showing all locks held in the system:
1 lock held by khungtaskd/1655:
 #0: ffffffff8b97f960 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 1655 Comm: khungtaskd Not tainted 5.14.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1ae/0x220 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xc1d/0xf50 kernel/hung_task.c:295
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 10 Comm: kworker/u4:1 Not tainted 5.14.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:__text_poke+0x5f0/0x8c0 arch/x86/kernel/alternative.c:878
Code: fb 48 8b 7c 24 60 e8 5f 67 0d 08 48 8b 4c 24 18 48 b8 00 00 00 00 00 fc ff df 48 c7 04 01 00 00 00 00 48 8b 84 24 98 00 00 00 <65> 48 2b 04 25 28 00 00 00 0f 85 2e 02 00 00 48 81 c4 a0 00 00 00
RSP: 0018:ffffc90000cf7a00 EFLAGS: 00000286
RAX: cfe52cf07638a100 RBX: 0000000000000007 RCX: 1ffff9200019ef48
RDX: 0000000000000001 RSI: 0000000000000004 RDI: 0000000000000001
RBP: ffff888011eaa558 R08: 0000000000000000 R09: ffff888010db8123
R10: ffffed10021b7024 R11: 000000000000003f R12: ffffffff81bcfd53
R13: 0000000000000001 R14: 0000000000000d54 R15: ffffffff8baa9900
FS:  0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f442218f000 CR3: 000000000b68e000 CR4: 0000000000350ef0
Call Trace:
 text_poke_bp_batch+0x3d7/0x560 arch/x86/kernel/alternative.c:1178
 text_poke_flush arch/x86/kernel/alternative.c:1268 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:1265 [inline]
 text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1275
 arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x1d5/0x430 kernel/jump_label.c:830
 static_key_enable_cpuslocked+0x1b1/0x260 kernel/jump_label.c:177
 static_key_enable+0x16/0x20 kernel/jump_label.c:190
 toggle_allocation_gate mm/kfence/core.c:623 [inline]
 toggle_allocation_gate+0x100/0x390 mm/kfence/core.c:615
 process_one_work+0x9bf/0x16b0 kernel/workqueue.c:2297
 worker_thread+0x658/0x11f0 kernel/workqueue.c:2444
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
----------------
Code disassembly (best guess):
   0:	fb                   	sti
   1:	48 8b 7c 24 60       	mov    0x60(%rsp),%rdi
   6:	e8 5f 67 0d 08       	callq  0x80d676a
   b:	48 8b 4c 24 18       	mov    0x18(%rsp),%rcx
  10:	48 b8 00 00 00 00 00 	movabs $0xdffffc0000000000,%rax
  17:	fc ff df
  1a:	48 c7 04 01 00 00 00 	movq   $0x0,(%rcx,%rax,1)
  21:	00
  22:	48 8b 84 24 98 00 00 	mov    0x98(%rsp),%rax
  29:	00
* 2a:	65 48 2b 04 25 28 00 	sub    %gs:0x28,%rax <-- trapping instruction
  31:	00 00
  33:	0f 85 2e 02 00 00    	jne    0x267
  39:	48 81 c4 a0 00 00 00 	add    $0xa0,%rsp

Crashes (628):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-kasan-gce-root 2021/09/07 17:37 upstream 4b93c544e90e 6ca60148 .config log report syz C INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/07 16:02 upstream 4b93c544e90e 6ca60148 .config log report syz C INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/07 13:26 upstream 4b93c544e90e 6ca60148 .config log report syz C INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/09 02:04 upstream e851dfae4371 8ab17e57 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/09 00:52 upstream e851dfae4371 8ab17e57 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/08 08:55 upstream 6b75d88fa81b 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/07 20:42 upstream b5013d084e03 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/11/07 07:26 upstream 512b7931ad05 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/06 23:47 upstream 512b7931ad05 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/06 09:35 upstream d4439a1189f9 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/11/06 07:13 upstream fe91c4725aee 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/11/05 17:25 upstream d4439a1189f9 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/05 05:14 upstream 7ddb58cb0eca 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/02 18:03 upstream cc0356d6a02e 17f3edd2 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/11/02 05:27 upstream d2fac0afe89f 098b5d53 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-smack-root 2021/11/02 00:43 upstream e66435936756 098b5d53 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/10/27 16:43 upstream d25f27432f80 373bf66b .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/10/19 19:51 upstream d9abdee5fd5a 466b7db1 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/10/14 09:57 upstream 348949d9a444 5462d470 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/10/07 05:24 upstream 5af4055fa813 62ee0987 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/10/06 00:15 upstream f6274b06e326 0a63fd36 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/23 15:40 upstream 58e2cf5d7946 8cac236e .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/23 12:52 upstream 58e2cf5d7946 8cac236e .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/22 18:19 upstream cf1d2c3e7e2f 8cac236e .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/21 14:41 upstream d9fb678414c0 169724fe .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/20 07:21 upstream bc1abb9e55ce 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/19 21:28 upstream bc1abb9e55ce 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/19 12:30 upstream d4d016caa4b8 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-smack-root 2021/09/19 05:13 upstream 93ff9f13be91 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/19 00:45 upstream d4d016caa4b8 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/18 22:53 upstream d4d016caa4b8 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/18 11:25 upstream 4357f03d6611 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/18 10:20 upstream 4357f03d6611 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/17 18:08 upstream ddf21bd8ab98 70b76c1d .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/17 09:43 upstream bdb575f87217 5b989942 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/17 03:25 upstream bdb575f87217 5b989942 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/15 18:48 upstream 3ca706c189db 07e953c1 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/15 06:40 upstream 1619b69edce1 07e953c1 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/14 19:07 upstream 1619b69edce1 07e953c1 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/09/14 13:10 upstream d0ee23f9d78b 58d09404 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/09/11 14:28 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/11 13:21 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-smack-root 2021/09/11 11:01 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/11 09:06 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce 2021/09/11 07:54 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/09/11 07:37 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/11 06:01 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-selinux-root 2021/09/11 04:48 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/09/11 04:42 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-root 2021/09/07 08:28 upstream 4b93c544e90e 6ca60148 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-smack-root 2021/08/31 13:15 upstream b91db6a0b52e 8f58a0ef .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-kasan-gce-386 2021/09/11 16:01 upstream 926de8c4326c 5ae8508a .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-linux-next-kasan-gce-root 2021/11/03 11:29 linux-next 8ccbda2840b7 17f3edd2 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-linux-next-kasan-gce-root 2021/11/03 08:54 linux-next 9150de4aac1e 17f3edd2 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-linux-next-kasan-gce-root 2021/11/01 10:19 linux-next bdcc9f6a5682 098b5d53 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-linux-next-kasan-gce-root 2021/10/13 22:12 linux-next 8006b911c90a 5462d470 .config log report info INFO: task hung in io_wq_put_and_exit
ci-upstream-linux-next-kasan-gce-root 2021/09/25 00:44 linux-next 5a5d008887b4 8cac236e .config log report info INFO: task hung in io_wq_put_and_exit
ci-qemu2-riscv64 2021/11/07 08:57 git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git fixes 64a19591a293 4c1be0be .config log report info INFO: task hung in io_wq_put_and_exit