syzbot


INFO: task hung in netfs_unbuffered_read_iter

Status: upstream: reported C repro on 2024/11/27 21:22
Subsystems: netfs
[Documentation on labels]
Reported-by: syzbot+8965fea6a159ab9aa32d@syzkaller.appspotmail.com
First crash: 26d, last: 15h30m
Cause bisection: introduced by (bisect log) :
commit 1bd9011ee163e11f186b72705978fd6b21bdc07b
Author: David Howells <dhowells@redhat.com>
Date: Fri Nov 8 17:32:29 2024 +0000

  netfs: Change the read result collector to only use one work item

Crash: INFO: task hung in netfs_unbuffered_read_iter (log)
Repro: C syz .config
  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [netfs?] INFO: task hung in netfs_unbuffered_read_iter 1 (3) 2024/11/29 17:15
Last patch testing requests (1)
Created Duration User Patch Repo Result
2024/11/29 16:20 42m dhowells@redhat.com patch git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git netfs-writeback OK log

Sample crash report:
INFO: task syz-executor246:5857 blocked for more than 143 seconds.
      Not tainted 6.12.0-next-20241126-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor246 state:D stack:22584 pid:5857  tgid:5857  ppid:5854   flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5369 [inline]
 __schedule+0x1850/0x4c30 kernel/sched/core.c:6756
 __schedule_loop kernel/sched/core.c:6833 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6848
 bit_wait+0x12/0xd0 kernel/sched/wait_bit.c:237
 __wait_on_bit+0xb0/0x2f0 kernel/sched/wait_bit.c:49
 out_of_line_wait_on_bit+0x1d5/0x260 kernel/sched/wait_bit.c:64
 wait_on_bit include/linux/wait_bit.h:77 [inline]
 netfs_dispatch_unbuffered_reads fs/netfs/direct_read.c:107 [inline]
 netfs_unbuffered_read fs/netfs/direct_read.c:146 [inline]
 netfs_unbuffered_read_iter_locked+0xd7e/0x1560 fs/netfs/direct_read.c:231
 netfs_unbuffered_read_iter+0xbf/0xe0 fs/netfs/direct_read.c:266
 new_sync_read fs/read_write.c:484 [inline]
 vfs_read+0x991/0xb70 fs/read_write.c:565
 ksys_read+0x18f/0x2b0 fs/read_write.c:708
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1999cb4f79
RSP: 002b:00007ffef9ec4f98 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
RAX: ffffffffffffffda RBX: 0030656c69662f2e RCX: 00007f1999cb4f79
RDX: 0000000000002020 RSI: 000000002001b640 RDI: 0000000000000006
RBP: 00007f1999cfe04e R08: 0000000000000006 R09: 0000000000000006
R10: 0000000000000006 R11: 0000000000000246 R12: 00007ffef9ec4fac
R13: 431bde82d7b634db R14: 0000000000000001 R15: 0000000000000001
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/30:
 #0: ffffffff8e937ae0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 #0: ffffffff8e937ae0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
 #0: ffffffff8e937ae0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x55/0x2a0 kernel/locking/lockdep.c:6744
5 locks held by kworker/u8:2/35:
2 locks held by getty/5592:
 #0: ffff888034e320a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
 #1: ffffc90002fde2f0 (&ldata->atomic_read_lock){+.+.}-{4:4}, at: n_tty_read+0x6a6/0x1e00 drivers/tty/n_tty.c:2211
1 lock held by syz-executor246/5857:
 #0: ffff888072050148 (&sb->s_type->i_mutex_key#15){++++}-{4:4}, at: netfs_start_io_direct+0x1d4/0x210 fs/netfs/locking.c:188

=============================================

NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 30 Comm: khungtaskd Not tainted 6.12.0-next-20241126-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 nmi_cpu_backtrace+0x49c/0x4d0 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x198/0x320 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:162 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:234 [inline]
 watchdog+0xffb/0x1040 kernel/hung_task.c:397
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 35 Comm: kworker/u8:2 Not tainted 6.12.0-next-20241126-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:insn_get_sib arch/x86/lib/insn.c:447 [inline]
RIP: 0010:insn_get_displacement+0x2ca/0x9a0 arch/x86/lib/insn.c:484
Code: cb f5 49 bc 00 00 00 00 00 fc ff df e9 12 04 00 00 49 8d 5d 25 48 89 d8 48 c1 e8 03 48 b9 00 00 00 00 00 fc ff df 0f b6 04 08 <84> c0 0f 85 95 05 00 00 0f b6 1b 31 ff 89 de e8 72 c9 cb f5 85 db
RSP: 0018:ffffc90000ab7790 EFLAGS: 00000a07
RAX: 0000000000000000 RBX: ffffc90000ab7945 RCX: dffffc0000000000
RDX: ffff888020a81e00 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc90000ab794c R08: ffffffff8bd3c21d R09: ffffffff8bd395b6
R10: 0000000000000002 R11: ffff888020a81e00 R12: 1ffff92000156f29
R13: ffffc90000ab7920 R14: 1ffff92000156f2a R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fbd0e4b4580 CR3: 000000000e736000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 insn_get_immediate+0x62/0x11f0 arch/x86/lib/insn.c:650
 insn_get_length arch/x86/lib/insn.c:723 [inline]
 insn_decode+0x2d6/0x4c0 arch/x86/lib/insn.c:762
 text_poke_loc_init+0xed/0x870 arch/x86/kernel/alternative.c:2432
 arch_jump_label_transform_queue+0x8f/0x100 arch/x86/kernel/jump_label.c:138
 __jump_label_update+0x177/0x3a0 kernel/jump_label.c:513
 static_key_disable_cpuslocked+0xd2/0x1c0 kernel/jump_label.c:240
 static_key_disable+0x1a/0x20 kernel/jump_label.c:248
 toggle_allocation_gate+0x1bf/0x260 mm/kfence/core.c:854
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.344 msecs

Crashes (47):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/26 22:13 linux-next ed9a4ad6e5bd 11dbc254 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/26 18:08 linux-next ed9a4ad6e5bd 11dbc254 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/20 22:24 linux-next 8155b4ef3466 1ac6aa80 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/20 20:39 linux-next 8155b4ef3466 1ac6aa80 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/20 18:18 linux-next 8155b4ef3466 1ac6aa80 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/20 13:00 linux-next 8155b4ef3466 1d58202c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/19 15:21 linux-next 8503810115fb 1d58202c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/18 22:15 linux-next 7fa366f1b6e3 1432fc84 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/18 11:10 linux-next 7fa366f1b6e3 a0626d3a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/18 11:09 linux-next 7fa366f1b6e3 a0626d3a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/18 08:51 linux-next 7fa366f1b6e3 a0626d3a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/10 08:04 linux-next af2ea8ab7a54 cfc402b4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/09 00:12 linux-next af2ea8ab7a54 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/06 20:19 linux-next af2ea8ab7a54 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/06 06:33 linux-next af2ea8ab7a54 946d28f0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/06 03:47 linux-next af2ea8ab7a54 946d28f0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/06 00:37 linux-next af2ea8ab7a54 6e50d07b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/05 21:21 linux-next af2ea8ab7a54 6e50d07b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/05 21:16 linux-next af2ea8ab7a54 6e50d07b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/05 19:54 linux-next af2ea8ab7a54 6e50d07b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/05 06:21 linux-next bcf2acd8f64b 29f61fce .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/05 06:11 linux-next bcf2acd8f64b 29f61fce .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/04 21:30 linux-next bcf2acd8f64b b50eb251 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/04 07:08 linux-next c245a7a79602 b50eb251 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 22:12 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 21:46 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 21:46 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 10:36 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 09:36 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/03 09:24 linux-next f486c8aa16b8 578925bc .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/02 21:12 linux-next f486c8aa16b8 bb326ffb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/02 19:02 linux-next f486c8aa16b8 bb326ffb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/02 05:08 linux-next f486c8aa16b8 68914665 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/12/01 02:14 linux-next f486c8aa16b8 68914665 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/30 05:41 linux-next f486c8aa16b8 68914665 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/29 15:43 linux-next f486c8aa16b8 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/28 00:03 linux-next 6f3d2b5299b0 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 09:16 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 08:49 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 08:49 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 08:48 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 08:48 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 07:43 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 07:41 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
2024/11/25 07:37 linux-next 85a2dd7d7c81 68da6d95 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in netfs_unbuffered_read_iter
* Struck through repros no longer work on HEAD.