syzbot


INFO: task hung in xfs_buf_item_unpin

Status: upstream: reported C repro on 2023/02/28 01:30
Labels: xfs (incorrect?)
Reported-by: syzbot+3f083e9e08b726fcfba2@syzkaller.appspotmail.com
First crash: 129d, last: 2d00h

Cause bisection: failed (error log, bisect log)
Discussions (3)
Title Replies (including bot) Last reply
[syzbot] Monthly xfs report (May 2023) 1 (2) 2023/05/05 13:59
[syzbot] Monthly xfs report 3 (4) 2023/04/12 21:54
[syzbot] [xfs?] INFO: task hung in xfs_buf_item_unpin 0 (1) 2023/02/28 01:30
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: task hung in xfs_buf_item_unpin 2 36d 47d 0/3 upstream: reported on 2023/04/12 09:34
linux-6.1 INFO: task hung in xfs_buf_item_unpin 9 44d 77d 0/3 upstream: reported on 2023/03/14 03:47

Sample crash report:
INFO: task kworker/1:1H:52 blocked for more than 143 seconds.
      Not tainted 6.4.0-rc2-syzkaller-00330-g0dd2a6fb1e34 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/1:1H    state:D stack:28080 pid:52    ppid:2      flags:0x00004000
Workqueue: xfs-log/loop0 xlog_ioend_work
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5343 [inline]
 __schedule+0xc9a/0x5880 kernel/sched/core.c:6669
 schedule+0xde/0x1a0 kernel/sched/core.c:6745
 schedule_timeout+0x276/0x2b0 kernel/time/timer.c:2143
 ___down_common kernel/locking/semaphore.c:225 [inline]
 __down_common+0x345/0x680 kernel/locking/semaphore.c:246
 down+0x74/0xa0 kernel/locking/semaphore.c:63
 xfs_buf_lock+0x125/0x490 fs/xfs/xfs_buf.c:1119
 xfs_buf_item_unpin+0x47b/0x6b0 fs/xfs/xfs_buf_item.c:547
 xfs_trans_committed_bulk+0x362/0x7c0 fs/xfs/xfs_trans.c:806
 xlog_cil_committed+0xd09/0x1090 fs/xfs/xfs_log_cil.c:795
 xlog_cil_process_committed+0x123/0x1f0 fs/xfs/xfs_log_cil.c:823
 xlog_state_shutdown_callbacks+0x1e3/0x370 fs/xfs/xfs_log.c:538
 xlog_force_shutdown+0x1e3/0x3f0 fs/xfs/xfs_log.c:3837
 xlog_ioend_work+0xe7/0x110 fs/xfs/xfs_log.c:1429
 process_one_work+0x99a/0x15e0 kernel/workqueue.c:2405
 worker_thread+0x67d/0x10c0 kernel/workqueue.c:2552
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
INFO: task syz-executor200:4998 blocked for more than 143 seconds.
      Not tainted 6.4.0-rc2-syzkaller-00330-g0dd2a6fb1e34 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor200 state:D stack:24808 pid:4998  ppid:4997   flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5343 [inline]
 __schedule+0xc9a/0x5880 kernel/sched/core.c:6669
 schedule+0xde/0x1a0 kernel/sched/core.c:6745
 xlog_wait fs/xfs/xfs_log_priv.h:617 [inline]
 xlog_wait_on_iclog+0x48f/0x6c0 fs/xfs/xfs_log.c:926
 xlog_force_lsn+0x6a6/0x910 fs/xfs/xfs_log.c:3372
 xfs_log_force_seq+0x22b/0x630 fs/xfs/xfs_log.c:3409
 __xfs_trans_commit+0xb93/0xfa0 fs/xfs/xfs_trans.c:1014
 xfs_sync_sb_buf+0x126/0x170 fs/xfs/libxfs/xfs_sb.c:1112
 xfs_ioc_setlabel fs/xfs/xfs_ioctl.c:1801 [inline]
 xfs_file_ioctl+0x15ba/0x1c30 fs/xfs/xfs_ioctl.c:1899
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:870 [inline]
 __se_sys_ioctl fs/ioctl.c:856 [inline]
 __x64_sys_ioctl+0x197/0x210 fs/ioctl.c:856
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f9ea16d92c9
RSP: 002b:00007ffdcbb25438 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9ea16d92c9
RDX: 0000000020000000 RSI: 0000000041009432 RDI: 0000000000000004
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffdcbb25460
R13: 0000000000000000 R14: 431bde82d7b634db R15: 0000000000000000
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/13:
 #0: ffffffff8c798430 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x31/0xd80 kernel/rcu/tasks.h:518
1 lock held by rcu_tasks_trace/14:
 #0: ffffffff8c798130 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x31/0xd80 kernel/rcu/tasks.h:518
1 lock held by khungtaskd/27:
 #0: ffffffff8c799040 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x340 kernel/locking/lockdep.c:6545
2 locks held by kworker/1:1H/52:
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1324 [inline]
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:643 [inline]
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:670 [inline]
 #0: ffff88802b26ad38 ((wq_completion)xfs-log/loop0){+.+.}-{0:0}, at: process_one_work+0x883/0x15e0 kernel/workqueue.c:2376
 #1: ffffc90000bd7db0 ((work_completion)(&iclog->ic_end_io_work)){+.+.}-{0:0}, at: process_one_work+0x8b7/0x15e0 kernel/workqueue.c:2380
2 locks held by getty/4749:
 #0: ffff88814aa99098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x26/0x80 drivers/tty/tty_ldisc.c:243
 #1: ffffc900015a02f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xef4/0x13e0 drivers/tty/n_tty.c:2176
1 lock held by syz-executor200/4998:
 #0: ffff88807d666460 (sb_writers#9){.+.+}-{0:0}, at: xfs_ioc_setlabel fs/xfs/xfs_ioctl.c:1783 [inline]
 #0: ffff88807d666460 (sb_writers#9){.+.+}-{0:0}, at: xfs_file_ioctl+0x1550/0x1c30 fs/xfs/xfs_ioctl.c:1899

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 6.4.0-rc2-syzkaller-00330-g0dd2a6fb1e34 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x29c/0x350 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x2a4/0x300 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
 watchdog+0xe16/0x1090 kernel/hung_task.c:379
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 12 Comm: kworker/u4:1 Not tainted 6.4.0-rc2-syzkaller-00330-g0dd2a6fb1e34 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:jump_entry_key include/linux/jump_label.h:135 [inline]
RIP: 0010:__jump_label_update+0x168/0x410 kernel/jump_label.c:470
Code: 84 66 01 00 00 e8 a8 26 d5 ff 48 3b 6c 24 08 0f 84 3e 01 00 00 e8 98 26 d5 ff 48 8d 53 10 48 89 d0 48 c1 e8 03 42 80 3c 28 00 <0f> 85 01 02 00 00 4c 8b 63 10 48 83 c5 10 4c 89 e0 48 83 e0 fc 48
RSP: 0018:ffffc90000117ba0 EFLAGS: 00000246
RAX: 1ffffffff1835687 RBX: ffffffff8c1ab428 RCX: 0000000000000000
RDX: ffffffff8c1ab438 RSI: ffffffff81af1c38 RDI: 0000000000000001
RBP: ffffffff8c1ab420 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000001
R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005565e5299680 CR3: 000000000c571000 CR4: 0000000000350ee0
Call Trace:
 <TASK>
 jump_label_update+0x32f/0x410 kernel/jump_label.c:829
 static_key_disable_cpuslocked+0x156/0x1b0 kernel/jump_label.c:235
 static_key_disable+0x1a/0x20 kernel/jump_label.c:243
 toggle_allocation_gate mm/kfence/core.c:836 [inline]
 toggle_allocation_gate+0x143/0x230 mm/kfence/core.c:823
 process_one_work+0x99a/0x15e0 kernel/workqueue.c:2405
 worker_thread+0x67d/0x10c0 kernel/workqueue.c:2552
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (124):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Manager Title
2023/05/21 09:38 upstream 0dd2a6fb1e34 4bce1a3e .config strace log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci-upstream-kasan-gce-root INFO: task hung in xfs_buf_item_unpin
2023/02/24 15:58 upstream d2980d8d8265 ee50e71c .config strace log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/02/24 01:14 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci a9b06ec42c0f 9e2ebb3c .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/05/28 07:35 upstream 4e893b5aa4ac cf184559 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/26 23:58 upstream 0d85b27b0cc6 cf184559 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/25 15:18 upstream 933174ae28ba 0513b3e6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/22 04:13 upstream e2065b8c1b01 4bce1a3e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/20 12:23 upstream d635f6cc934b 4bce1a3e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/17 04:46 upstream f1fcbaa18b28 11c89444 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/09 13:43 upstream ba0ad6ed89fd 30aa2a7e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/07 04:20 upstream fc4354c6e5c2 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/06 18:15 upstream 2e1e1337881b 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/05 22:02 upstream 7163a2111f6c de870ca5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/05 12:18 upstream 78b421b6a7c6 518a39a6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/03 07:36 upstream 7df047b3f0aa 48e0a81d .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/02 12:24 upstream 865fdb08197e 52d40fd2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/01 18:08 upstream 58390c8ce1bd 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/01 15:17 upstream 58390c8ce1bd 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/05/01 03:57 upstream 58390c8ce1bd 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/30 23:16 upstream 58390c8ce1bd 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/28 14:42 upstream 91ec4b0d11fe 70a605de .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/28 13:33 upstream 91ec4b0d11fe 70a605de .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/28 10:38 upstream 91ec4b0d11fe 70a605de .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/28 06:42 upstream 91ec4b0d11fe 70a605de .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/28 00:54 upstream 6e98b09da931 6f3d6fa7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/27 22:28 upstream 6e98b09da931 6f3d6fa7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/27 19:05 upstream 6e98b09da931 6f3d6fa7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/27 17:31 upstream 6e98b09da931 6f3d6fa7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/27 08:38 upstream 5c7ecada25d2 19a3dabe .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/23 17:42 upstream 622322f53c6d 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/22 23:51 upstream 2caeeb9d4a1b 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/22 21:12 upstream 2caeeb9d4a1b 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/22 07:10 upstream 8e41e0a57566 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/22 03:13 upstream 8e41e0a57566 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/21 03:11 upstream 6a66fdd29ea1 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/20 19:59 upstream cb0856346a60 a219f34e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/19 13:34 upstream af67688dca57 94b4184e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/18 22:20 upstream af67688dca57 d931e9f0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/17 18:47 upstream 6a8f57ae2eb0 c6ec7083 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/16 06:24 upstream a7a55e27ad72 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/15 13:46 upstream 7a934f4bd7d6 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/04/15 11:49 upstream 7a934f4bd7d6 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/03/28 06:58 upstream 3a93e40326c8 47f3aaf1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in xfs_buf_item_unpin
2023/02/12 14:23 upstream f339c2597ebb 93e26d60 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in xfs_buf_item_unpin
2023/01/30 18:05 linux-next e2f86c02fdc9 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in xfs_buf_item_unpin
2023/05/09 16:40 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 14f8db1c0f9a 30aa2a7e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/05/08 15:39 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 14f8db1c0f9a 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/04/18 05:40 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 327bf9bb94cf 436577a9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/04/18 00:43 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 327bf9bb94cf 436577a9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/04/16 09:22 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 1f5b16c51aef ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/04/16 01:31 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 1f5b16c51aef ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
2023/04/15 21:20 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 1f5b16c51aef ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in xfs_buf_item_unpin
* Struck through repros no longer work on HEAD.