syzbot


INFO: task hung in gfs2_gl_hash_clear

Status: upstream: reported C repro on 2023/08/13 12:57
Bug presence: origin:lts-only
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+84d611282adb63889ac8@syzkaller.appspotmail.com
First crash: 260d, last: 1h28m
Fix commit to backport (bisect log) :
tree: upstream
commit 71733b4922007500ae259af9e96017080f5d36d9
Author: Edward Adam Davis <eadavis@qq.com>
Date: Sat Dec 2 09:25:49 2023 +0000

  gfs2: fix kernel BUG in gfs2_quota_cleanup

  
Bug presence (2)
Date Name Commit Repro Result
2024/01/21 linux-6.1.y (ToT) 8fd7f4462453 C [report] INFO: task hung in gfs2_gl_hash_clear
2024/01/21 upstream (ToT) 7a396820222d C Didn't crash
Similar bugs (6)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in gfs2_gl_hash_clear gfs2 156 1272d 1277d 0/26 auto-closed as invalid on 2021/02/02 22:47
upstream INFO: task hung in gfs2_gl_hash_clear (2) gfs2 1 693d 691d 0/26 auto-closed as invalid on 2022/09/04 20:20
linux-5.15 INFO: task hung in gfs2_gl_hash_clear origin:lts-only C inconclusive 457 1h32m 273d 0/3 upstream: reported C repro on 2023/08/01 02:35
linux-4.19 INFO: task hung in gfs2_gl_hash_clear gfs2 1 482d 482d 0/1 upstream: reported on 2023/01/04 01:01
upstream INFO: task can't die in gfs2_gl_hash_clear (2) gfs2 3 1104d 1198d 0/26 auto-closed as invalid on 2021/07/21 03:15
upstream INFO: task hung in gfs2_gl_hash_clear (3) gfs2 C error done 80 129d 517d 26/26 fixed on 2024/01/30 15:47
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/01/23 20:50 4h38m fix candidate upstream job log (1)

Sample crash report:
INFO: task syz-executor237:3593 blocked for more than 143 seconds.
      Not tainted 6.1.74-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor237 state:D stack:23880 pid:3593  ppid:3591   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5245 [inline]
 __schedule+0x142d/0x4550 kernel/sched/core.c:6558
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1935
 gfs2_gl_hash_clear+0x19f/0x310 fs/gfs2/glock.c:2265
 gfs2_put_super+0x823/0x890 fs/gfs2/super.c:625
 generic_shutdown_super+0x130/0x340 fs/super.c:501
 kill_block_super+0x7a/0xe0 fs/super.c:1459
 deactivate_locked_super+0xa0/0x110 fs/super.c:332
 cleanup_mnt+0x490/0x520 fs/namespace.c:1186
 task_work_run+0x246/0x300 kernel/task_work.c:179
 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
 exit_to_user_mode_loop+0xde/0x100 kernel/entry/common.c:171
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:204
 __syscall_exit_to_user_mode_work kernel/entry/common.c:286 [inline]
 syscall_exit_to_user_mode+0x60/0x270 kernel/entry/common.c:297
 do_syscall_64+0x49/0xb0 arch/x86/entry/common.c:87
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f1af2c07387
RSP: 002b:00007ffc7fa98998 EFLAGS: 00000202 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f1af2c07387
RDX: 0000000000000000 RSI: 000000000000000a RDI: 00007ffc7fa98a50
RBP: 00007ffc7fa98a50 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000202 R12: 00007ffc7fa99ab0
R13: 0000555556e206c0 R14: 0000000000000001 R15: 431bde82d7b634db
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8d12a490 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8d12ac90 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by khungtaskd/28:
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:306 [inline]
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
 #0: ffffffff8d12a2c0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x51/0x290 kernel/locking/lockdep.c:6494
2 locks held by getty/3309:
 #0: ffff888028d64098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc900031262f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2188
1 lock held by syz-executor237/3593:
 #0: ffff88807a1be0e0 (&type->s_umount_key#42){+.+.}-{3:3}, at: deactivate_super+0xa9/0xe0 fs/super.c:362

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 28 Comm: khungtaskd Not tainted 6.1.74-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b0/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf88/0xfd0 kernel/hung_task.c:377
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 56 Comm: kworker/u4:4 Not tainted 6.1.74-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:__jump_label_patch+0x227/0x440
Code: 0f 85 31 02 00 00 48 89 d8 48 c1 e8 03 42 80 3c 38 00 74 13 48 89 df 41 89 d6 e8 34 f2 b0 00 44 89 f2 48 8b 74 24 08 4c 8b 33 <4c> 8b 64 24 18 41 83 fc 01 49 c7 c5 80 aa d8 8f 48 c7 c3 80 aa d8
RSP: 0018:ffffc90001577940 EFLAGS: 00000246
RAX: 1ffffffff15ccf29 RBX: ffffffff8ae67948 RCX: 000000008fd8aa04
RDX: 0000000000000005 RSI: ffffffff81e0de7a RDI: 0000000000000000
RBP: ffffc90001577a70 R08: ffffffff8a886038 R09: ffffffff8a88515a
R10: 0000000000000003 R11: ffff88801aaa9dc0 R12: ffffffff81e0de7a
R13: ffffffff8fd8aa81 R14: ffffffff8ae678ca R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000563aaa107680 CR3: 000000000ce8e000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 arch_jump_label_transform_queue+0x4a/0xd0 arch/x86/kernel/jump_label.c:137
 __jump_label_update+0x177/0x3a0 kernel/jump_label.c:447
 static_key_disable_cpuslocked+0xca/0x1b0 kernel/jump_label.c:207
 static_key_disable+0x16/0x20 kernel/jump_label.c:215
 toggle_allocation_gate+0x3e0/0x480 mm/kfence/core.c:818
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.409 msecs

Crashes (631):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/01/21 01:22 linux-6.1.y 8fd7f4462453 9bd8dcda .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/19 01:46 linux-6.1.y 6741e066ec76 af24b050 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/03/28 06:36 linux-6.1.y e5cd595e23c1 120789fd .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/03/28 02:03 linux-6.1.y e5cd595e23c1 120789fd .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/02/24 15:11 linux-6.1.y 81e1dc2f7001 8d446f15 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2023/11/22 07:51 linux-6.1.y 69e434a1cb21 cb976f63 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2023/08/13 12:57 linux-6.1.y 1321ab403b38 39990d51 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/30 06:21 linux-6.1.y dcbc050cb0d3 f10afd69 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 21:35 linux-6.1.y dcbc050cb0d3 f10afd69 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 18:15 linux-6.1.y dcbc050cb0d3 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 17:31 linux-6.1.y dcbc050cb0d3 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 16:11 linux-6.1.y dcbc050cb0d3 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 09:27 linux-6.1.y f2295faba5e8 27e33c58 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 01:32 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/28 08:20 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 12:53 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 09:34 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 07:24 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 07:24 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 04:34 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 03:22 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/27 02:16 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/26 19:38 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/26 16:29 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/26 07:45 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 19:55 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 19:55 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 18:29 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 14:50 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 13:48 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 10:15 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 07:31 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 03:23 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/25 02:17 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/29 04:32 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/29 03:49 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/28 14:51 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/28 12:41 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/27 23:01 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/27 21:14 linux-6.1.y f2295faba5e8 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/27 08:27 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/27 06:22 linux-6.1.y 6741e066ec76 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 22:32 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 21:26 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 17:57 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 16:51 linux-6.1.y 6741e066ec76 059e9963 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 08:02 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 04:53 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/26 03:13 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/25 12:03 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/25 07:33 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/04/25 06:16 linux-6.1.y 6741e066ec76 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
* Struck through repros no longer work on HEAD.