syzbot


INFO: task hung in gfs2_gl_hash_clear

Status: upstream: reported C repro on 2023/08/13 12:57
Bug presence: origin:lts-only
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+84d611282adb63889ac8@syzkaller.appspotmail.com
First crash: 282d, last: 4h14m
Fix commit to backport (bisect log) :
tree: upstream
commit 71733b4922007500ae259af9e96017080f5d36d9
Author: Edward Adam Davis <eadavis@qq.com>
Date: Sat Dec 2 09:25:49 2023 +0000

  gfs2: fix kernel BUG in gfs2_quota_cleanup

  
Bug presence (2)
Date Name Commit Repro Result
2024/01/21 linux-6.1.y (ToT) 8fd7f4462453 C [report] INFO: task hung in gfs2_gl_hash_clear
2024/01/21 upstream (ToT) 7a396820222d C Didn't crash
Similar bugs (6)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in gfs2_gl_hash_clear gfs2 156 1293d 1298d 0/26 auto-closed as invalid on 2021/02/02 22:47
upstream INFO: task hung in gfs2_gl_hash_clear (2) gfs2 1 714d 712d 0/26 auto-closed as invalid on 2022/09/04 20:20
linux-5.15 INFO: task hung in gfs2_gl_hash_clear origin:lts-only C inconclusive 1099 1h30m 294d 0/3 upstream: reported C repro on 2023/08/01 02:35
linux-4.19 INFO: task hung in gfs2_gl_hash_clear gfs2 1 503d 503d 0/1 upstream: reported on 2023/01/04 01:01
upstream INFO: task can't die in gfs2_gl_hash_clear (2) gfs2 3 1125d 1219d 0/26 auto-closed as invalid on 2021/07/21 03:15
upstream INFO: task hung in gfs2_gl_hash_clear (3) gfs2 C error done 80 150d 539d 26/26 fixed on 2024/01/30 15:47
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/01/23 20:50 4h38m fix candidate upstream job log (1)

Sample crash report:
INFO: task syz-executor267:3582 blocked for more than 143 seconds.
      Not tainted 6.1.90-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor267 state:D stack:23880 pid:3582  ppid:3580   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5245 [inline]
 __schedule+0x142d/0x4550 kernel/sched/core.c:6558
 schedule+0xbf/0x180 kernel/sched/core.c:6634
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1965
 gfs2_gl_hash_clear+0x19f/0x310 fs/gfs2/glock.c:2265
 gfs2_put_super+0x823/0x890 fs/gfs2/super.c:625
 generic_shutdown_super+0x130/0x340 fs/super.c:501
 kill_block_super+0x7a/0xe0 fs/super.c:1459
 deactivate_locked_super+0xa0/0x110 fs/super.c:332
 cleanup_mnt+0x490/0x520 fs/namespace.c:1186
 task_work_run+0x246/0x300 kernel/task_work.c:179
 resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
 exit_to_user_mode_loop+0xde/0x100 kernel/entry/common.c:177
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:210
 __syscall_exit_to_user_mode_work kernel/entry/common.c:292 [inline]
 syscall_exit_to_user_mode+0x60/0x270 kernel/entry/common.c:303
 do_syscall_64+0x47/0xb0 arch/x86/entry/common.c:87
 entry_SYSCALL_64_after_hwframe+0x68/0xd2
RIP: 0033:0x7f90c961c407
RSP: 002b:00007ffc1417a5f8 EFLAGS: 00000202 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f90c961c407
RDX: 0000000000000000 RSI: 0000000000000009 RDI: 00007ffc1417a6b0
RBP: 00007ffc1417a6b0 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000202 R12: 00007ffc1417b710
R13: 00005555569fd6c0 R14: 0000000000000001 R15: 431bde82d7b634db
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8d12ae50 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8d12b650 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by khungtaskd/28:
 #0: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #0: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #0: ffffffff8d12ac80 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x51/0x290 kernel/locking/lockdep.c:6494
2 locks held by getty/3303:
 #0: ffff888028bec098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc900031262f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2188
1 lock held by syz-executor267/3582:
 #0: ffff88807bc8a0e0 (&type->s_umount_key#42){++++}-{3:3}, at: deactivate_super+0xa9/0xe0 fs/super.c:362

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 28 Comm: khungtaskd Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b0/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf88/0xfd0 kernel/hung_task.c:377
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 46 Comm: kworker/u4:3 Not tainted 6.1.90-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/02/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:kasan_check_range+0x1ac/0x290 mm/kasan/generic.c:189
Code: 4c 89 d5 48 8d 5d 07 48 85 ed 48 0f 49 dd 48 83 e3 f8 48 29 dd 74 12 41 80 39 00 0f 85 a2 00 00 00 49 ff c1 48 ff cd 75 ee 5b <41> 5e 41 5f 5d c3 45 84 f6 75 61 41 f7 c6 00 ff 00 00 75 5d 41 f7
RSP: 0018:ffffc90000b77840 EFLAGS: 00000056
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffffffff816a8827
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff8e738228
RBP: 0000000000000001 R08: dffffc0000000000 R09: fffffbfff1ce7046
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff9200016ef18
R13: ffffffff8a94bdec R14: fffffbfff1ce7046 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055a9c4152600 CR3: 000000000ce8e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 instrument_atomic_read include/linux/instrumented.h:72 [inline]
 _test_bit include/asm-generic/bitops/instrumented-non-atomic.h:141 [inline]
 cpumask_test_cpu include/linux/cpumask.h:444 [inline]
 cpu_online include/linux/cpumask.h:1030 [inline]
 trace_lock_release include/trace/events/lock.h:69 [inline]
 lock_release+0xc7/0xa20 kernel/locking/lockdep.c:5673
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:149 [inline]
 _raw_spin_unlock_irqrestore+0x75/0x130 kernel/locking/spinlock.c:194
 init_timer_on_stack_key kernel/time/timer.c:805 [inline]
 schedule_timeout+0x11c/0x300 kernel/time/timer.c:1963
 toggle_allocation_gate+0x23b/0x480 mm/kfence/core.c:811
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.204 msecs

Crashes (1241):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/11 05:47 linux-6.1.y 909ba1f1b414 f7c35481 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/01/21 01:22 linux-6.1.y 8fd7f4462453 9bd8dcda .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/04/19 01:46 linux-6.1.y 6741e066ec76 af24b050 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/03/28 06:36 linux-6.1.y e5cd595e23c1 120789fd .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/03/28 02:03 linux-6.1.y e5cd595e23c1 120789fd .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/02/24 15:11 linux-6.1.y 81e1dc2f7001 8d446f15 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2023/11/22 07:51 linux-6.1.y 69e434a1cb21 cb976f63 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2023/08/13 12:57 linux-6.1.y 1321ab403b38 39990d51 .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/21 09:43 linux-6.1.y 4078fa637fcd 4c0d3ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/21 09:38 linux-6.1.y 4078fa637fcd 4c0d3ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/20 15:42 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/20 01:06 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/18 19:44 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/18 18:41 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/15 05:04 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/15 04:37 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/15 01:54 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/14 08:59 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/14 08:59 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/14 07:36 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/14 04:56 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/14 02:31 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/13 16:04 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/13 16:04 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/13 04:48 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/12 19:55 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/11 23:41 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/11 23:41 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in gfs2_gl_hash_clear
2024/05/20 07:47 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/20 03:16 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/19 19:23 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/19 08:28 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/19 08:23 linux-6.1.y 4078fa637fcd c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/17 01:10 linux-6.1.y 909ba1f1b414 c2e07261 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/16 16:47 linux-6.1.y 909ba1f1b414 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/16 04:54 linux-6.1.y 909ba1f1b414 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/16 03:39 linux-6.1.y 909ba1f1b414 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/16 01:42 linux-6.1.y 909ba1f1b414 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/16 01:28 linux-6.1.y 909ba1f1b414 ef5d53ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/15 16:43 linux-6.1.y 909ba1f1b414 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/15 15:36 linux-6.1.y 909ba1f1b414 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/15 15:04 linux-6.1.y 909ba1f1b414 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/15 12:36 linux-6.1.y 909ba1f1b414 94b087b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/15 00:04 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/14 15:20 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/14 12:49 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/14 03:53 linux-6.1.y 909ba1f1b414 fdb4c10c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/13 23:05 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/13 17:20 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/13 14:09 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
2024/05/12 07:47 linux-6.1.y 909ba1f1b414 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 INFO: task hung in gfs2_gl_hash_clear
* Struck through repros no longer work on HEAD.