syzbot


possible deadlock in freeze_super (2)

Status: upstream: reported on 2022/10/10 07:37
Reported-by: syzbot+be899d4f10b2a9522dce@syzkaller.appspotmail.com
First crash: 50d, last: 1d14h
similar bugs (1):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in freeze_super 938 605d 736d 0/24 auto-closed as invalid on 2021/07/02 04:50

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.1.0-rc6-syzkaller-32662-g6d464646530f #0 Not tainted
------------------------------------------------------
kworker/1:1H/77 is trying to acquire lock:
ffff0000f7cc50e0 (&type->s_umount_key#92){+.+.}-{3:3}, at: freeze_super+0x40/0x1f0 fs/super.c:1697

but task is already holding lock:
ffff80000fb93d80 ((work_completion)(&(&gl->gl_work)->work)){+.+.}-{0:0}, at: process_one_work+0x29c/0x504 kernel/workqueue.c:2264

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 ((work_completion)(&(&gl->gl_work)->work)){+.+.}-{0:0}:
       process_one_work+0x2c4/0x504 kernel/workqueue.c:2265
       worker_thread+0x340/0x610 kernel/workqueue.c:2436
       kthread+0x12c/0x158 kernel/kthread.c:376
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:863

-> #1 ((wq_completion)glock_workqueue){+.+.}-{0:0}:
       __flush_workqueue+0xb8/0x6dc kernel/workqueue.c:2809
       gfs2_gl_hash_clear+0x4c/0x1b0 fs/gfs2/glock.c:2260
       gfs2_put_super+0x318/0x390 fs/gfs2/super.c:620
       generic_shutdown_super+0x94/0x198 fs/super.c:492
       kill_block_super+0x30/0x78 fs/super.c:1428
       gfs2_kill_sb+0x68/0x78
       deactivate_locked_super+0x70/0xe8 fs/super.c:332
       deactivate_super+0xd0/0xd4 fs/super.c:363
       cleanup_mnt+0x184/0x1c0 fs/namespace.c:1186
       __cleanup_mnt+0x20/0x30 fs/namespace.c:1193
       task_work_run+0x100/0x148 kernel/task_work.c:179
       resume_user_mode_work include/linux/resume_user_mode.h:49 [inline]
       do_notify_resume+0x174/0x1f0 arch/arm64/kernel/signal.c:1127
       prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:137 [inline]
       exit_to_user_mode arch/arm64/kernel/entry-common.c:142 [inline]
       el0_svc+0x9c/0x150 arch/arm64/kernel/entry-common.c:638
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:655
       el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:584

-> #0 (&type->s_umount_key#92){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3097 [inline]
       check_prevs_add kernel/locking/lockdep.c:3216 [inline]
       validate_chain kernel/locking/lockdep.c:3831 [inline]
       __lock_acquire+0x1530/0x3084 kernel/locking/lockdep.c:5055
       lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5668
       down_write+0x5c/0x88 kernel/locking/rwsem.c:1562
       freeze_super+0x40/0x1f0 fs/super.c:1697
       freeze_go_sync+0x84/0x1a8 fs/gfs2/glops.c:573
       do_xmote+0x188/0xa98 fs/gfs2/glock.c:776
       run_queue+0x294/0x3c4 fs/gfs2/glock.c:914
       glock_work_func+0x190/0x288 fs/gfs2/glock.c:1082
       process_one_work+0x2d8/0x504 kernel/workqueue.c:2289
       worker_thread+0x340/0x610 kernel/workqueue.c:2436
       kthread+0x12c/0x158 kernel/kthread.c:376
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:863

other info that might help us debug this:

Chain exists of:
  &type->s_umount_key#92 --> (wq_completion)glock_workqueue --> (work_completion)(&(&gl->gl_work)->work)

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock((work_completion)(&(&gl->gl_work)->work));
                               lock((wq_completion)glock_workqueue);
                               lock((work_completion)(&(&gl->gl_work)->work));
  lock(&type->s_umount_key#92);

 *** DEADLOCK ***

2 locks held by kworker/1:1H/77:
 #0: ffff0000c028cd38 ((wq_completion)glock_workqueue){+.+.}-{0:0}, at: process_one_work+0x270/0x504 kernel/workqueue.c:2262
 #1: ffff80000fb93d80 ((work_completion)(&(&gl->gl_work)->work)){+.+.}-{0:0}, at: process_one_work+0x29c/0x504 kernel/workqueue.c:2264

stack backtrace:
CPU: 1 PID: 77 Comm: kworker/1:1H Not tainted 6.1.0-rc6-syzkaller-32662-g6d464646530f #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/30/2022
Workqueue: glock_workqueue glock_work_func
Call trace:
 dump_backtrace+0x1c4/0x1f0 arch/arm64/kernel/stacktrace.c:156
 show_stack+0x2c/0x54 arch/arm64/kernel/stacktrace.c:163
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x104/0x16c lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x2c4/0x2c8 kernel/locking/lockdep.c:2055
 check_noncircular+0x14c/0x154 kernel/locking/lockdep.c:2177
 check_prev_add kernel/locking/lockdep.c:3097 [inline]
 check_prevs_add kernel/locking/lockdep.c:3216 [inline]
 validate_chain kernel/locking/lockdep.c:3831 [inline]
 __lock_acquire+0x1530/0x3084 kernel/locking/lockdep.c:5055
 lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5668
 down_write+0x5c/0x88 kernel/locking/rwsem.c:1562
 freeze_super+0x40/0x1f0 fs/super.c:1697
 freeze_go_sync+0x84/0x1a8 fs/gfs2/glops.c:573
 do_xmote+0x188/0xa98 fs/gfs2/glock.c:776
 run_queue+0x294/0x3c4 fs/gfs2/glock.c:914
 glock_work_func+0x190/0x288 fs/gfs2/glock.c:1082
 process_one_work+0x2d8/0x504 kernel/workqueue.c:2289
 worker_thread+0x340/0x610 kernel/workqueue.c:2436
 kthread+0x12c/0x158 kernel/kthread.c:376
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:863

Crashes (2):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-gce-arm64 2022/11/27 21:39 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6d464646530f 74a66371 .config log report info possible deadlock in freeze_super
ci-upstream-gce-arm64 2022/10/10 07:24 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci bbed346d5a96 aea5da89 .config log report info possible deadlock in freeze_super
* Struck through repros no longer work on HEAD.