syzbot


INFO: task hung in __bch2_fs_stop

Status: upstream: reported C repro on 2024/06/15 09:58
Subsystems: bcachefs
[Documentation on labels]
Reported-by: syzbot+6d3e28b33490b3085412@syzkaller.appspotmail.com
First crash: 177d, last: 6d19h
Cause bisection: introduced by (bisect log) :
commit 267b801fda10b70eca4001a819fcac07f023df6b
Author: Kent Overstreet <kent.overstreet@linux.dev>
Date: Mon Dec 4 18:45:33 2023 +0000

  bcachefs: BCH_IOCTL_FSCK_ONLINE

Crash: INFO: task hung in __bch2_fs_stop (log)
Repro: C syz .config
  
Discussions (6)
Title Replies (including bot) Last reply
[syzbot] Monthly bcachefs report (Nov 2024) 0 (1) 2024/11/15 10:49
[syzbot] Monthly bcachefs report (Oct 2024) 0 (1) 2024/10/14 19:38
[syzbot] Monthly bcachefs report (Aug 2024) 0 (1) 2024/08/13 08:53
[syzbot] Monthly bcachefs report (Jul 2024) 0 (1) 2024/07/12 12:53
[PATCH] bcachefs: Check and reset ro_ref in bch2_fsck_thread_exit 1 (1) 2024/06/17 13:02
[syzbot] [bcachefs?] INFO: task hung in __bch2_fs_stop 0 (1) 2024/06/15 09:58
Last patch testing requests (2)
Created Duration User Patch Repo Result
2024/10/19 01:59 15m retest repro upstream report log
2024/07/29 14:07 22m retest repro upstream OK log

Sample crash report:
INFO: task syz-executor:5325 blocked for more than 143 seconds.
      Not tainted 6.11.0-syzkaller-11653-g3630400697a3 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor    state:D stack:20888 pid:5325  tgid:5325  ppid:1      flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5315 [inline]
 __schedule+0x1843/0x4ae0 kernel/sched/core.c:6675
 __schedule_loop kernel/sched/core.c:6752 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6767
 schedule_preempt_disabled+0x13/0x30 kernel/sched/core.c:6824
 rwsem_down_write_slowpath+0xeee/0x13b0 kernel/locking/rwsem.c:1178
 __down_write_common kernel/locking/rwsem.c:1306 [inline]
 __down_write kernel/locking/rwsem.c:1315 [inline]
 down_write+0x1d7/0x220 kernel/locking/rwsem.c:1580
 __bch2_fs_stop+0xfd/0x540 fs/bcachefs/super.c:618
 generic_shutdown_super+0x139/0x2d0 fs/super.c:642
 bch2_kill_sb+0x41/0x50 fs/bcachefs/fs.c:2179
 deactivate_locked_super+0xc4/0x130 fs/super.c:473
 cleanup_mnt+0x41f/0x4b0 fs/namespace.c:1373
 task_work_run+0x24f/0x310 kernel/task_work.c:228
 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline]
 exit_to_user_mode_loop kernel/entry/common.c:114 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0x168/0x370 kernel/entry/common.c:218
 do_syscall_64+0x100/0x230 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f71cc97f327
RSP: 002b:00007ffe11e1e1a8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00007f71cc97f327
RDX: 0000000000000000 RSI: 0000000000000009 RDI: 00007ffe11e1e260
RBP: 00007ffe11e1e260 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000246 R12: 00007ffe11e1f2e0
R13: 00007f71cc9f0134 R14: 0000000000014cb3 R15: 00007ffe11e1f320
 </TASK>
INFO: task kworker/0:7:5407 blocked for more than 144 seconds.
      Not tainted 6.11.0-syzkaller-11653-g3630400697a3 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/0:7     state:D stack:25104 pid:5407  tgid:5407  ppid:2      flags:0x00004000
Workqueue: events_long bch2_fs_read_only_work
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5315 [inline]
 __schedule+0x1843/0x4ae0 kernel/sched/core.c:6675
 __schedule_loop kernel/sched/core.c:6752 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6767
 schedule_timeout+0xb0/0x310 kernel/time/timer.c:2591
 do_wait_for_common kernel/sched/completion.c:95 [inline]
 __wait_for_common kernel/sched/completion.c:116 [inline]
 wait_for_common kernel/sched/completion.c:127 [inline]
 wait_for_completion+0x355/0x620 kernel/sched/completion.c:148
 kthread_stop+0x19e/0x640 kernel/kthread.c:710
 bch2_copygc_stop+0x4f/0x150 fs/bcachefs/movinggc.c:405
 __bch2_fs_read_only+0x47/0x430 fs/bcachefs/super.c:264
 bch2_fs_read_only+0xb57/0x1200 fs/bcachefs/super.c:354
 bch2_fs_read_only_work+0x2d/0x40 fs/bcachefs/super.c:392
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Showing all locks held in the system:
1 lock held by pool_workqueue_/3:
 #0: ffffffff8e93d338 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:297 [inline]
 #0: ffffffff8e93d338 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x381/0x830 kernel/rcu/tree_exp.h:976
1 lock held by khungtaskd/30:
 #0: ffffffff8e937da0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 #0: ffffffff8e937da0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
 #0: ffffffff8e937da0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x2a0 kernel/locking/lockdep.c:6701
2 locks held by getty/4977:
 #0: ffff888032d420a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x25/0x70 drivers/tty/tty_ldisc.c:243
 #1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a6/0x1e00 drivers/tty/n_tty.c:2211
2 locks held by syz-executor/5325:
 #0: ffff88807bf100e0 (&type->s_umount_key#53){+.+.}-{3:3}, at: __super_lock fs/super.c:56 [inline]
 #0: ffff88807bf100e0 (&type->s_umount_key#53){+.+.}-{3:3}, at: __super_lock_excl fs/super.c:71 [inline]
 #0: ffff88807bf100e0 (&type->s_umount_key#53){+.+.}-{3:3}, at: deactivate_super+0xb5/0xf0 fs/super.c:505
 #1: ffff888060980278 (&c->state_lock){+.+.}-{3:3}, at: __bch2_fs_stop+0xfd/0x540 fs/bcachefs/super.c:618
3 locks held by kworker/0:7/5407:
 #0: ffff88801ac81148 ((wq_completion)events_long){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3204 [inline]
 #0: ffff88801ac81148 ((wq_completion)events_long){+.+.}-{0:0}, at: process_scheduled_works+0x93b/0x1850 kernel/workqueue.c:3310
 #1: ffffc90003dd7d00 ((work_completion)(&c->read_only_work)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3205 [inline]
 #1: ffffc90003dd7d00 ((work_completion)(&c->read_only_work)){+.+.}-{0:0}, at: process_scheduled_works+0x976/0x1850 kernel/workqueue.c:3310
 #2: ffff888060980278 (&c->state_lock){+.+.}-{3:3}, at: bch2_fs_read_only_work+0x25/0x40 fs/bcachefs/super.c:391
1 lock held by syz.4.400/8128:

=============================================

NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 30 Comm: khungtaskd Not tainted 6.11.0-syzkaller-11653-g3630400697a3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
 nmi_cpu_backtrace+0x49c/0x4d0 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x198/0x320 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:162 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:223 [inline]
 watchdog+0xff4/0x1040 kernel/hung_task.c:379
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 35 Comm: kworker/u8:2 Not tainted 6.11.0-syzkaller-11653-g3630400697a3 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: bat_events batadv_nc_worker
RIP: 0010:variable_test_bit arch/x86/include/asm/bitops.h:227 [inline]
RIP: 0010:arch_test_bit arch/x86/include/asm/bitops.h:239 [inline]
RIP: 0010:_test_bit include/asm-generic/bitops/instrumented-non-atomic.h:142 [inline]
RIP: 0010:hlock_class kernel/locking/lockdep.c:228 [inline]
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4848 [inline]
RIP: 0010:__lock_acquire+0x890/0x2050 kernel/locking/lockdep.c:5149
Code: c0 0f 85 4d 01 00 00 41 8b 5d 00 81 e3 ff 1f 00 00 48 89 d8 48 c1 e8 06 48 8d 3c c5 c0 f7 23 94 be 08 00 00 00 e8 b0 21 8b 00 <48> 0f a3 1d 58 d4 b3 12 73 1a 48 69 c3 c8 00 00 00 48 8d 98 c0 76
RSP: 0018:ffffc90000ab78f0 EFLAGS: 00000056
RAX: 0000000000000001 RBX: 0000000000000748 RCX: ffffffff81702360
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff9423f8a8
RBP: 00000000ffffffff R08: ffffffff9423f8af R09: 1ffffffff2847f15
R10: dffffc0000000000 R11: fffffbfff2847f16 R12: 0000000000000000
R13: ffff888020aa0b00 R14: 0000000000000000 R15: ffff888020aa0b00
FS:  0000000000000000(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f18397e6000 CR3: 000000000e734000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5822
 rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 rcu_read_lock include/linux/rcupdate.h:849 [inline]
 batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:408 [inline]
 batadv_nc_worker+0xec/0x610 net/batman-adv/network-coding.c:719
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (1263):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/28 03:01 upstream 3630400697a3 440b26ec .config console log report syz / log [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/06/11 09:46 upstream 83a7eefedc9b b7d9eb04 .config strace log report syz / log C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/14 10:48 upstream 0a9b9d17f3a7 a8c99394 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/12 21:07 upstream 3022e9d00ebe 62026c85 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 19:38 upstream da4373fbcf00 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 15:09 upstream da4373fbcf00 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 11:21 upstream da4373fbcf00 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 08:37 upstream da4373fbcf00 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 03:49 upstream f1dce1f09380 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/09 00:52 upstream f1dce1f09380 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/08 22:37 upstream f1dce1f09380 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/08 20:16 upstream f1dce1f09380 6b856513 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/08 14:04 upstream 906bd684e4b1 179b040e .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/06 17:52 upstream 2e1b3cc9d7f7 df3dc63b .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/05 09:11 upstream 2e1b3cc9d7f7 509da429 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/05 02:28 upstream 557329bcecc2 509da429 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 23:42 upstream 557329bcecc2 509da429 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 17:13 upstream 59b723cd2adb 509da429 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 15:59 upstream 59b723cd2adb f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 14:43 upstream 59b723cd2adb f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 12:16 upstream 59b723cd2adb f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 09:20 upstream 59b723cd2adb f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 05:58 upstream a33ab3f94f51 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/04 01:50 upstream a33ab3f94f51 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/03 22:40 upstream a33ab3f94f51 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/03 04:28 upstream 3e5e6c9900c3 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/02 22:09 upstream 11066801dd4b f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 22:25 upstream 6c52d4da1c74 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 18:42 upstream 6c52d4da1c74 f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 15:49 upstream 6c52d4da1c74 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 13:40 upstream 6c52d4da1c74 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 12:20 upstream 6c52d4da1c74 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 01:07 upstream 0fc810ae3ae1 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/11/01 01:07 upstream 0fc810ae3ae1 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/31 01:33 upstream 4236f913808c 96eb609f .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 16:41 upstream c1e939a21eb1 fb888278 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 13:09 upstream c1e939a21eb1 fb888278 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 10:52 upstream c1e939a21eb1 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 09:02 upstream c1e939a21eb1 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 06:28 upstream c1e939a21eb1 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/30 04:33 upstream c1e939a21eb1 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __bch2_fs_stop
2024/10/30 01:25 upstream e42b1a9a2557 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/29 23:13 upstream e42b1a9a2557 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/29 19:26 upstream e42b1a9a2557 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/29 14:45 upstream e42b1a9a2557 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __bch2_fs_stop
2024/10/29 09:41 upstream e42b1a9a2557 66aeb999 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __bch2_fs_stop
2024/10/24 12:26 upstream c2ee9f594da8 15fa2979 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __bch2_fs_stop
2024/06/26 09:01 upstream 55027e689933 dec8bc94 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in __bch2_fs_stop
2024/06/26 04:21 upstream 55027e689933 da0bd5ca .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in __bch2_fs_stop
2024/05/27 13:32 upstream 1613e604df0c a10a183e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __bch2_fs_stop
2024/09/29 16:17 linux-next 40e0c9d414f5 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in __bch2_fs_stop
2024/10/04 15:32 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 5f5673607153 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in __bch2_fs_stop
* Struck through repros no longer work on HEAD.