syzbot


INFO: task hung in blk_trace_remove (2)

Status: upstream: reported C repro on 2024/01/25 19:31
Subsystems: block trace
[Documentation on labels]
Reported-by: syzbot+2373f6be3e6de4f92562@syzkaller.appspotmail.com
First crash: 90d, last: 24d
Cause bisection: introduced by (bisect log) :
commit 0d345996e4cb573f8cc81d49b3ee9a7fd2035bef
Author: Pengfei Xu <pengfei.xu@intel.com>
Date: Mon Jul 31 03:04:18 2023 +0000

  x86/kernel: increase kcov coverage under arch/x86/kernel folder

Crash: INFO: task hung in blk_trace_remove (log)
Repro: C syz .config
  
Discussions (3)
Title Replies (including bot) Last reply
[syzbot] Monthly block report (Mar 2024) 0 (1) 2024/03/12 09:29
[PATCH] trace/blktrace: fix task hung in blk_trace_remove 2 (2) 2024/01/31 14:54
[syzbot] [block?] [trace?] INFO: task hung in blk_trace_remove (2) 14 (29) 2024/01/31 12:40
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: task hung in blk_trace_remove 1 763d 763d 0/1 auto-closed as invalid on 2022/07/18 01:50
upstream INFO: task hung in blk_trace_remove block trace 1 1496d 1492d 0/26 auto-closed as invalid on 2020/07/14 22:05
upstream INFO: task can't die in reclaim_throttle mm C error 76 857d 864d 0/26 auto-obsoleted due to no activity on 2022/10/21 07:45
Last patch testing requests (17)
Created Duration User Patch Repo Result
2024/04/20 10:54 27m retest repro linux-next error OK
2024/02/06 16:33 22m retest repro linux-next error OK
2024/02/06 02:38 18m retest repro upstream report log
2024/01/31 11:54 23m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master OK log
2024/01/31 10:15 25m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master OK log
2024/01/31 10:13 24m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master OK log
2024/01/31 09:38 17m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/31 09:32 16m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/31 08:18 17m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/31 01:04 3h09m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/30 13:39 17m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/30 10:53 45m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master OK log
2024/01/30 09:56 16m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/30 08:38 21m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/30 06:48 20m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
2024/01/30 02:16 4h15m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master error OK
2024/01/30 01:01 30m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master report log
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/03/11 18:10 3h54m bisect fix upstream job log (0) log

Sample crash report:
INFO: task syz-executor208:5137 blocked for more than 143 seconds.
      Not tainted 6.7.0-syzkaller-12991-g7a396820222d #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor208 state:D stack:29312 pid:5137  tgid:5134  ppid:5112   flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5400 [inline]
 __schedule+0xf12/0x5c00 kernel/sched/core.c:6727
 __schedule_loop kernel/sched/core.c:6802 [inline]
 schedule+0xe9/0x270 kernel/sched/core.c:6817
 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6874
 __mutex_lock_common kernel/locking/mutex.c:684 [inline]
 __mutex_lock+0x5b9/0x9d0 kernel/locking/mutex.c:752
 blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406
 sg_ioctl_common drivers/scsi/sg.c:1126 [inline]
 sg_ioctl+0x9ac/0x2760 drivers/scsi/sg.c:1160
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:871 [inline]
 __se_sys_ioctl fs/ioctl.c:857 [inline]
 __x64_sys_ioctl+0x18f/0x210 fs/ioctl.c:857
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xd3/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7f61c0ac50d9
RSP: 002b:00007f61c0a60168 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f61c0b4c3d8 RCX: 00007f61c0ac50d9
RDX: 0000000000000000 RSI: 0000000000001276 RDI: 0000000000000003
RBP: 00007f61c0b4c3d0 R08: 00007ffdb9ff9147 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f61c0b4c3dc
R13: 000000000000006e R14: 00007ffdb9ff9060 R15: 00007ffdb9ff9148
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
 #0: ffffffff8d1acba0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:298 [inline]
 #0: ffffffff8d1acba0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:750 [inline]
 #0: ffffffff8d1acba0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6614
2 locks held by getty/4817:
 #0: ffff8880297d30a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
 #1: ffffc90002f062f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc6/0x1490 drivers/tty/n_tty.c:2201
3 locks held by syz-executor208/5136:
1 lock held by syz-executor208/5137:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406
1 lock held by syz-executor208/5140:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_setup+0x33/0x60 kernel/trace/blktrace.c:648
1 lock held by syz-executor208/5141:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406
1 lock held by syz-executor208/5149:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_setup+0x33/0x60 kernel/trace/blktrace.c:648
1 lock held by syz-executor208/5152:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406
1 lock held by syz-executor208/5151:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_setup+0x33/0x60 kernel/trace/blktrace.c:648
1 lock held by syz-executor208/5153:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406
1 lock held by syz-executor208/5155:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_setup+0x33/0x60 kernel/trace/blktrace.c:648
1 lock held by syz-executor208/5156:
 #0: ffff88801e9032b0 (&q->debugfs_mutex){+.+.}-{3:3}, at: blk_trace_remove+0x1f/0x40 kernel/trace/blktrace.c:406

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 29 Comm: khungtaskd Not tainted 6.7.0-syzkaller-12991-g7a396820222d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
 watchdog+0xf87/0x1210 kernel/hung_task.c:379
 kthread+0x2c6/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5136 Comm: syz-executor208 Not tainted 6.7.0-syzkaller-12991-g7a396820222d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:write_comp_data+0x19/0x80 kernel/kcov.c:236
Code: 8b 80 f0 15 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 49 89 d2 49 89 f8 49 89 f1 65 48 8b 14 25 80 c2 03 00 65 8b 05 bf a6 7b 7e <a9> 00 01 ff 00 74 0f f6 c4 01 74 59 8b 82 fc 15 00 00 85 c0 74 4f
RSP: 0018:ffffc900045e77e0 EFLAGS: 00000202
RAX: 0000000080000001 RBX: 0000000000000002 RCX: ffffffff813a4c8d
RDX: ffff88806d973b80 RSI: 0000000000000003 RDI: 0000000000000001
RBP: ffffffff8ac9ccc0 R08: 0000000000000001 R09: 0000000000000003
R10: 0000000000000002 R11: 0000000000000002 R12: 0000000000000002
R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000002
FS:  00007f61c0a606c0(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055e6254f9440 CR3: 000000006e15a000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 __sanitizer_cov_trace_switch+0x54/0x90 kernel/kcov.c:341
 unwind_next_frame+0x77d/0x2390 arch/x86/kernel/unwind_orc.c:581
 __unwind_start+0x5a4/0x880 arch/x86/kernel/unwind_orc.c:760
 unwind_start arch/x86/include/asm/unwind.h:64 [inline]
 arch_stack_walk+0xaf/0x170 arch/x86/kernel/stacktrace.c:24
 stack_trace_save+0x96/0xd0 kernel/stacktrace.c:122
 save_stack+0x160/0x1f0 mm/page_owner.c:129
 __reset_page_owner+0x51/0x2e0 mm/page_owner.c:150
 reset_page_owner include/linux/page_owner.h:24 [inline]
 free_pages_prepare mm/page_alloc.c:1140 [inline]
 free_unref_page_prepare+0x51f/0xb10 mm/page_alloc.c:2346
 free_unref_page+0x33/0x3c0 mm/page_alloc.c:2486
 relay_destroy_buf+0x121/0x3e0 kernel/relay.c:201
 relay_remove_buf kernel/relay.c:221 [inline]
 kref_put include/linux/kref.h:65 [inline]
 relay_close_buf+0x153/0x1b0 kernel/relay.c:430
 relay_close kernel/relay.c:766 [inline]
 relay_close+0x3a8/0x5d0 kernel/relay.c:752
 blk_trace_free+0x37/0x170 kernel/trace/blktrace.c:316
 blk_trace_cleanup kernel/trace/blktrace.c:384 [inline]
 __blk_trace_remove+0x7f/0x130 kernel/trace/blktrace.c:397
 blk_trace_remove+0x27/0x40 kernel/trace/blktrace.c:407
 sg_ioctl_common drivers/scsi/sg.c:1126 [inline]
 sg_ioctl+0x9ac/0x2760 drivers/scsi/sg.c:1160
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:871 [inline]
 __se_sys_ioctl fs/ioctl.c:857 [inline]
 __x64_sys_ioctl+0x18f/0x210 fs/ioctl.c:857
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xd3/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7f61c0ac50d9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 91 1a 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f61c0a60168 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f61c0b4c3d8 RCX: 00007f61c0ac50d9
RDX: 0000000000000000 RSI: 0000000000001276 RDI: 0000000000000003
RBP: 00007f61c0b4c3d0 R08: 00007ffdb9ff9147 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f61c0b4c3dc
R13: 000000000000006e R14: 00007ffdb9ff9060 R15: 00007ffdb9ff9148
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.919 msecs

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/01/21 22:08 upstream 7a396820222d 9bd8dcda .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in blk_trace_remove
2024/01/21 19:21 linux-next ad5c60d66016 9bd8dcda .config strace log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in blk_trace_remove
2024/03/28 09:07 upstream fe46a7dd189e 454571b6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in blk_trace_remove
2024/01/22 17:21 upstream 6613476e225e 9bd8dcda .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in blk_trace_remove
2024/01/21 16:02 linux-next ad5c60d66016 9bd8dcda .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in blk_trace_remove
* Struck through repros no longer work on HEAD.