syzbot


INFO: task hung in nbd_add_socket

Status: upstream: reported C repro on 2023/05/21 19:27
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+3e648ac717510d6c66c9@syzkaller.appspotmail.com
First crash: 312d, last: 7d08h
Bug presence (1)
Date Name Commit Repro Result
2023/05/21 upstream (ToT) 4927cb98f0ee C [report] INFO: task hung in blkdev_put
Similar bugs (6)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in nbd_add_socket (2) block C error 35 24d 738d 0/26 upstream: reported C repro on 2022/03/21 16:06
upstream INFO: task hung in nbd_add_socket block 75 781d 860d 0/26 closed as dup on 2021/11/21 13:59
linux-4.14 INFO: task hung in nbd_add_socket (2) C error 2 571d 824d 0/1 upstream: reported C repro on 2021/12/25 23:00
linux-4.14 INFO: task hung in nbd_add_socket 1 1116d 1116d 0/1 auto-closed as invalid on 2021/07/06 19:04
linux-4.19 INFO: task hung in nbd_add_socket C error 11 393d 1116d 0/1 upstream: reported C repro on 2021/03/08 23:07
upstream INFO: task can't die in blk_mq_freeze_queue_wait block 221 857d 865d 20/26 fixed on 2022/03/08 16:11
Fix bisection attempts (8)
Created Duration User Patch Repo Result
2024/03/22 05:35 1h51m bisect fix linux-6.1.y job log (0) log
2024/02/20 17:08 1h31m bisect fix linux-6.1.y job log (0) log
2024/01/10 20:46 1h30m bisect fix linux-6.1.y job log (0) log
2023/12/11 11:22 2h26m bisect fix linux-6.1.y job log (0) log
2023/11/10 14:15 1h42m bisect fix linux-6.1.y job log (0) log
2023/10/07 10:16 1h26m bisect fix linux-6.1.y job log (0) log
2023/08/11 21:43 2h54m bisect fix linux-6.1.y job log (0) log
2023/06/20 22:07 57m bisect fix linux-6.1.y job log (0) log

Sample crash report:
INFO: task syz-executor189:3633 blocked for more than 143 seconds.
      Not tainted 6.1.29-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor189 state:D stack:26936 pid:3633  ppid:3579   flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x132c/0x4330 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 blk_mq_freeze_queue_wait+0xf8/0x180 block/blk-mq.c:180
 nbd_add_socket+0x17c/0x8f0 drivers/block/nbd.c:1121
 __nbd_ioctl drivers/block/nbd.c:1464 [inline]
 nbd_ioctl+0x261/0xe70 drivers/block/nbd.c:1521
 blkdev_ioctl+0x3a9/0x760 block/ioctl.c:615
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:870 [inline]
 __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f4382c5a249
RSP: 002b:00007f4382c07308 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f4382cde408 RCX: 00007f4382c5a249
RDX: 0000000000000004 RSI: 000000000000ab00 RDI: 0000000000000003
RBP: 00007f4382cde400 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f4382cde40c
R13: 00007f4382cab1a4 R14: 64626e2f7665642f R15: 0000000000022000
 </TASK>
INFO: task syz-executor189:3634 blocked for more than 143 seconds.
      Not tainted 6.1.29-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor189 state:D stack:28664 pid:3634  ppid:3579   flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x132c/0x4330 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6689
 __mutex_lock_common+0xe2b/0x2520 kernel/locking/mutex.c:679
 __mutex_lock kernel/locking/mutex.c:747 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
 nbd_ioctl+0x149/0xe70 drivers/block/nbd.c:1514
 blkdev_ioctl+0x3a9/0x760 block/ioctl.c:615
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:870 [inline]
 __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f4382c5a249
RSP: 002b:00007f4382be6308 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f4382cde418 RCX: 00007f4382c5a249
RDX: 0000000000000000 RSI: 000000000000ab03 RDI: 0000000000000003
RBP: 00007f4382cde410 R08: 00007f4382be6700 R09: 0000000000000000
R10: 00007f4382be6700 R11: 0000000000000246 R12: 00007f4382cde41c
R13: 00007f4382cab1a4 R14: 64626e2f7665642f R15: 0000000000022000
 </TASK>
INFO: task syz-executor189:3635 blocked for more than 143 seconds.
      Not tainted 6.1.29-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor189 state:D stack:29096 pid:3635  ppid:3579   flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x132c/0x4330 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6689
 __mutex_lock_common+0xe2b/0x2520 kernel/locking/mutex.c:679
 __mutex_lock kernel/locking/mutex.c:747 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
 nbd_ioctl+0x149/0xe70 drivers/block/nbd.c:1514
 blkdev_ioctl+0x3a9/0x760 block/ioctl.c:615
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:870 [inline]
 __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f4382c5a249
RSP: 002b:00007f4382bc5308 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f4382cde428 RCX: 00007f4382c5a249
RDX: 0000000000010000 RSI: 000000000000ab07 RDI: 0000000000000003
RBP: 00007f4382cde420 R08: 00007f4382bc5700 R09: 0000000000000000
R10: 00007f4382bc5700 R11: 0000000000000246 R12: 00007f4382cde42c
R13: 00007f4382cab1a4 R14: 64626e2f7665642f R15: 0000000000022000
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8cf274b0 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xd20 kernel/rcu/tasks.h:510
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8cf27cb0 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xd20 kernel/rcu/tasks.h:510
1 lock held by khungtaskd/28:
 #0: ffffffff8cf272e0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x30
1 lock held by klogd/2988:
 #0: ffff8880b9939dd8 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x26/0x140 kernel/sched/core.c:537
2 locks held by getty/3308:
 #0: ffff88814b0cd098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc900031262f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2177
1 lock held by syz-executor189/3633:
 #0: ffff888142fe0198 (&nbd->config_lock){+.+.}-{3:3}, at: nbd_ioctl+0x149/0xe70 drivers/block/nbd.c:1514
1 lock held by syz-executor189/3634:
 #0: ffff888142fe0198 (&nbd->config_lock){+.+.}-{3:3}, at: nbd_ioctl+0x149/0xe70 drivers/block/nbd.c:1514
1 lock held by syz-executor189/3635:
 #0: ffff888142fe0198 (&nbd->config_lock){+.+.}-{3:3}, at: nbd_ioctl+0x149/0xe70 drivers/block/nbd.c:1514

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 28 Comm: khungtaskd Not tainted 6.1.29-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b0/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf18/0xf60 kernel/hung_task.c:377
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9 Comm: kworker/u4:0 Not tainted 6.1.29-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/28/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:separate_irq_context kernel/locking/lockdep.c:4571 [inline]
RIP: 0010:__lock_acquire+0xd71/0x1f80 kernel/locking/lockdep.c:5040
Code: e8 48 c1 e8 03 80 3c 38 00 74 12 48 89 ef e8 e6 33 76 00 48 bf 00 00 00 00 00 fc ff df 4c 89 7d 00 48 8b 44 24 28 0f b6 04 38 <84> c0 0f 85 36 0b 00 00 41 8b 1c 24 48 85 db 74 58 83 fb 31 0f 83
RSP: 0018:ffffc900000e76a0 EFLAGS: 00000046
RAX: 0000000000000000 RBX: ffff88813fe38ad0 RCX: ffffffff8169d5bf
RDX: 0000000000000000 RSI: 0000000000000008 RDI: dffffc0000000000
RBP: ffff88813fe38b80 R08: dffffc0000000000 R09: fffffbfff2050845
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff88813fe38ad8
R13: 0000000000000000 R14: ffff88813fe38ba0 R15: cdf6fcfc9f7cb4f8
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055a57581fef0 CR3: 000000000cc8e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5669
 __mutex_lock_common+0x1d4/0x2520 kernel/locking/mutex.c:603
 __mutex_lock kernel/locking/mutex.c:747 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
 arch_jump_label_transform_queue+0x40/0xd0 arch/x86/kernel/jump_label.c:136
 __jump_label_update+0x177/0x3a0 kernel/jump_label.c:447
 static_key_disable_cpuslocked+0xca/0x1b0 kernel/jump_label.c:207
 static_key_disable+0x16/0x20 kernel/jump_label.c:215
 toggle_allocation_gate+0x3e0/0x480 mm/kfence/core.c:818
 process_one_work+0x8aa/0x11f0 kernel/workqueue.c:2289
 worker_thread+0xa5f/0x1210 kernel/workqueue.c:2436
 kthread+0x26e/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.113 msecs

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/05/21 19:27 linux-6.1.y fa74641fb6b9 4bce1a3e .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in nbd_add_socket
2023/08/14 07:42 linux-6.1.y 1321ab403b38 39990d51 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan INFO: task hung in nbd_add_socket
* Struck through repros no longer work on HEAD.