syzbot


INFO: task hung in vhost_net_stop_vq

Status: fixed on 2019/05/27 12:48
Subsystems: kvm net virt
[Documentation on labels]
Reported-by: syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com
Fix commit: 813dbeb656d6 vhost: reject zero size iova range
First crash: 1978d, last: 1835d
Cause bisection: introduced by (bisect log) :
commit 6b1e6cc7855b09a0a9bfa1d9f30172ba366f161c
Author: Jason Wang <jasowang@redhat.com>
Date: Thu Jun 23 06:04:32 2016 +0000

  vhost: new device IOTLB API

Crash: possible deadlock in vhost_process_iotlb_msg (log)
Repro: C syz .config
  
Discussions (6)
Title Replies (including bot) Last reply
[PATCH 5.0 000/115] 5.0.10-stable review 124 (124) 2019/04/27 06:49
[PATCH 4.9 00/44] 4.9.171-stable review 50 (50) 2019/04/25 21:40
[PATCH 4.19 00/96] 4.19.37-stable review 102 (102) 2019/04/25 19:38
[PATCH 4.14 00/70] 4.14.114-stable review 76 (76) 2019/04/25 19:38
[PATCH net] vhost: reject zero size iova range 3 (3) 2019/04/11 05:46
INFO: task hung in vhost_net_stop_vq 4 (6) 2019/04/09 03:31
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: task hung in vhost_net_stop_vq 1 1837d 1837d 0/1 auto-closed as invalid on 2019/10/12 08:44

Sample crash report:
protocol 88fb is buggy, dev hsr_slave_1
protocol 88fb is buggy, dev hsr_slave_0
protocol 88fb is buggy, dev hsr_slave_1
protocol 88fb is buggy, dev hsr_slave_0
protocol 88fb is buggy, dev hsr_slave_1
INFO: task syz-executor895:8515 blocked for more than 140 seconds.
      Not tainted 5.0.0-rc3+ #48
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor895 D19704  8515   8514 0x80000002
Call Trace:
 context_switch kernel/sched/core.c:2831 [inline]
 __schedule+0x897/0x1e60 kernel/sched/core.c:3472
 schedule+0xfe/0x350 kernel/sched/core.c:3516
 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3574
 __mutex_lock_common kernel/locking/mutex.c:1002 [inline]
 __mutex_lock+0xa3b/0x1670 kernel/locking/mutex.c:1072
 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:1087
 vhost_net_stop_vq+0x2d/0x120 drivers/vhost/net.c:1359
 vhost_net_stop drivers/vhost/net.c:1373 [inline]
 vhost_net_release+0x5d/0x260 drivers/vhost/net.c:1405
 __fput+0x3c5/0xb10 fs/file_table.c:278
 ____fput+0x16/0x20 fs/file_table.c:309
 task_work_run+0x1f4/0x2b0 kernel/task_work.c:113
 exit_task_work include/linux/task_work.h:22 [inline]
 do_exit+0xad7/0x26e0 kernel/exit.c:867
 do_group_exit+0x177/0x430 kernel/exit.c:971
 __do_sys_exit_group kernel/exit.c:982 [inline]
 __se_sys_exit_group kernel/exit.c:980 [inline]
 __x64_sys_exit_group+0x44/0x50 kernel/exit.c:980
 do_syscall_64+0x1a3/0x800 arch/x86/entry/common.c:290
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4405d8
Code: 2a 2a 0a 00 00 00 00 00 00 00 53 77 69 74 63 68 69 6e 67 20 64 65 62 75 67 67 69 6e 67 5f 6f 6e 20 74 6f 20 74 72 75 65 20 61 <74> 20 25 32 2e 32 64 3a 25 32 2e 32 64 3a 25 32 2e 32 64 0a 00 00
RSP: 002b:00007ffe805fb1c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00000000004405d8
RDX: 0000000000000001 RSI: 000000000000003c RDI: 0000000000000001
RBP: 00000000004c69f0 R08: 00000000000000e7 R09: ffffffffffffffd0
R10: 0000000001bbbbbb R11: 0000000000000246 R12: 0000000000000001
R13: 00000000006d95e0 R14: 0000000000000000 R15: 0000000000000000

Showing all locks held in the system:
1 lock held by khungtaskd/1040:
 #0: 00000000ca4748c8 (rcu_read_lock){....}, at: debug_show_all_locks+0xc6/0x41d kernel/locking/lockdep.c:4389
1 lock held by rsyslogd/8365:
 #0: 000000004834a5e6 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0x1b3/0x1f0 fs/file.c:795
2 locks held by getty/8487:
 #0: 00000000f0da31e0 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 0000000048e8e5df (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8488:
 #0: 000000007e78e51c (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 0000000059399ca2 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8489:
 #0: 00000000477a8ed2 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000000d031a19 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8490:
 #0: 0000000065c0fc71 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000003bd85ad2 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8491:
 #0: 000000004fae7936 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000510789d7 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8492:
 #0: 0000000012274aab (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000baaf8640 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
2 locks held by getty/8493:
 #0: 000000007002eb22 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000d9a26124 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x30a/0x1eb0 drivers/tty/n_tty.c:2154
1 lock held by syz-executor895/8515:
 #0: 00000000c11bbb81 (&vq->mutex){+.+.}, at: vhost_net_stop_vq+0x2d/0x120 drivers/vhost/net.c:1359
1 lock held by vhost-8515/8522:

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 1040 Comm: khungtaskd Not tainted 5.0.0-rc3+ #48
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1db/0x2d0 lib/dump_stack.c:113
 nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x1be/0x236 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:203 [inline]
 watchdog+0xbbb/0x1170 kernel/hung_task.c:287
 kthread+0x357/0x430 kernel/kthread.c:246
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352
protocol 88fb is buggy, dev hsr_slave_0
protocol 88fb is buggy, dev hsr_slave_1
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 8522 Comm: vhost-8515 Not tainted 5.0.0-rc3+ #48
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:26 [inline]
RIP: 0010:check_kcov_mode kernel/kcov.c:67 [inline]
RIP: 0010:write_comp_data+0x9/0x70 kernel/kcov.c:122
Code: 12 00 00 8b 80 dc 12 00 00 48 8b 11 48 83 c2 01 48 39 d0 76 07 48 89 34 d1 48 89 11 5d c3 0f 1f 00 65 4c 8b 04 25 40 ee 01 00 <65> 8b 05 20 ec 7f 7e a9 00 01 1f 00 75 51 41 8b 80 d8 12 00 00 83
RSP: 0018:ffff88809929fa40 EFLAGS: 00000293
RAX: ffff8880a802a580 RBX: ffff888098dbb100 RCX: ffffffff85e37c68
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000006
RBP: ffff88809929fa48 R08: ffff8880a802a580 R09: ffffed101165c07f
R10: ffff88809929fb18 R11: ffff88808b2e03f3 R12: 0000000000000000
R13: 0000000000000000 R14: ffffffffffffffff R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880ae600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffff600400 CR3: 00000000a5fe2000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 iotlb_access_ok+0x388/0x540 drivers/vhost/vhost.c:1224
 vq_iotlb_prefetch+0x114/0x230 drivers/vhost/vhost.c:1254
 handle_rx+0x251/0x1fa0 drivers/vhost/net.c:1143
 handle_rx_net+0x19/0x20 drivers/vhost/net.c:1285
 vhost_worker+0x2a2/0x4a0 drivers/vhost/vhost.c:362
 kthread+0x357/0x430 kernel/kthread.c:246
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352

Crashes (136):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/01/28 06:10 upstream 983542434e6b c73f090a .config console log report syz C ci-upstream-kasan-gce-selinux-root
2018/11/28 07:09 upstream ef78e5ec9214 4b6d14f2 .config console log report syz C ci-upstream-kasan-gce-selinux-root
2018/11/28 06:14 upstream ef78e5ec9214 4b6d14f2 .config console log report syz C ci-upstream-kasan-gce-smack-root
2018/11/28 06:13 upstream ef78e5ec9214 4b6d14f2 .config console log report syz C ci-upstream-kasan-gce
2019/02/09 15:02 linux-next a46228f6598a fa6c7b70 .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2018/11/28 03:31 linux-next 442b8cea2477 4b6d14f2 .config console log report syz C ci-upstream-linux-next-kasan-gce-root
2019/04/17 11:33 upstream 444fe9913539 b0e8efcb .config console log report ci-upstream-kasan-gce-selinux-root
2019/04/16 18:25 upstream b5de3c5026f5 505ab413 .config console log report ci-upstream-kasan-gce-root
2019/04/16 06:21 upstream 618d919cae2f 505ab413 .config console log report ci-upstream-kasan-gce-root
2019/04/15 11:18 upstream dc4060a5dc25 505ab413 .config console log report ci-upstream-kasan-gce-selinux-root
2019/03/23 14:54 upstream a5ed1e96cafd a2cef203 .config console log report ci-upstream-kasan-gce
2019/03/23 09:54 upstream fd1f297b794c 3361bde5 .config console log report ci-upstream-kasan-gce-root
2019/03/22 10:41 upstream 0939221e6468 dce6e62f .config console log report ci-upstream-kasan-gce
2019/03/21 20:21 upstream 54c490164523 dce6e62f .config console log report ci-upstream-kasan-gce
2019/03/20 09:55 upstream b7a42146dca3 2458c1c6 .config console log report ci-upstream-kasan-gce
2019/03/17 01:44 upstream a9dce6679d73 bab43553 .config console log report ci-upstream-kasan-gce-selinux-root
2019/03/16 22:01 upstream 9c7dc824d9a4 bab43553 .config console log report ci-upstream-kasan-gce-selinux-root
2019/03/16 17:08 upstream 9c7dc824d9a4 bab43553 .config console log report ci-upstream-kasan-gce-root
2019/03/11 13:43 upstream 12ad143e1b80 12365b99 .config console log report ci-upstream-kasan-gce-root
2019/03/03 19:46 upstream c027c7cf1577 1c0e457a .config console log report ci-upstream-kasan-gce-root
2019/03/03 00:01 upstream c93d9218ea56 1c0e457a .config console log report ci-upstream-kasan-gce-root
2019/03/02 17:35 upstream a215ce8f0e00 1c0e457a .config console log report ci-upstream-kasan-gce-smack-root
2019/03/01 05:57 upstream 7d762d69145a 8a4b3a6b .config console log report ci-upstream-kasan-gce-smack-root
2019/02/24 17:18 upstream e60b5f79bd75 7a06e792 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/24 10:29 upstream e60b5f79bd75 7a06e792 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/24 07:58 upstream e60b5f79bd75 7a06e792 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/23 13:37 upstream cb268d806972 18107ce0 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/21 02:05 upstream 2137397c92ae c95f0707 .config console log report ci-upstream-kasan-gce-selinux-root
2019/02/20 18:51 upstream 2137397c92ae c95f0707 .config console log report ci-upstream-kasan-gce-selinux-root
2019/02/20 02:15 upstream 40e196a906d9 4df543c9 .config console log report ci-upstream-kasan-gce-selinux-root
2019/02/19 23:05 upstream b5372fe5dc84 4df543c9 .config console log report ci-upstream-kasan-gce-selinux-root
2019/02/19 16:23 upstream b5372fe5dc84 4df543c9 .config console log report ci-upstream-kasan-gce
2019/02/19 15:08 upstream b5372fe5dc84 4df543c9 .config console log report ci-upstream-kasan-gce
2019/02/19 07:28 upstream b5372fe5dc84 59f36113 .config console log report ci-upstream-kasan-gce
2019/02/19 02:57 upstream a3b22b9f11d9 59f36113 .config console log report ci-upstream-kasan-gce
2019/02/19 01:53 upstream a3b22b9f11d9 59f36113 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/18 21:16 upstream a3b22b9f11d9 59f36113 .config console log report ci-upstream-kasan-gce-root
2019/02/18 05:42 upstream 2fee036af043 59f36113 .config console log report ci-upstream-kasan-gce-smack-root
2019/02/09 09:34 upstream 46c291e277f9 fa6c7b70 .config console log report ci-upstream-kasan-gce-smack-root
2019/03/31 14:15 upstream b5c8314f0eba 0c624d4d .config console log report ci-upstream-kasan-gce-386
2019/03/31 10:50 upstream 922c010cf236 0c624d4d .config console log report ci-upstream-kasan-gce-386
2019/03/31 01:34 upstream 922c010cf236 0c624d4d .config console log report ci-upstream-kasan-gce-386
2019/03/30 23:03 upstream 922c010cf236 0c624d4d .config console log report ci-upstream-kasan-gce-386
2019/03/29 05:51 upstream 8c7ae38d1ce1 14c58f8d .config console log report ci-upstream-kasan-gce-386
2019/03/28 01:04 upstream 65ae689329c5 4e668495 .config console log report ci-upstream-kasan-gce-386
2019/03/27 12:00 upstream 65ae689329c5 55684ce1 .config console log report ci-upstream-kasan-gce-386
2019/03/09 15:00 upstream 38e7571c07be 12365b99 .config console log report ci-upstream-kasan-gce-386
2019/03/09 13:24 upstream 3601fe43e816 12365b99 .config console log report ci-upstream-kasan-gce-386
2019/03/08 11:32 upstream 610cd4eadec4 12365b99 .config console log report ci-upstream-kasan-gce-386
2019/03/06 07:45 upstream 63bdf4284c38 16559f86 .config console log report ci-upstream-kasan-gce-386
2019/03/16 23:34 linux-next cf08baa29613 bab43553 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/03/01 02:17 linux-next 42fd8df9d1d9 09aeeba4 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/02/21 16:16 linux-next 550f4769c7c4 3133098b .config console log report ci-upstream-linux-next-kasan-gce-root
2019/02/19 04:46 linux-next cb916fc5eabf 59f36113 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/02/18 21:21 linux-next cb916fc5eabf 59f36113 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/02/09 11:06 linux-next a46228f6598a fa6c7b70 .config console log report ci-upstream-linux-next-kasan-gce-root
2019/02/09 03:34 linux-next a46228f6598a fa6c7b70 .config console log report ci-upstream-linux-next-kasan-gce-root
2018/11/24 18:00 linux-next 442b8cea2477 ecc7c870 .config console log report ci-upstream-linux-next-kasan-gce-root
* Struck through repros no longer work on HEAD.