bisecting fixing commit since 5692097116094a4a7045abcc1dbc172dbdc5657e
building syzkaller on d850e9d08467726cd94b6a9caaf8090c594721ba
testing commit 5692097116094a4a7045abcc1dbc172dbdc5657e with gcc (GCC) 8.4.1 20210217
kernel signature: 3f133f5906d1b104b1ce5ccff50ae3137e42dcb4baabebfb2b95c4ff531c2387
run #0: crashed: BUG: workqueue lockup
run #1: crashed: BUG: workqueue lockup
run #2: crashed: BUG: workqueue lockup
run #3: crashed: INFO: rcu detected stall in corrupted
run #4: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #5: crashed: INFO: rcu detected stall in corrupted
run #6: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #7: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #8: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #9: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #10: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #11: crashed: INFO: rcu detected stall in corrupted
run #12: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #13: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #14: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #15: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #16: crashed: INFO: rcu detected stall in corrupted
run #17: crashed: INFO: rcu detected stall in corrupted
run #18: crashed: INFO: rcu detected stall in corrupted
run #19: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
testing current HEAD eb575cd5d7f60241d016fdd13a9e86d962093c9b
testing commit eb575cd5d7f60241d016fdd13a9e86d962093c9b with gcc (GCC) 8.4.1 20210217
kernel signature: 1eb2fc602f921584e3203c34295e293b0ff73247d8ca9a7c6707c682543072cd
run #0: crashed: BUG: workqueue lockup
run #1: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #2: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #3: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #4: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #5: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #6: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
run #7: crashed: INFO: rcu detected stall in corrupted
run #8: crashed: INFO: rcu detected stall in corrupted
run #9: crashed: INFO: rcu detected stall in kvm_vcpu_ioctl
revisions tested: 2, total time: 28m13.875000169s (build: 17m38.877329753s, test: 9m41.976770689s)
the crash still happens on HEAD
commit msg: Linux 4.19.195
crash: INFO: rcu detected stall in kvm_vcpu_ioctl
L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.
hrtimer: interrupt took 36351 ns
kvm: emulating exchange as write
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 1-...!: (10500 ticks this GP) idle=276/1/0x4000000000000002 softirq=7952/7952 fqs=3
rcu: (t=10502 jiffies g=4961 q=203)
rcu: rcu_preempt kthread starved for 10497 jiffies! g4961 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: RCU grace-period kthread stack dump:
rcu_preempt R running task 29208 10 2 0x80000000
Call Trace:
context_switch kernel/sched/core.c:2828 [inline]
__schedule+0xdb5/0x1d40 kernel/sched/core.c:3517
schedule+0x7f/0x1b0 kernel/sched/core.c:3561
schedule_timeout+0x3ab/0xe00 kernel/time/timer.c:1818
rcu_gp_kthread+0xce6/0x23d0 kernel/rcu/tree.c:2202
kthread+0x347/0x410 kernel/kthread.c:259
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
NMI backtrace for cpu 1
CPU: 1 PID: 7184 Comm: syz-executor.4 Not tainted 4.19.195-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x123/0x171 lib/dump_stack.c:118
nmi_cpu_backtrace.cold.4+0x3e/0x76 lib/nmi_backtrace.c:101
nmi_trigger_cpumask_backtrace+0xe6/0x120 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_dump_cpu_stacks+0x175/0x1c2 kernel/rcu/tree.c:1340
print_cpu_stall kernel/rcu/tree.c:1478 [inline]
check_cpu_stall kernel/rcu/tree.c:1550 [inline]
__rcu_pending kernel/rcu/tree.c:3293 [inline]
rcu_pending kernel/rcu/tree.c:3336 [inline]
rcu_check_callbacks.cold.81+0x5b1/0xd89 kernel/rcu/tree.c:2682
update_process_times+0x2a/0x60 kernel/time/timer.c:1650
tick_sched_handle+0x77/0x140 kernel/time/tick-sched.c:168
tick_sched_timer+0x37/0xf0 kernel/time/tick-sched.c:1278
__run_hrtimer kernel/time/hrtimer.c:1419 [inline]
__hrtimer_run_queues+0x2f7/0xab0 kernel/time/hrtimer.c:1481
hrtimer_interrupt+0x2e2/0x760 kernel/time/hrtimer.c:1539
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1071 [inline]
smp_apic_timer_interrupt+0x114/0x540 arch/x86/kernel/apic/apic.c:1096
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
RIP: 0010:vcpu_enter_guest+0xba4/0x5030 arch/x86/kvm/x86.c:7818
Code: 00 fc ff df 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 a3 43 00 00 48 83 3d 21 67 46 07 00 0f 84 63 31 00 00 fb 66 0f 1f 44 00 00 01 00 00 00 e8 92 74 35 00 65 8b 05 fb 58 f6 7e 85 c0 0f 84 45
RSP: 0018:ffff8881d9137940 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: dffffc0000000000 RBX: ffff8881d7ca8040 RCX: 1ffff1103b27094d
RDX: 1ffffffff10a3f7b RSI: ffff8881d9384a48 RDI: ffffffff8851fbd8
RBP: ffff8881d9137a80 R08: ffff8881d9384a68 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8881d93841c0
R13: 000000384bd4cabd R14: ffffffe4012b6a16 R15: ffff8881d7ca8070
vcpu_run arch/x86/kvm/x86.c:7900 [inline]
kvm_arch_vcpu_ioctl_run+0x2e9/0x1340 arch/x86/kvm/x86.c:8100
kvm_vcpu_ioctl+0x535/0xb60 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2822
vfs_ioctl fs/ioctl.c:46 [inline]
file_ioctl fs/ioctl.c:501 [inline]
do_vfs_ioctl+0x196/0x10c0 fs/ioctl.c:688
ksys_ioctl+0x62/0x90 fs/ioctl.c:705
__do_sys_ioctl fs/ioctl.c:712 [inline]
__se_sys_ioctl fs/ioctl.c:710 [inline]
__x64_sys_ioctl+0x6e/0xb0 fs/ioctl.c:710
do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x463fe9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fb803efc198 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00000000000003bf RCX: 0000000000463fe9
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000006
RBP: 0000000000000010 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000053bf00
R13: 00007fb803efc6bc R14: 00000000ffffffff R15: 0000000000000003