syzbot

 sign-in | mailing list | source | docs
🐞 Open [717] 🐞 Fixed [181] 🐞 Invalid [640] 📈 Kernel Health 📈 Bugs/Month 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in hsr_announce

Status: upstream: reported C repro on 2019/12/17 23:42
Reported-by: syzbot+58767780977855002f83@syzkaller.appspotmail.com
First crash: 1645d, last: 1447d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in hsr_announce (3) net 1 1660d 1659d 0/27 closed as invalid on 2019/12/04 14:04
upstream INFO: rcu detected stall in hsr_announce (5) net 4 947d 959d 0/27 closed as invalid on 2022/02/08 10:00
upstream INFO: rcu detected stall in hsr_announce (6) kvm 2 785d 785d 0/27 auto-closed as invalid on 2022/06/25 07:59
upstream INFO: rcu detected stall in hsr_announce (4) net 1 1286d 1286d 0/27 auto-closed as invalid on 2021/03/11 15:56
upstream INFO: rcu detected stall in hsr_announce net 1 1740d 1740d 0/27 auto-closed as invalid on 2019/11/13 18:24
upstream INFO: rcu detected stall in hsr_announce (2) net 3 1664d 1664d 0/27 closed as invalid on 2019/11/29 14:24
linux-4.19 INFO: rcu detected stall in hsr_announce 1 1359d 1359d 0/1 auto-closed as invalid on 2021/01/27 07:34
▶ ▼ Last patch testing requests (4)
Created Duration User Patch Repo Result
2023/01/27 14:32 9m retest repro linux-4.14.y report log
2023/01/27 13:32 10m retest repro linux-4.14.y report log
2022/09/09 05:27 8m retest repro linux-4.14.y report log
2022/09/09 04:27 8m retest repro linux-4.14.y report log

Sample crash report:
IPv6: ADDRCONF(NETDEV_UP): veth1_virt_wifi: link is not ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_virt_wifi: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0_virt_wifi: link becomes ready
INFO: rcu_preempt self-detected stall on CPU
	1-...: (1 GPs behind) idle=ba6/140000000000002/0 softirq=38845/38847 fqs=67 
	 (t=10500 jiffies g=1767 c=1766 q=421)
NMI backtrace for cpu 1
CPU: 1 PID: 7816 Comm: syz-executor728 Not tainted 4.14.166-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x142/0x197 lib/dump_stack.c:58
 nmi_cpu_backtrace.cold+0x57/0x94 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x141/0x189 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
 rcu_dump_cpu_stacks+0x186/0x1d2 kernel/rcu/tree.c:1396
 print_cpu_stall kernel/rcu/tree.c:1542 [inline]
 check_cpu_stall kernel/rcu/tree.c:1610 [inline]
 __rcu_pending kernel/rcu/tree.c:3390 [inline]
 rcu_pending kernel/rcu/tree.c:3452 [inline]
 rcu_check_callbacks.cold+0x43d/0xd0a kernel/rcu/tree.c:2792
 update_process_times+0x31/0x70 kernel/time/timer.c:1590
 tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
 tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
 __run_hrtimer kernel/time/hrtimer.c:1223 [inline]
 __hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1287
 hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1321
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:__sanitizer_cov_trace_pc+0x9/0x60 kernel/kcov.c:65
RSP: 0018:ffff8880aed07760 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: ffff88808c59a680 RBX: 0000000000000000 RCX: 1ffffffff0f82dad
RDX: 0000000000000100 RSI: ffffffff87047cc0 RDI: ffffffff87e0dbe8
RBP: ffff8880aed077a8 R08: 0000000000000000 R09: ffff88808c59af98
R10: ffff88808c59af78 R11: ffff88808c59a680 R12: 0000000000000000
R13: dffffc0000000000 R14: 0000000000000000 R15: 0000000000000000
 is_bpf_text_address+0x61/0x120 kernel/bpf/core.c:457
 kernel_text_address+0x73/0xf0 kernel/extable.c:150
 __kernel_text_address+0xd/0x40 kernel/extable.c:105
 unwind_get_return_address arch/x86/kernel/unwind_frame.c:18 [inline]
 unwind_get_return_address+0x61/0xa0 arch/x86/kernel/unwind_frame.c:13
 __save_stack_trace+0x7b/0xd0 arch/x86/kernel/stacktrace.c:45
 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
 save_stack+0x45/0xd0 mm/kasan/kasan.c:447
 set_track mm/kasan/kasan.c:459 [inline]
 kasan_kmalloc mm/kasan/kasan.c:551 [inline]
 kasan_kmalloc+0xce/0xf0 mm/kasan/kasan.c:529
 kasan_slab_alloc+0xf/0x20 mm/kasan/kasan.c:489
 slab_post_alloc_hook mm/slab.h:442 [inline]
 slab_alloc mm/slab.c:3390 [inline]
 kmem_cache_alloc+0x11b/0x780 mm/slab.c:3550
 skb_clone+0x129/0x320 net/core/skbuff.c:1282
 frame_get_tagged_skb net/hsr/hsr_forward.c:195 [inline]
 hsr_forward_do net/hsr/hsr_forward.c:283 [inline]
 hsr_forward_skb+0xf46/0x1940 net/hsr/hsr_forward.c:373
 send_hsr_supervision_frame+0x908/0xf40 net/hsr/hsr_device.c:321
 hsr_announce+0xf5/0x340 net/hsr/hsr_device.c:350
 call_timer_fn+0x161/0x670 kernel/time/timer.c:1279
 expire_timers kernel/time/timer.c:1318 [inline]
 __run_timers kernel/time/timer.c:1636 [inline]
 __run_timers kernel/time/timer.c:1604 [inline]
 run_timer_softirq+0x5b7/0x1520 kernel/time/timer.c:1649
 __do_softirq+0x244/0x9a0 kernel/softirq.c:288
 invoke_softirq kernel/softirq.c:368 [inline]
 irq_exit+0x160/0x1b0 kernel/softirq.c:409
 exiting_irq arch/x86/include/asm/apic.h:648 [inline]
 smp_apic_timer_interrupt+0x146/0x5e0 arch/x86/kernel/apic/apic.c:1102
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
 </IRQ>
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x54/0x90 kernel/locking/spinlock.c:200
RSP: 0018:ffff888088e8f9a0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fe2d2b RBX: ffff88808c59a680 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: ffff88808c59af00 RDI: ffff88808c59aefc
RBP: ffff888088e8f9a8 R08: ffff88808c59a680 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880aed2b540
R13: ffff8880a9d44440 R14: ffff88809659ac80 R15: ffff8880aed2be90
 finish_lock_switch kernel/sched/sched.h:1346 [inline]
 finish_task_switch+0x178/0x650 kernel/sched/core.c:2675
 context_switch kernel/sched/core.c:2811 [inline]
 __schedule+0x7c0/0x1cd0 kernel/sched/core.c:3384
 preempt_schedule_irq+0xb5/0x140 kernel/sched/core.c:3611
 retint_kernel+0x1b/0x2d
RIP: 0010:qlink_to_object mm/kasan/quarantine.c:136 [inline]
RIP: 0010:qlink_free mm/kasan/quarantine.c:141 [inline]
RIP: 0010:qlist_free_all+0x39/0x150 mm/kasan/quarantine.c:166
RSP: 0018:ffff888088e8fba0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: 0000000000000000 RBX: 0000000000000286 RCX: ffffea000235de5f
RDX: 0000000000000000 RSI: ffff888085793b00 RDI: ffff888085793b00
RBP: ffff888088e8fbd0 R08: ffff88808c59a680 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: ffff8880963de850 R14: ffff888088e8fbe0 R15: ffff8880aa810040
 quarantine_reduce+0x147/0x180 mm/kasan/quarantine.c:259
 kasan_kmalloc+0xa0/0xf0 mm/kasan/kasan.c:536
 kasan_slab_alloc+0xf/0x20 mm/kasan/kasan.c:489
 kmem_cache_alloc+0x12e/0x780 mm/slab.c:3552
 kmem_cache_zalloc include/linux/slab.h:651 [inline]
 file_alloc_security security/selinux/hooks.c:368 [inline]
 selinux_file_alloc_security+0xb4/0x190 security/selinux/hooks.c:3475
 security_file_alloc+0x6d/0xa0 security/security.c:874
 get_empty_filp+0x162/0x3f0 fs/file_table.c:129
 alloc_file+0x23/0x440 fs/file_table.c:164
 sock_alloc_file+0x11c/0x2f0 net/socket.c:416
 sock_map_fd+0x36/0x80 net/socket.c:439
 SYSC_socket net/socket.c:1349 [inline]
 SyS_socket+0xf1/0x170 net/socket.c:1325
 do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x448999
RSP: 002b:00007f55b4ee3db8 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 00000000006dec28 RCX: 0000000000448999
RDX: 0000000000000084 RSI: 0000000000000001 RDI: 000000000000000a
RBP: 00000000006dec20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dec2c
R13: 00007ffd35f5db4f R14: 00007f55b4ee49c0 R15: 0000000000000000
INFO: rcu_sched detected stalls on CPUs/tasks:
	1-...: (1 GPs behind) idle=ba6/140000000000002/0 softirq=38845/38847 fqs=64 
	(detected by 0, t=10507 jiffies, g=1182, c=1181, q=4)
Sending NMI from CPU 0 to CPUs 1:
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_virt_wifi: link becomes ready
NMI backtrace for cpu 1
CPU: 1 PID: 7688 Comm: kworker/1:3 Not tainted 4.14.166-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events linkwatch_event
task: ffff888090f86400 task.stack: ffff888095cd0000
RIP: 0010:irqtime_account_irq+0x83/0x290 kernel/sched/cputime.c:63
RSP: 0018:ffff8880aed07f78 EFLAGS: 00000046
RAX: 0000010590a0d2ed RBX: ffff8880aed1f5f0 RCX: 1ffff11015da3ec0
RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffff8880aed1f600
RBP: ffff8880aed07f98 R08: 00000106df6b559f R09: ffff88821fff7048
R10: ffff88821fff7050 R11: 0000000000000001 R12: ffff888090f86400
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880aed00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f55b4ec2e78 CR3: 0000000096505000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 account_irq_enter_time include/linux/vtime.h:108 [inline]
 irq_enter+0x66/0xe0 kernel/softirq.c:353
 entering_irq arch/x86/include/asm/apic.h:629 [inline]
 entering_ack_irq arch/x86/include/asm/apic.h:635 [inline]
 smp_apic_timer_interrupt+0x24/0x5e0 arch/x86/kernel/apic/apic.c:1098
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
 </IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:779 [inline]
RIP: 0010:console_trylock_spinning kernel/printk/printk.c:1685 [inline]
RIP: 0010:vprintk_emit kernel/printk/printk.c:1922 [inline]
RIP: 0010:vprintk_emit+0x524/0x600 kernel/printk/printk.c:1888
RSP: 0018:ffff888095cd7910 EFLAGS: 00000297 ORIG_RAX: ffffffffffffff10
RAX: ffff888090f86400 RBX: 1ffffffff0fe2d29 RCX: 1ffff110121f0d9e
RDX: 0000000000000000 RSI: ffff888090f86cd0 RDI: 0000000000000297
RBP: ffff888095cd7950 R08: ffff888090f86400 R09: ffff888090f86cf0
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000042
R13: 0000000000000297 R14: ffff8880898dc3c0 R15: 0000000000000000
 vprintk_default+0x28/0x30 kernel/printk/printk.c:1963
 vprintk_func+0x5d/0x159 kernel/printk/printk_safe.c:401
 printk+0x9e/0xbc kernel/printk/printk.c:1996
 addrconf_notify.cold+0xfb/0x12f net/ipv6/addrconf.c:3442
 notifier_call_chain+0x111/0x1b0 kernel/notifier.c:93
 __raw_notifier_call_chain kernel/notifier.c:394 [inline]
 raw_notifier_call_chain+0x2e/0x40 kernel/notifier.c:401
 call_netdevice_notifiers_info+0x56/0x70 net/core/dev.c:1671
 netdev_state_change net/core/dev.c:1318 [inline]
 netdev_state_change+0xbf/0xe0 net/core/dev.c:1312
 linkwatch_do_dev+0x8d/0xf0 net/core/link_watch.c:164
 __linkwatch_run_queue+0x271/0x4b0 net/core/link_watch.c:202
 linkwatch_event+0x40/0x60 net/core/link_watch.c:237
 process_one_work+0x863/0x1600 kernel/workqueue.c:2114
 worker_thread+0x5d9/0x1050 kernel/workqueue.c:2248
 kthread+0x319/0x430 kernel/kthread.c:232
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
Code: 1c c5 60 6d c1 87 e8 0d c8 cc 01 89 c7 e8 b6 f1 ff ff 48 ba 00 00 00 00 00 fc ff df 48 8d 7b 10 48 89 f9 48 c1 e9 03 80 3c 11 00 <0f> 85 97 01 00 00 49 89 c5 4c 2b 6b 10 48 89 43 10 65 8b 05 35 
INFO: rcu_bh detected stalls on CPUs/tasks:
IPv6: ADDRCONF(NETDEV_CHANGE): veth0_virt_wifi: link becomes ready
	0-...: (1 GPs behind) idle=0b6/140000000000001/0 softirq=12313/12317 fqs=0 
	1-...: (1 ticks this GP) idle=ba6/140000000000001/0 softirq=38849/38850 fqs=0 
	(detected by 0, t=1 jiffies, g=-298, c=-299, q=46)
NMI backtrace for cpu 0
CPU: 0 PID: 7662 Comm: syz-executor728 Not tainted 4.14.166-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
IPv6: ADDRCONF(NETDEV_UP): veth0_macvtap: link is not ready
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x142/0x197 lib/dump_stack.c:58
 nmi_cpu_backtrace.cold+0x57/0x94 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x141/0x189 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
IPv6: ADDRCONF(NETDEV_UP): veth1_macvtap: link is not ready
 trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
 rcu_dump_cpu_stacks+0x186/0x1d2 kernel/rcu/tree.c:1396
device veth0_macvtap entered promiscuous mode
 print_other_cpu_stall kernel/rcu/tree.c:1485 [inline]
 check_cpu_stall kernel/rcu/tree.c:1616 [inline]
 __rcu_pending kernel/rcu/tree.c:3390 [inline]
 rcu_pending kernel/rcu/tree.c:3452 [inline]
 rcu_check_callbacks.cold+0x744/0xd0a kernel/rcu/tree.c:2792
IPv6: ADDRCONF(NETDEV_UP): macvtap0: link is not ready
 update_process_times+0x31/0x70 kernel/time/timer.c:1590
 tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
 tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
 __run_hrtimer kernel/time/hrtimer.c:1223 [inline]
 __hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1287
 hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1321
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
device veth1_macvtap entered promiscuous mode
 </IRQ>
IPv6: ADDRCONF(NETDEV_UP): macsec0: link is not ready
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x54/0x90 kernel/locking/spinlock.c:200
IPv6: ADDRCONF(NETDEV_UP): veth0_to_batadv: link is not ready
RSP: 0018:ffff888088e77b80 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff10
IPv6: ADDRCONF(NETDEV_UP): veth1_to_batadv: link is not ready
RAX: 1ffffffff0fe2d2b RBX: ffff8880898dc3c0 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffff8880898dcc3c
batman_adv: The newly added mac address (aa:aa:aa:aa:aa:3d) already exists on: batadv_slave_0
RBP: ffff888088e77b88 R08: ffff8880898dc3c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880aec2b540
batman_adv: It is strongly recommended to keep mac addresses unique to avoid problems!
R13: ffff88809373a2c0 R14: 0000000000000000 R15: ffff8880aec2be90
 finish_lock_switch kernel/sched/sched.h:1346 [inline]
 finish_task_switch+0x178/0x650 kernel/sched/core.c:2675
batman_adv: The newly added mac address (aa:aa:aa:aa:aa:3d) already exists on: batadv_slave_0
 context_switch kernel/sched/core.c:2811 [inline]
 __schedule+0x7c0/0x1cd0 kernel/sched/core.c:3384
 schedule+0x92/0x1c0 kernel/sched/core.c:3428
batman_adv: It is strongly recommended to keep mac addresses unique to avoid problems!
 freezable_schedule include/linux/freezer.h:172 [inline]
 do_nanosleep+0x1d7/0x5e0 kernel/time/hrtimer.c:1483
 hrtimer_nanosleep+0x218/0x4a0 kernel/time/hrtimer.c:1537
IPv6: ADDRCONF(NETDEV_UP): batadv_slave_0: link is not ready
 SYSC_nanosleep kernel/time/hrtimer.c:1569 [inline]
 SyS_nanosleep+0x11a/0x170 kernel/time/hrtimer.c:1556
 do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
batman_adv: batadv0: Interface activated: batadv_slave_0
RIP: 0033:0x446fe0
RSP: 002b:00007ffd35f5dba8 EFLAGS: 00000246 ORIG_RAX: 0000000000000023
RAX: ffffffffffffffda RBX: 0000000000112188 RCX: 0000000000446fe0
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00007ffd35f5dbb0
RBP: 0000000000000012 R08: 0000000000000001 R09: 000000000088f880
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000006
R13: 0000000000409580 R14: 0000000000000000 R15: 0000000000000000
batman_adv: The newly added mac address (aa:aa:aa:aa:aa:3e) already exists on: batadv_slave_1
batman_adv: It is strongly recommended to keep mac addresses unique to avoid problems!
batman_adv: The newly added mac address (aa:aa:aa:aa:aa:3e) already exists on: batadv_slave_1
batman_adv: It is strongly recommended to keep mac addresses unique to avoid problems!
IPv6: ADDRCONF(NETDEV_UP): batadv_slave_1: link is not ready
batman_adv: batadv0: Interface activated: batadv_slave_1
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_macvtap: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0_macvtap: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): macsec0: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_0: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0_to_batadv: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready

Crashes (25):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/01/22 06:44 linux-4.14.y c1141b3aab36 8eda0b95 .config console log report syz C ci2-linux-4-14
2019/12/22 08:33 linux-4.14.y e1f7d50ae3a3 bc586918 .config console log report syz C ci2-linux-4-14
2020/07/03 12:15 linux-4.14.y b850307b279c 6e569755 .config console log report ci2-linux-4-14
2020/06/18 08:37 linux-4.14.y b850307b279c d45a4d69 .config console log report ci2-linux-4-14
2020/05/07 04:11 linux-4.14.y d71f695ce745 4618eb2d .config console log report ci2-linux-4-14
2020/04/29 10:18 linux-4.14.y 050272a0423e ba2806db .config console log report ci2-linux-4-14
2020/04/15 03:51 linux-4.14.y c10b57a567e4 3f3c5574 .config console log report ci2-linux-4-14
2020/03/24 04:17 linux-4.14.y 01364dad1d45 33e14df3 .config console log report ci2-linux-4-14
2020/03/11 12:57 linux-4.14.y 78d697fc93f9 e103bc9e .config console log report ci2-linux-4-14
2020/03/06 02:45 linux-4.14.y 78d697fc93f9 b655d91b .config console log report ci2-linux-4-14
2020/03/02 02:25 linux-4.14.y 78d697fc93f9 4a4e0509 .config console log report ci2-linux-4-14
2020/02/28 17:09 linux-4.14.y 78d697fc93f9 c88c7b75 .config console log report ci2-linux-4-14
2020/02/16 10:32 linux-4.14.y 98db2bf27b9e cf914200 .config console log report ci2-linux-4-14
2020/02/13 02:31 linux-4.14.y e0f8b8a65a47 84f4fc8a .config console log report ci2-linux-4-14
2020/02/04 04:10 linux-4.14.y 9fa690a2a016 93e5e335 .config console log report ci2-linux-4-14
2020/01/15 22:15 linux-4.14.y c04fc6fa5c96 f9b69507 .config console log report ci2-linux-4-14
2020/01/11 06:00 linux-4.14.y b0cdffaa546e 4c04afaa .config console log report ci2-linux-4-14
2020/01/10 11:42 linux-4.14.y b0cdffaa546e 4de4e9f0 .config console log report ci2-linux-4-14
2020/01/09 12:05 linux-4.14.y b0cdffaa546e ddc3e859 .config console log report ci2-linux-4-14
2020/01/04 06:13 linux-4.14.y 4c5bf01e16a7 68256974 .config console log report ci2-linux-4-14
2020/01/04 05:34 linux-4.14.y 4c5bf01e16a7 68256974 .config console log report ci2-linux-4-14
2020/01/03 21:56 linux-4.14.y 4c5bf01e16a7 68256974 .config console log report ci2-linux-4-14
2019/12/24 14:49 linux-4.14.y e1f7d50ae3a3 be5c2c81 .config console log report ci2-linux-4-14
2019/12/24 10:50 linux-4.14.y e1f7d50ae3a3 be5c2c81 .config console log report ci2-linux-4-14
2019/12/17 23:42 linux-4.14.y bfb9e5c03076 1af3875f .config console log report ci2-linux-4-14
* Struck through repros no longer work on HEAD.