syzbot


BUG: soft lockup in expire_timers

Status: upstream: reported C repro on 2022/03/12 18:01
Reported-by: syzbot+adadd45f55064a0e9d5c@syzkaller.appspotmail.com
First crash: 824d, last: 480d
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 BUG: soft lockup in expire_timers 1 1331d 1331d 0/1 auto-closed as invalid on 2021/02/19 12:04
linux-4.19 BUG: soft lockup in expire_timers (2) 44 492d 1078d 0/1 upstream: reported on 2021/07/01 19:31
Fix bisection attempts (7)
Created Duration User Patch Repo Result
2023/02/19 22:08 25m bisect fix linux-4.14.y job log (0) log
2023/01/20 15:15 29m bisect fix linux-4.14.y job log (0) log
2022/10/28 14:15 30m bisect fix linux-4.14.y job log (0) log
2022/09/23 01:42 23m bisect fix linux-4.14.y job log (0) log
2022/08/24 01:04 30m bisect fix linux-4.14.y job log (0) log
2022/07/25 00:07 24m bisect fix linux-4.14.y job log (0) log
2022/06/24 23:40 26m bisect fix linux-4.14.y job log (0) log

Sample crash report:
watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [syz-executor271:29653]
Modules linked in:
irq event stamp: 10748615
hardirqs last  enabled at (10748614): [<ffffffff87247a64>] __raw_spin_unlock_irq include/linux/spinlock_api_smp.h:168 [inline]
hardirqs last  enabled at (10748614): [<ffffffff87247a64>] _raw_spin_unlock_irq+0x24/0x80 kernel/locking/spinlock.c:200
hardirqs last disabled at (10748615): [<ffffffff874018ae>] apic_timer_interrupt+0x8e/0xa0 arch/x86/entry/entry_64.S:793
softirqs last  enabled at (33396): [<ffffffff86363836>] inet6_fill_ifla6_attrs+0x1846/0x1d10 net/ipv6/addrconf.c:5208
softirqs last disabled at (33457): [<ffffffff81321cf3>] invoke_softirq kernel/softirq.c:368 [inline]
softirqs last disabled at (33457): [<ffffffff81321cf3>] irq_exit+0x193/0x240 kernel/softirq.c:409
CPU: 1 PID: 29653 Comm: syz-executor271 Not tainted 4.14.281-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff88804a56c040 task.stack: ffff8880552b0000
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x50/0x80 kernel/locking/spinlock.c:200
RSP: 0018:ffff8880ba507df8 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff11e1313 RBX: dffffc0000000000 RCX: 1ffff110094ad922
RDX: dffffc0000000000 RSI: ffff88804a56c8f0 RDI: ffff88804a56c8c4
RBP: ffff8880ba52cb00 R08: ffffffff8b9c9430 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880ba52cb00
R13: ffff8880ba507e98 R14: 1ffff110174a0fd3 R15: ffffffff85de5480
FS:  00007f37390d3700(0000) GS:ffff8880ba500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f37390b2718 CR3: 000000005f1b3000 CR4: 00000000003406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 expire_timers+0x222/0x4d0 kernel/time/timer.c:1318
 __run_timers kernel/time/timer.c:1637 [inline]
 run_timer_softirq+0x1d5/0x5a0 kernel/time/timer.c:1650
 __do_softirq+0x24d/0x9ff kernel/softirq.c:288
 invoke_softirq kernel/softirq.c:368 [inline]
 irq_exit+0x193/0x240 kernel/softirq.c:409
 exiting_irq arch/x86/include/asm/apic.h:638 [inline]
 smp_apic_timer_interrupt+0x141/0x5e0 arch/x86/kernel/apic/apic.c:1106
 apic_timer_interrupt+0x93/0xa0 arch/x86/entry/entry_64.S:793
 </IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:779 [inline]
RIP: 0010:generic_exec_single+0x2b5/0x420 kernel/smp.c:154
RSP: 0018:ffff8880552b7b70 EFLAGS: 00000297 ORIG_RAX: ffffffffffffff10
RAX: ffff88804a56c040 RBX: ffffffff816354d0 RCX: 1ffff110094ad922
RDX: 0000000000000000 RSI: ffff88804a56c8f0 RDI: 0000000000000297
RBP: 0000000000000297 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880552b7bf8
R13: ffff8880552b7c88 R14: 0000000000000001 R15: ffff88804a56c07c
 smp_call_function_single+0x16f/0x370 kernel/smp.c:299
 task_function_call+0xcc/0x130 kernel/events/core.c:115
 perf_install_in_context+0x1ad/0x2f0 kernel/events/core.c:2517
 SYSC_perf_event_open kernel/events/core.c:10404 [inline]
 SyS_perf_event_open+0x1aa4/0x2530 kernel/events/core.c:10015
 do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x46/0xbb
RIP: 0033:0x7f37391229d9
RSP: 002b:00007f37390d3318 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007f37391ab428 RCX: 00007f37391229d9
RDX: ffffffffffffffff RSI: 0000000000000000 RDI: 0000000020000440
RBP: 00007f37391ab420 R08: 0000000000000000 R09: 0000000000000000
R10: 00000000ffffffff R11: 0000000000000246 R12: 00007f3739178174
R13: 00007ffc364bfbff R14: 00007f37390d3400 R15: 0000000000022000
Code: c7 c0 98 98 f0 88 48 ba 00 00 00 00 00 fc ff df 48 c1 e8 03 80 3c 10 00 75 31 48 83 3d 11 1e cc 01 00 74 25 fb 66 0f 1f 44 00 00 <bf> 01 00 00 00 e8 26 a9 14 fa 65 8b 05 af 04 de 78 85 c0 74 02 
----------------
Code disassembly (best guess):
   0:	c7 c0 98 98 f0 88    	mov    $0x88f09898,%eax
   6:	48 ba 00 00 00 00 00 	movabs $0xdffffc0000000000,%rdx
   d:	fc ff df
  10:	48 c1 e8 03          	shr    $0x3,%rax
  14:	80 3c 10 00          	cmpb   $0x0,(%rax,%rdx,1)
  18:	75 31                	jne    0x4b
  1a:	48 83 3d 11 1e cc 01 	cmpq   $0x0,0x1cc1e11(%rip)        # 0x1cc1e33
  21:	00
  22:	74 25                	je     0x49
  24:	fb                   	sti
  25:	66 0f 1f 44 00 00    	nopw   0x0(%rax,%rax,1)
* 2b:	bf 01 00 00 00       	mov    $0x1,%edi <-- trapping instruction
  30:	e8 26 a9 14 fa       	callq  0xfa14a95b
  35:	65 8b 05 af 04 de 78 	mov    %gs:0x78de04af(%rip),%eax        # 0x78de04eb
  3c:	85 c0                	test   %eax,%eax
  3e:	74 02                	je     0x42

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/05/25 23:40 linux-4.14.y 501eec4f9e13 3037caa9 .config console log report syz C ci2-linux-4-14 BUG: soft lockup in expire_timers
2022/03/12 18:01 linux-4.14.y af48f51cb593 9e8eaa75 .config console log report info ci2-linux-4-14 BUG: soft lockup in expire_timers
* Struck through repros no longer work on HEAD.