syzbot

 sign-in | mailing list | source | docs
🐞 Open [118]
🐞 Fixed [70]
🐞 Invalid [157]
Kernel Health Bugs/Month Bug Lifetimes Fuzzing Crashes
💬 Send us feedback

BUG: soft lockup in __run_timers

Status: premoderation: reported on 2024/06/12 12:48
Reported-by: syzbot+e4678ccace18fcbbcbda@syzkaller.appspotmail.com
First crash: 125d, last: 56d
▶ ▼ Similar bugs (6)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-54 BUG: soft lockup in __run_timers 1 109d 109d 0/2 auto-obsoleted due to no activity on 2024/09/26 00:22
android-5-15 BUG: soft lockup in __run_timers 1 27d 27d 0/2 premoderation: reported on 2024/09/17 23:46
upstream BUG: soft lockup in __run_timers kernel 1 618d 614d 0/28 auto-obsoleted due to no activity on 2023/05/06 12:08
upstream INFO: rcu detected stall in __run_timers (5) kernel C error error 63 987d 1501d 0/28 closed as invalid on 2022/02/08 10:39
linux-5.15 INFO: rcu detected stall in __run_timers 1 546d 546d 0/3 auto-obsoleted due to no activity on 2023/08/15 19:18
upstream INFO: rcu detected stall in __run_timers (6) kernel 2 955d 972d 0/28 auto-closed as invalid on 2022/06/02 21:46

Sample crash report:
watchdog: BUG: soft lockup - CPU#1 stuck for 122s! [syz.2.673:4056]
Modules linked in:
CPU: 1 PID: 4056 Comm: syz.2.673 Tainted: G        W         5.10.223-syzkaller-01561-g0890c03b8b7d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x49/0x70 kernel/locking/spinlock.c:199
Code: 08 00 74 0c 48 c7 c7 f0 77 6c 86 e8 c1 07 f6 fc 48 83 3d c9 3d ba 01 00 74 2a 48 89 df e8 f3 64 9e fc 66 90 fb bf 01 00 00 00 <e8> 12 d5 96 fc 65 8b 05 47 33 50 7b 85 c0 74 03 5b 5d c3 e8 b3 41
RSP: 0000:ffffc90000170da8 EFLAGS: 00000246
RAX: 0000000000000001 RBX: ffff8881f7147840 RCX: dffffc0000000000
RDX: 0000000000000102 RSI: 0000000000000000 RDI: 0000000000000001
RBP: ffffc90000170db0 R08: ffffffff815adacf R09: 0000000000000003
R10: fffff5200002e1a8 R11: dffffc0000000001 R12: dffffc0000000000
R13: ffff8881107d5268 R14: ffffffff842813b0 R15: ffff8881f7147848
FS:  00007f53565d06c0(0000) GS:ffff8881f7100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fdddcd82370 CR3: 000000012566c000 CR4: 00000000003506a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 expire_timers kernel/time/timer.c:1494 [inline]
 __run_timers+0x71a/0xa10 kernel/time/timer.c:1789
 run_timer_softirq+0x69/0xf0 kernel/time/timer.c:1802
 __do_softirq+0x268/0x5bb kernel/softirq.c:309
 asm_call_irq_on_stack+0xf/0x20
 </IRQ>
 __run_on_irqstack arch/x86/include/asm/irq_stack.h:26 [inline]
 run_on_irqstack_cond arch/x86/include/asm/irq_stack.h:77 [inline]
 do_softirq_own_stack+0x60/0x80 arch/x86/kernel/irq_64.c:77
 invoke_softirq kernel/softirq.c:405 [inline]
 __irq_exit_rcu+0x128/0x150 kernel/softirq.c:435
 irq_exit_rcu+0x9/0x10 kernel/softirq.c:447
 sysvec_apic_timer_interrupt+0xbf/0xe0 arch/x86/kernel/apic/apic.c:1094
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:635
RIP: 0010:schedule_debug kernel/sched/core.c:4553 [inline]
RIP: 0010:__schedule+0x11c/0x1330 kernel/sched/core.c:4698
Code: 3b 49 8d 5f 18 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 33 3b f7 fc 48 8b 1b 48 89 d8 48 c1 e8 03 42 80 3c 28 00 <74> 08 48 89 df e8 1a 3b f7 fc 48 81 3b 9d 6e ac 57 0f 85 c0 0d 00
RSP: 0000:ffffc90000d67c00 EFLAGS: 00000246
RAX: 1ffff920001ac000 RBX: ffffc90000d60000 RCX: 1ffff920001acf8c
RDX: 1ffff920001acf8c RSI: ffffffff85601f20 RDI: ffffffff85601ee0
RBP: ffffc90000d67cf0 R08: ffffffff87084048 R09: ffffffff87084058
R10: ffffffff87084050 R11: ffffffff87084043 R12: 0000000000000001
R13: dffffc0000000000 R14: ffff8881f7100000 R15: ffff888128464f00
 preempt_schedule_irq+0xc7/0x140 kernel/sched/core.c:5065
 irqentry_exit_cond_resched kernel/entry/common.c:365 [inline]
 irqentry_exit+0x4f/0x60 kernel/entry/common.c:395
 sysvec_apic_timer_interrupt+0xcb/0xe0 arch/x86/kernel/apic/apic.c:1094
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:635
RIP: 0010:preempt_count_add+0x12f/0x1a0 kernel/sched/core.c:4454
Code: 9a 05 00 0f 85 68 ff ff ff 48 c7 c7 80 1a 08 85 48 c7 c6 c0 1a 08 85 e8 ff 20 f5 ff 0f 0b e9 4e ff ff ff 48 c7 c1 00 b9 08 87 <80> e1 07 80 c1 03 38 c1 0f 8c f0 fe ff ff 48 c7 c7 00 b9 08 87 e8
RSP: 0000:ffffc90000d67e80 EFLAGS: 00000202
RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffffffff8708b900
RDX: 1ffffffff0e10808 RSI: 0000000000000008 RDI: 0000000000000001
RBP: ffffc90000d67e98 R08: ffffffff87084048 R09: ffffffff87084058
R10: ffffffff87084050 R11: ffffffff87084043 R12: dffffc0000000000
R13: ffff888128464f00 R14: ffff888128464f10 R15: dffffc0000000000
 schedule+0x136/0x1d0 kernel/sched/core.c:4881
 exit_to_user_mode_loop+0x48/0xd0 kernel/entry/common.c:160
 exit_to_user_mode_prepare kernel/entry/common.c:199 [inline]
 irqentry_exit_to_user_mode+0x4e/0x80 kernel/entry/common.c:287
 irqentry_exit+0x12/0x60 kernel/entry/common.c:375
 sysvec_apic_timer_interrupt+0xcb/0xe0 arch/x86/kernel/apic/apic.c:1094
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:635
RIP: 0033:0x7f5357952e79
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f53565d00e8 EFLAGS: 00000246
RAX: 0000000000000001 RBX: 00007f5357aeef88 RCX: 00007f5357952e79
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00007f5357aeef8c
RBP: 00007f5357aeef80 R08: 00007fffe459f0b0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f5357aeef8c
R13: 0000000000000000 R14: 00007fffe454f680 R15: 00007fffe454f768
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 3954 Comm: syz.4.655 Tainted: G        W         5.10.223-syzkaller-01561-g0890c03b8b7d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
RIP: 0010:__sanitizer_cov_trace_switch+0x32/0xe0 kernel/kcov.c:319
Code: 41 54 53 48 8b 46 08 48 83 c0 f8 48 c1 c0 3d 48 83 f8 07 0f 87 ab 00 00 00 b9 8b 00 00 00 0f a3 c1 0f 83 9d 00 00 00 4c 8b 16 <4d> 85 d2 0f 84 91 00 00 00 4c 8b 04 c5 08 4d 43 85 4c 8b 4d 08 65
RSP: 0018:ffffc900000077c0 EFLAGS: 00000097
RAX: 0000000000000003 RBX: 0000000000000003 RCX: 000000000000008b
RDX: 0000000000010101 RSI: ffffffff867ab200 RDI: 0000000000000003
RBP: ffffc900000077e0 R08: ffffffff815df026 R09: ffff88810dd8a960
R10: 0000000000000002 R11: dffffc0000000001 R12: 00000114f7d0a67e
R13: dffffc0000000000 R14: 0000000000000000 R15: ffff8881f7026e80
FS:  0000000000000000(0000) GS:ffff8881f7000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055555623b4a8 CR3: 000000011d331000 CR4: 00000000003506b0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 clockevents_program_event+0x7d/0x2c0 kernel/time/clockevents.c:315
 tick_program_event+0x9f/0x120 kernel/time/tick-oneshot.c:44
 hrtimer_interrupt+0x4cf/0x8b0 kernel/time/hrtimer.c:1722
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1083 [inline]
 __sysvec_apic_timer_interrupt+0xfd/0x3c0 arch/x86/kernel/apic/apic.c:1100
 run_sysvec_on_irqstack_cond arch/x86/include/asm/irq_stack.h:91 [inline]
 sysvec_apic_timer_interrupt+0xba/0xe0 arch/x86/kernel/apic/apic.c:1094
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:635
RIP: 0010:ip6_mc_input+0x6a/0x2a0 net/ipv6/ip6_input.c:490
Code: 1b 48 85 db 0f 84 d2 01 00 00 48 81 c3 e0 02 00 00 48 89 d8 48 c1 e8 03 42 80 3c 28 00 74 08 48 89 df e8 b9 e8 82 fd 48 8b 1b <48> 85 db 0f 84 c2 01 00 00 e8 c8 4f 45 fd 48 81 c3 50 02 00 00 48
RSP: 0018:ffffc90000007a88 EFLAGS: 00000246
RAX: 1ffff11023957c5c RBX: ffff8881115d9000 RCX: ffff888128554f00
RDX: 0000000000000101 RSI: 0000000000000000 RDI: ffff88811fca8500
RBP: ffffc90000007ab8 R08: ffffffff8434bb46 R09: ffff88811fca8500
R10: ffffc90000007560 R11: dffffc0000000001 R12: dffffc0000000000
R13: dffffc0000000000 R14: 1ffff11023f950a2 R15: ffff88811fca8500
 dst_input include/net/dst.h:443 [inline]
 ip6_rcv_finish+0x185/0x350 net/ipv6/ip6_input.c:80
 NF_HOOK include/linux/netfilter.h:299 [inline]
 ipv6_rcv+0xee/0x270 net/ipv6/ip6_input.c:301
 __netif_receive_skb_one_core net/core/dev.c:5382 [inline]
 __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5496
 process_backlog+0x31c/0x650 net/core/dev.c:6403
 napi_poll net/core/dev.c:6854 [inline]
 net_rx_action+0x516/0x10d0 net/core/dev.c:6924
 __do_softirq+0x268/0x5bb kernel/softirq.c:309
 asm_call_irq_on_stack+0xf/0x20
 </IRQ>
 __run_on_irqstack arch/x86/include/asm/irq_stack.h:26 [inline]
 run_on_irqstack_cond arch/x86/include/asm/irq_stack.h:77 [inline]
 do_softirq_own_stack+0x60/0x80 arch/x86/kernel/irq_64.c:77
 invoke_softirq kernel/softirq.c:405 [inline]
 __irq_exit_rcu+0x128/0x150 kernel/softirq.c:435
 irq_exit_rcu+0x9/0x10 kernel/softirq.c:447
 sysvec_apic_timer_interrupt+0xbf/0xe0 arch/x86/kernel/apic/apic.c:1094
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:635
RIP: 0010:bytes_is_nonzero mm/kasan/generic.c:85 [inline]
RIP: 0010:memory_is_nonzero mm/kasan/generic.c:102 [inline]
RIP: 0010:memory_is_poisoned_n mm/kasan/generic.c:128 [inline]
RIP: 0010:memory_is_poisoned mm/kasan/generic.c:159 [inline]
RIP: 0010:check_region_inline mm/kasan/generic.c:180 [inline]
RIP: 0010:kasan_check_range+0x82/0x2a0 mm/kasan/generic.c:189
Code: 00 00 00 00 fc ff df 4f 8d 34 1a 4d 89 f5 4d 29 cd 49 83 fd 10 7f 26 4d 85 ed 0f 84 44 01 00 00 49 f7 d2 4d 01 e2 41 80 39 00 <0f> 85 d2 01 00 00 49 ff c1 49 ff c2 75 ee e9 27 01 00 00 45 89 cf
RSP: 0018:ffffc900011873f8 EFLAGS: 00000246
RAX: 0000000000000001 RBX: 000000000001ffff RCX: ffffffff84b23277
RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffffea00047abca8
RBP: ffffc90001187420 R08: dffffc0000000000 R09: fffff940008f5795
R10: ffffffffffffffff R11: dffffc0000000001 R12: 1ffffd40008f5795
R13: 0000000000000001 R14: fffff940008f5796 R15: ffffc90001187460
 __kasan_check_write+0x14/0x20 mm/kasan/shadow.c:37
 instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
 atomic_try_cmpxchg_acquire include/asm-generic/atomic-instrumented.h:705 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:82 [inline]
 do_raw_spin_lock include/linux/spinlock.h:183 [inline]
 __raw_spin_lock include/linux/spinlock_api_smp.h:143 [inline]
 _raw_spin_lock+0x97/0x1b0 kernel/locking/spinlock.c:151
 spin_lock include/linux/spinlock.h:354 [inline]
 zap_pte_range mm/memory.c:1272 [inline]
 zap_pmd_range mm/memory.c:1435 [inline]
 zap_pud_range mm/memory.c:1464 [inline]
 zap_p4d_range mm/memory.c:1485 [inline]
 unmap_page_range+0xab0/0x23b0 mm/memory.c:1506
 unmap_single_vma mm/memory.c:1551 [inline]
 unmap_vmas+0x37f/0x4f0 mm/memory.c:1583
 exit_mmap+0x2f2/0x5c0 mm/mmap.c:3355
 __mmput+0x95/0x2d0 kernel/fork.c:1153
 mmput+0x59/0x170 kernel/fork.c:1176
 exit_mm kernel/exit.c:539 [inline]
 do_exit+0xbda/0x2a50 kernel/exit.c:850
 do_group_exit+0x141/0x310 kernel/exit.c:985
 get_signal+0x10a0/0x1410 kernel/signal.c:2782
 arch_do_signal_or_restart+0xbd/0x17c0 arch/x86/kernel/signal.c:805
 handle_signal_work kernel/entry/common.c:145 [inline]
 exit_to_user_mode_loop+0x9b/0xd0 kernel/entry/common.c:169
 exit_to_user_mode_prepare kernel/entry/common.c:199 [inline]
 syscall_exit_to_user_mode+0xa2/0x1a0 kernel/entry/common.c:274
 do_syscall_64+0x40/0x70 arch/x86/entry/common.c:56
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7f6aded26e79
Code: Unable to access opcode bytes at RIP 0x7f6aded26e4f.
RSP: 002b:00007f6add9a40e8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
RAX: 0000000000000001 RBX: 00007f6adeec2f88 RCX: 00007f6aded26e79
RDX: 00000000000f4240 RSI: 0000000000000081 RDI: 00007f6adeec2f8c
RBP: 00007f6adeec2f80 R08: 00007ffe531390b0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f6adeec2f8c
R13: 0000000000000000 R14: 00007ffe53093480 R15: 00007ffe53093568

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/08/20 04:41 android13-5.10-lts 0890c03b8b7d 9f0ab3fb .config console log report info [disk image] [vmlinux] [kernel image] ci2-android-5-10-perf BUG: soft lockup in __run_timers
2024/06/12 12:47 android13-5.10-lts d3c4d815bf09 f815599d .config console log report info [disk image] [vmlinux] [kernel image] ci2-android-5-10-perf BUG: soft lockup in __run_timers
* Struck through repros no longer work on HEAD.