syzbot

 sign-in | mailing list | source | docs
🐞 Open [546] 🐞 Fixed [32] 🐞 Invalid [297] ⬇ Missing Backports [45] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in __do_softirq

Status: auto-obsoleted due to no activity on 2024/02/08 12:45
Reported-by: syzbot+82bb8c294e6a20f1a15e@syzkaller.appspotmail.com
First crash: 198d, last: 179d
▶ ▼ Similar bugs (5)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: rcu detected stall in __do_softirq (2) 9 1322d 1508d 0/1 auto-closed as invalid on 2021/01/10 17:25
linux-4.14 INFO: rcu detected stall in __do_softirq 1 1775d 1775d 0/1 auto-closed as invalid on 2019/10/25 08:47
linux-4.19 INFO: rcu detected stall in __do_softirq 2 1710d 1774d 0/1 auto-closed as invalid on 2019/12/20 09:57
upstream INFO: rcu detected stall in __do_softirq net syz done 35 493d 1726d 0/26 auto-obsoleted due to no activity on 2023/04/21 04:43
linux-4.14 BUG: soft lockup in __do_softirq syz error 3 1088d 1130d 0/1 upstream: reported syz repro on 2021/03/23 20:32

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=a03/1/0x4000000000000000 softirq=75805/75805 fqs=0 
	(detected by 1, t=10502 jiffies, g=116781, q=823)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 22970 Comm: syz-executor.0 Not tainted 5.15.137-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
RIP: 0010:__lock_release kernel/locking/lockdep.c:5278 [inline]
RIP: 0010:lock_release+0x188/0x9a0 kernel/locking/lockdep.c:5642
Code: 80 3c 3b 00 74 08 4c 89 f7 e8 b4 33 67 00 48 8b 9c 24 b0 00 00 00 fa 48 c7 c7 e0 19 8b 8a e8 5f d9 b8 08 65 ff 05 d8 d4 9f 7e <48> 8d 94 24 80 00 00 00 48 c1 ea 03 42 0f b6 04 3a 84 c0 4c 8b 6c
RSP: 0000:ffffc900000078e0 EFLAGS: 00000002
RAX: 0000000000000000 RBX: 0000000000000046 RCX: ffffffff8162a348
RDX: 0000000000000000 RSI: ffffffff8a8b19e0 RDI: ffffffff8ad87b40
RBP: ffffc90000007a10 R08: dffffc0000000000 R09: fffffbfff1bc71e6
R10: 0000000000000000 R11: dffffc0000000001 R12: 1ffff92000000f28
R13: ffffffff816f346c R14: ffffc90000007990 R15: dffffc0000000000
FS:  00005555563f6480(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff9c7682934 CR3: 00000000880f1000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:158 [inline]
 _raw_spin_unlock_irqrestore+0x75/0x130 kernel/locking/spinlock.c:194
 __run_hrtimer kernel/time/hrtimer.c:1681 [inline]
 __hrtimer_run_queues+0x48c/0xcf0 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x3e/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:ffs arch/x86/include/asm/bitops.h:297 [inline]
RIP: 0010:__do_softirq+0x1d2/0x93a kernel/softirq.c:546
Code: 24 78 4c 89 7c 24 70 0f b7 db 48 c7 c7 40 f3 89 8a e8 c2 7b bb ff 65 66 c7 05 f8 8a a3 75 00 00 e8 c3 a1 26 f7 fb 89 5c 24 34 <b8> ff ff ff ff 0f bc 44 24 34 41 89 c4 41 ff c4 0f 85 05 01 00 00
RSP: 0000:ffffc90000007e20 EFLAGS: 00000282
RAX: ce353f5cfe1e9200 RBX: 0000000000000008 RCX: ffffffff8162ea18
RDX: dffffc0000000000 RSI: ffffffff8a8b0be0 RDI: ffffffff8ad87b40
RBP: ffffc90000007f30 R08: dffffc0000000000 R09: fffffbfff1f79a27
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888081dbd940
R13: 1ffff92000000fe8 R14: dffffc0000000000 R15: 1ffff92000000fd8
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:637
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649
 common_interrupt+0xa4/0xc0 arch/x86/kernel/irq.c:240
 </IRQ>
 <TASK>
 asm_common_interrupt+0x22/0x40 arch/x86/include/asm/idtentry.h:629
RIP: 0010:lock_is_held include/linux/lockdep.h:287 [inline]
RIP: 0010:___might_sleep+0xe0/0x6a0 kernel/sched/core.c:9586
Code: 75 1f c6 05 62 34 77 0c 01 48 c7 c7 a0 93 8a 8a be 72 25 00 00 48 c7 c2 e0 9a 8a 8a e8 19 e7 09 00 e8 44 7c c2 08 85 c0 74 46 <48> c7 c7 a0 ef 91 8c be ff ff ff ff e8 df 78 c2 08 85 c0 74 31 e8
RSP: 0000:ffffc900057b7740 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff888081dbd96c RCX: ffff888081dbd940
RDX: 0000000000000000 RSI: ffffffff8a8b1e80 RDI: ffffffff8ad87b40
RBP: ffffc900057b7858 R08: dffffc0000000000 R09: fffffbfff1bc71e6
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000000
R13: ffffc900057b7940 R14: dffffc0000000000 R15: 1ffff92000af6ef0
 prepare_alloc_pages+0x1ca/0x5b0 mm/page_alloc.c:5196
 __alloc_pages+0x14f/0x700 mm/page_alloc.c:5410
 alloc_pages_vma+0x39a/0x800 mm/mempolicy.c:2146
 wp_page_copy+0x221/0x2070 mm/memory.c:3021
 handle_pte_fault mm/memory.c:4639 [inline]
 __handle_mm_fault mm/memory.c:4756 [inline]
 handle_mm_fault+0x2a3d/0x5950 mm/memory.c:4854
 do_user_addr_fault arch/x86/mm/fault.c:1397 [inline]
 handle_page_fault arch/x86/mm/fault.c:1485 [inline]
 exc_page_fault+0x271/0x740 arch/x86/mm/fault.c:1541
 asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:568
RIP: 0033:0x7ff9c7525980
Code: 89 1c 24 48 83 c4 28 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 84 00 00 00 00 00 41 89 c5 e9 7f fe ff ff 0f 1f 84 00 00 00 00 00 <43> 89 2c 8e e9 a5 fe ff ff 0f 1f 80 00 00 00 00 48 39 c3 74 99 48
RSP: 002b:00007ffe42ef8930 EFLAGS: 00010246
RAX: 0000000081a6d64d RBX: 00007ff9c7689018 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000018151
RBP: 0000000081a6d64d R08: 0000001b33220000 R09: 000000000000164d
R10: 0000000081a6d651 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000001 R14: 00007ff9c767d000 R15: ffffffff81a6df27
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10501 jiffies! g116781 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=0 timer-softirq=100010
rcu: rcu_preempt kthread starved for 10502 jiffies! g116781 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:26840 pid:   15 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5026 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6372
 schedule+0x11b/0x1f0 kernel/sched/core.c:6455
 schedule_timeout+0x1b9/0x300 kernel/time/timer.c:1884
 rcu_gp_fqs_loop+0x2af/0xf70 kernel/rcu/tree.c:1959
 rcu_gp_kthread+0xa4/0x360 kernel/rcu/tree.c:2132
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 22970 Comm: syz-executor.0 Not tainted 5.15.137-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
RIP: 0010:check_kcov_mode kernel/kcov.c:163 [inline]
RIP: 0010:write_comp_data kernel/kcov.c:218 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp4+0x21/0x80 kernel/kcov.c:284
Code: ff c1 4c 89 09 c3 0f 1f 00 4c 8b 04 24 65 48 8b 15 04 4a 82 7e 65 8b 05 05 4a 82 7e a9 00 01 ff 00 74 10 a9 00 01 00 00 74 5b <83> ba 34 16 00 00 00 74 52 8b 82 10 16 00 00 83 f8 03 75 47 48 8b
RSP: 0000:ffffc90000007a80 EFLAGS: 00000006
RAX: 0000000080010101 RBX: 0000000000000000 RCX: ffff888081dbd940
RDX: ffff888081dbd940 RSI: 0000000000000000 RDI: 0000000000000007
RBP: 0000000000000000 R08: ffffffff816f627d R09: ffffc900000079c0
R10: 0000000000000000 R11: dffffc0000000001 R12: dffffc0000000000
R13: ffff8880b9a2a300 R14: ffff88801942e340 R15: dffffc0000000000
FS:  00005555563f6480(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff9c7682934 CR3: 00000000880f1000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 cpu_max_bits_warn include/linux/cpumask.h:108 [inline]
 cpumask_check include/linux/cpumask.h:115 [inline]
 cpumask_test_cpu include/linux/cpumask.h:344 [inline]
 cpu_online include/linux/cpumask.h:895 [inline]
 trace_hrtimer_start include/trace/events/timer.h:199 [inline]
 debug_activate kernel/time/hrtimer.c:476 [inline]
 enqueue_hrtimer+0x4d/0x310 kernel/time/hrtimer.c:1084
 __run_hrtimer kernel/time/hrtimer.c:1702 [inline]
 __hrtimer_run_queues+0x6b6/0xcf0 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x392/0x980 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1085 [inline]
 __sysvec_apic_timer_interrupt+0x139/0x470 arch/x86/kernel/apic/apic.c:1102
 sysvec_apic_timer_interrupt+0x3e/0xb0 arch/x86/kernel/apic/apic.c:1096
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:ffs arch/x86/include/asm/bitops.h:297 [inline]
RIP: 0010:__do_softirq+0x1d2/0x93a kernel/softirq.c:546
Code: 24 78 4c 89 7c 24 70 0f b7 db 48 c7 c7 40 f3 89 8a e8 c2 7b bb ff 65 66 c7 05 f8 8a a3 75 00 00 e8 c3 a1 26 f7 fb 89 5c 24 34 <b8> ff ff ff ff 0f bc 44 24 34 41 89 c4 41 ff c4 0f 85 05 01 00 00
RSP: 0000:ffffc90000007e20 EFLAGS: 00000282
RAX: ce353f5cfe1e9200 RBX: 0000000000000008 RCX: ffffffff8162ea18
RDX: dffffc0000000000 RSI: ffffffff8a8b0be0 RDI: ffffffff8ad87b40
RBP: ffffc90000007f30 R08: dffffc0000000000 R09: fffffbfff1f79a27
R10: 0000000000000000 R11: dffffc0000000001 R12: ffff888081dbd940
R13: 1ffff92000000fe8 R14: dffffc0000000000 R15: 1ffff92000000fd8
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x155/0x240 kernel/softirq.c:637
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:649
 common_interrupt+0xa4/0xc0 arch/x86/kernel/irq.c:240
 </IRQ>
 <TASK>
 asm_common_interrupt+0x22/0x40 arch/x86/include/asm/idtentry.h:629
RIP: 0010:lock_is_held include/linux/lockdep.h:287 [inline]
RIP: 0010:___might_sleep+0xe0/0x6a0 kernel/sched/core.c:9586
Code: 75 1f c6 05 62 34 77 0c 01 48 c7 c7 a0 93 8a 8a be 72 25 00 00 48 c7 c2 e0 9a 8a 8a e8 19 e7 09 00 e8 44 7c c2 08 85 c0 74 46 <48> c7 c7 a0 ef 91 8c be ff ff ff ff e8 df 78 c2 08 85 c0 74 31 e8
RSP: 0000:ffffc900057b7740 EFLAGS: 00000202
RAX: 0000000000000001 RBX: ffff888081dbd96c RCX: ffff888081dbd940
RDX: 0000000000000000 RSI: ffffffff8a8b1e80 RDI: ffffffff8ad87b40
RBP: ffffc900057b7858 R08: dffffc0000000000 R09: fffffbfff1bc71e6
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000000
R13: ffffc900057b7940 R14: dffffc0000000000 R15: 1ffff92000af6ef0
 prepare_alloc_pages+0x1ca/0x5b0 mm/page_alloc.c:5196
 __alloc_pages+0x14f/0x700 mm/page_alloc.c:5410
 alloc_pages_vma+0x39a/0x800 mm/mempolicy.c:2146
 wp_page_copy+0x221/0x2070 mm/memory.c:3021
 handle_pte_fault mm/memory.c:4639 [inline]
 __handle_mm_fault mm/memory.c:4756 [inline]
 handle_mm_fault+0x2a3d/0x5950 mm/memory.c:4854
 do_user_addr_fault arch/x86/mm/fault.c:1397 [inline]
 handle_page_fault arch/x86/mm/fault.c:1485 [inline]
 exc_page_fault+0x271/0x740 arch/x86/mm/fault.c:1541
 asm_exc_page_fault+0x22/0x30 arch/x86/include/asm/idtentry.h:568
RIP: 0033:0x7ff9c7525980
Code: 89 1c 24 48 83 c4 28 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 84 00 00 00 00 00 41 89 c5 e9 7f fe ff ff 0f 1f 84 00 00 00 00 00 <43> 89 2c 8e e9 a5 fe ff ff 0f 1f 80 00 00 00 00 48 39 c3 74 99 48
RSP: 002b:00007ffe42ef8930 EFLAGS: 00010246
RAX: 0000000081a6d64d RBX: 00007ff9c7689018 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000018151
RBP: 0000000081a6d64d R08: 0000001b33220000 R09: 000000000000164d
R10: 0000000081a6d651 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000001 R14: 00007ff9c767d000 R15: ffffffff81a6df27
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/10/31 12:44 linux-5.15.y 12952a23a5da 58499c95 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in __do_softirq
2023/10/12 11:34 linux-5.15.y 02e21884dcf2 1b231e3c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in __do_softirq
* Struck through repros no longer work on HEAD.