syzbot

 sign-in | mailing list | source | docs
🐞 Open [1030] ≡ Subsystems 🐞 Fixed [5264] 🐞 Invalid [12591] ⬇ Missing Backports [85] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

BUG: soft lockup in fq_pie_timer (2)

Status: closed as invalid on 2023/05/26 05:46
Subsystems: net
[Documentation on labels]
First crash: 444d, last: 378d
Cause bisection: failed (error log, bisect log)
  
Fix bisection: fixed by (bisect log) :
commit e2c7fa724626e4bde70e753cdeb7827d0d225364
Author: Philippe Schenker <philippe.schenker@toradex.com>
Date: Tue Mar 14 10:24:03 2023 +0000

  arm64: dts: colibri-imx8x: Set thermal thresholds

  
▶ ▼ Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 BUG: soft lockup in fq_pie_timer (2) origin:upstream C inconclusive 5 18d 252d 0/3 upstream: reported C repro on 2023/08/29 01:49
upstream BUG: soft lockup in fq_pie_timer (4) net syz 5 238d 288d 23/26 fixed on 2023/10/12 12:48
upstream INFO: rcu detected stall in fq_pie_timer net C error 35 831d 1049d 20/26 fixed on 2022/03/08 16:11
linux-5.15 INFO: rcu detected stall in fq_pie_timer origin:lts-only C done 23 260d 350d 0/3 upstream: reported C repro on 2023/05/22 23:26
linux-6.1 BUG: soft lockup in fq_pie_timer C done 7 332d 340d 3/3 fixed on 2023/07/22 07:09
upstream BUG: soft lockup in fq_pie_timer net C error error 10 573d 611d 0/26 closed as invalid on 2022/11/18 11:06
upstream INFO: rcu detected stall in fq_pie_timer (2) net C done 1 89d 132d 0/26 upstream: reported C repro on 2023/12/27 13:54

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (1 GPs behind) idle=92e4/1/0x4000000000000000 softirq=7532/7533 fqs=0
rcu: 	(t=10501 jiffies g=6977 q=1216 ncpus=2)
rcu: rcu_preempt kthread starved for 10502 jiffies! g6977 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28680 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5307 [inline]
 __schedule+0xc91/0x5770 kernel/sched/core.c:6625
 schedule+0xde/0x1a0 kernel/sched/core.c:6701
 schedule_timeout+0x14e/0x2b0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x190/0x910 kernel/rcu/tree.c:1608
 rcu_gp_kthread+0x23a/0x360 kernel/rcu/tree.c:1807
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5143 Comm: kworker/0:5 Not tainted 6.3.0-rc4-syzkaller-01128-gd74aab2ca198 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
Workqueue: ipv6_addrconf addrconf_dad_work
RIP: 0010:__sanitizer_cov_trace_const_cmp4+0x0/0x20 kernel/kcov.c:303
Code: d6 fe ff ff 66 0f 1f 44 00 00 f3 0f 1e fa 48 8b 0c 24 0f b7 d6 0f b7 f7 bf 03 00 00 00 e9 b8 fe ff ff 0f 1f 84 00 00 00 00 00 <f3> 0f 1e fa 48 8b 0c 24 89 f2 89 fe bf 05 00 00 00 e9 9a fe ff ff
RSP: 0018:ffffc90000007c68 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff88806ce2a100 RCX: 0000000000000100
RDX: ffff888028001d40 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000007 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: dffffc0000000000
R13: ffff888077738300 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd62d9ed88 CR3: 000000002b850000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 pie_calculate_probability+0x103/0x7c0 net/sched/sch_pie.c:324
 fq_pie_timer+0x174/0x2a0 net/sched/sch_fq_pie.c:380
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 do_softirq.part.0+0xde/0x130 kernel/softirq.c:472
 </IRQ>
 <TASK>
 do_softirq kernel/softirq.c:464 [inline]
 __local_bh_enable_ip+0x106/0x130 kernel/softirq.c:396
 local_bh_enable include/linux/bottom_half.h:33 [inline]
 rcu_read_unlock_bh include/linux/rcupdate.h:843 [inline]
 __dev_queue_xmit+0x1e3d/0x3b30 net/core/dev.c:4273
 dev_queue_xmit include/linux/netdevice.h:3079 [inline]
 neigh_hh_output include/net/neighbour.h:528 [inline]
 neigh_output include/net/neighbour.h:542 [inline]
 ip6_finish_output2+0xfc5/0x1560 net/ipv6/ip6_output.c:134
 __ip6_finish_output net/ipv6/ip6_output.c:195 [inline]
 ip6_finish_output+0x69a/0x1170 net/ipv6/ip6_output.c:206
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x1f1/0x540 net/ipv6/ip6_output.c:227
 dst_output include/net/dst.h:458 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 NF_HOOK include/linux/netfilter.h:296 [inline]
 mld_sendpack+0xa09/0xed0 net/ipv6/mcast.c:1820
 mld_send_initial_cr.part.0+0x1a6/0x260 net/ipv6/mcast.c:2239
 mld_send_initial_cr net/ipv6/mcast.c:1232 [inline]
 ipv6_mc_dad_complete+0x1d4/0x680 net/ipv6/mcast.c:2247
 addrconf_dad_completed+0xa01/0xe00 net/ipv6/addrconf.c:4234
 addrconf_dad_work+0x75d/0x1390 net/ipv6/addrconf.c:4162
 process_one_work+0x991/0x15c0 kernel/workqueue.c:2390
 worker_thread+0x669/0x1090 kernel/workqueue.c:2537
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.779 msecs
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5143 Comm: kworker/0:5 Not tainted 6.3.0-rc4-syzkaller-01128-gd74aab2ca198 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
Workqueue: ipv6_addrconf addrconf_dad_work
RIP: 0010:arch_static_branch arch/x86/include/asm/jump_label.h:27 [inline]
RIP: 0010:static_key_false include/linux/jump_label.h:207 [inline]
RIP: 0010:native_write_msr arch/x86/include/asm/msr.h:147 [inline]
RIP: 0010:wrmsr arch/x86/include/asm/msr.h:254 [inline]
RIP: 0010:native_apic_msr_write arch/x86/include/asm/apic.h:205 [inline]
RIP: 0010:native_apic_msr_write+0x29/0x40 arch/x86/include/asm/apic.h:199
Code: 90 f3 0f 1e fa 89 f8 83 e0 ef 83 f8 20 74 0b 8d 87 30 ff ff ff 83 e0 ef 75 01 c3 c1 ef 04 31 d2 89 f0 8d 8f 00 08 00 00 0f 30 <66> 90 c3 89 f6 31 d2 89 cf e9 f9 40 08 03 66 0f 1f 84 00 00 00 00
RSP: 0018:ffffc90000007a60 EFLAGS: 00000046
RAX: 0000000000097e01 RBX: ffffffff8c10cbc0 RCX: 0000000000000838
RDX: 0000000000000000 RSI: 0000000000097e01 RDI: 0000000000000038
RBP: ffff8880b9828240 R08: 0000000000000005 R09: 000000000000003f
R10: 0000000000000020 R11: 0000000000000001 R12: 0000000000097e01
R13: 0000000000000020 R14: 0000000000000000 R15: ffff8880b982b800
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffd62d9ed88 CR3: 000000002b850000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 apic_write arch/x86/include/asm/apic.h:393 [inline]
 lapic_next_event+0x51/0x80 arch/x86/kernel/apic/apic.c:479
 clockevents_program_event+0x258/0x370 kernel/time/clockevents.c:334
 tick_program_event+0xb0/0x140 kernel/time/tick-oneshot.c:44
 hrtimer_interrupt+0x372/0x7b0 kernel/time/hrtimer.c:1824
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1096 [inline]
 __sysvec_apic_timer_interrupt+0x14a/0x430 arch/x86/kernel/apic/apic.c:1113
 sysvec_apic_timer_interrupt+0x44/0xc0 arch/x86/kernel/apic/apic.c:1107
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:get_current arch/x86/include/asm/current.h:41 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x17/0x70 kernel/kcov.c:206
Code: ff ff ff 31 c0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa 65 8b 05 2d e0 80 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 <65> 48 8b 14 25 80 b8 03 00 a9 00 01 ff 00 74 0e 85 c9 74 35 8b 82
RSP: 0018:ffffc90000007c68 EFLAGS: 00000206
RAX: 0000000000000102 RBX: ffff88806d652750 RCX: 0000000000000100
RDX: ffff888028001d40 RSI: ffffffff883811e9 RDI: 0000000000000006
RBP: 000000000001c9c3 R08: 0000000000000006 R09: 000000000001c9c3
R10: 0000000000000000 R11: 0000000000000001 R12: fffffff0a3da8872
R13: ffff888075503300 R14: 0000000000000000 R15: 0000000000000001
 pie_calculate_probability+0x4e9/0x7c0 net/sched/sch_pie.c:409
 fq_pie_timer+0x174/0x2a0 net/sched/sch_fq_pie.c:380
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 do_softirq.part.0+0xde/0x130 kernel/softirq.c:472
 </IRQ>
 <TASK>
 do_softirq kernel/softirq.c:464 [inline]
 __local_bh_enable_ip+0x106/0x130 kernel/softirq.c:396
 local_bh_enable include/linux/bottom_half.h:33 [inline]
 rcu_read_unlock_bh include/linux/rcupdate.h:843 [inline]
 __dev_queue_xmit+0x1e3d/0x3b30 net/core/dev.c:4273
 dev_queue_xmit include/linux/netdevice.h:3079 [inline]
 neigh_hh_output include/net/neighbour.h:528 [inline]
 neigh_output include/net/neighbour.h:542 [inline]
 ip6_finish_output2+0xfc5/0x1560 net/ipv6/ip6_output.c:134
 __ip6_finish_output net/ipv6/ip6_output.c:195 [inline]
 ip6_finish_output+0x69a/0x1170 net/ipv6/ip6_output.c:206
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x1f1/0x540 net/ipv6/ip6_output.c:227
 dst_output include/net/dst.h:458 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 NF_HOOK include/linux/netfilter.h:296 [inline]
 mld_sendpack+0xa09/0xed0 net/ipv6/mcast.c:1820
 mld_send_initial_cr.part.0+0x1a6/0x260 net/ipv6/mcast.c:2239
 mld_send_initial_cr net/ipv6/mcast.c:1232 [inline]
 ipv6_mc_dad_complete+0x1d4/0x680 net/ipv6/mcast.c:2247
 addrconf_dad_completed+0xa01/0xe00 net/ipv6/addrconf.c:4234
 addrconf_dad_work+0x75d/0x1390 net/ipv6/addrconf.c:4162
 process_one_work+0x991/0x15c0 kernel/workqueue.c:2390
 worker_thread+0x669/0x1090 kernel/workqueue.c:2537
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 2.025 msecs
CPU: 1 PID: 21 Comm: ksoftirqd/1 Not tainted 6.3.0-rc4-syzkaller-01128-gd74aab2ca198 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x31/0x70 kernel/kcov.c:207
Code: 2d e0 80 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b 14 25 80 b8 03 00 a9 00 01 ff 00 74 0e 85 c9 74 35 8b 82 74 15 00 00 <85> c0 74 2b 8b 82 50 15 00 00 83 f8 02 75 20 48 8b 8a 58 15 00 00
RSP: 0018:ffffc900001b7b38 EFLAGS: 00000206
RAX: 0000000000000000 RBX: ffff88806d1297d0 RCX: 0000000000000100
RDX: ffff888017289d40 RSI: ffffffff88381086 RDI: 0000000000000001
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000001 R12: fffffff0a3da8872
R13: ffff88801ddcf300 R14: 0000000000000000 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fce7f409370 CR3: 000000000c571000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 pie_calculate_probability+0x386/0x7c0 net/sched/sch_pie.c:397
 fq_pie_timer+0x174/0x2a0 net/sched/sch_fq_pie.c:380
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 run_ksoftirqd kernel/softirq.c:934 [inline]
 run_ksoftirqd+0x31/0x60 kernel/softirq.c:926
 smpboot_thread_fn+0x659/0x9e0 kernel/smpboot.c:164
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/04/02 04:11 net-next d74aab2ca198 f325deb0 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in fq_pie_timer
2023/04/25 20:45 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 14f8db1c0f9a 65320f8e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in fq_pie_timer
2023/02/18 14:12 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 2d3827b3f393 d02e9a70 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in fq_pie_timer
* Struck through repros no longer work on HEAD.