syzbot

 sign-in | mailing list | source | docs
🐞 Open [717] 🐞 Fixed [181] 🐞 Invalid [640] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in br_handle_frame (2)

Status: fixed on 2020/01/19 15:05
Reported-by: syzbot+7bfb0d9b58d75975719b@syzkaller.appspotmail.com
Fix commit: 73a6f18d8390 pkt_sched: fq: do not accept silly TCA_FQ_QUANTUM
First crash: 1588d, last: 1588d
Fix bisection: fixed by (bisect log) :
commit 73a6f18d8390abc233212085ba4f06088f9fb075
Author: Eric Dumazet <edumazet@google.com>
Date: Mon Jan 6 14:10:39 2020 +0000

  pkt_sched: fq: do not accept silly TCA_FQ_QUANTUM

  
▶ ▼ Similar bugs (9)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 INFO: rcu detected stall in br_handle_frame (3) 1 1303d 1303d 0/1 auto-closed as invalid on 2021/01/28 07:46
upstream INFO: rcu detected stall in br_handle_frame C done 341 1683d 1688d 13/26 fixed on 2019/10/09 10:54
upstream INFO: rcu detected stall in br_handle_frame (2) net C done 2 1587d 1583d 15/26 fixed on 2020/02/18 14:31
upstream INFO: rcu detected stall in br_handle_frame (3) bridge 1 1012d 1012d 0/26 auto-closed as invalid on 2021/10/15 13:41
linux-4.14 INFO: rcu detected stall in br_handle_frame C done 15 1680d 1692d 1/1 fixed on 2019/12/07 19:24
linux-4.19 INFO: rcu detected stall in br_handle_frame (2) C error 31 447d 1288d 0/1 upstream: reported C repro on 2020/10/14 18:56
linux-4.19 INFO: rcu detected stall in br_handle_frame C done 41 1680d 1692d 1/1 fixed on 2019/12/07 19:18
linux-5.15 INFO: rcu detected stall in br_handle_frame origin:lts-only C error 1 76d 76d 0/3 upstream: reported C repro on 2024/02/08 13:52
upstream INFO: rcu detected stall in br_handle_frame (4) kernel 1 851d 851d 0/26 closed as invalid on 2022/02/08 10:10

Sample crash report:
INFO: rcu_preempt self-detected stall on CPU
	1-...: (1 GPs behind) idle=efa/140000000000002/0 softirq=12113/12122 fqs=0 
	 (t=10500 jiffies g=1252 c=1251 q=127)
rcu_preempt kthread starved for 10500 jiffies! g1252 c1251 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=0
rcu_preempt     I29776     8      2 0x80000000
Call Trace:
 context_switch kernel/sched/core.c:2808 [inline]
 __schedule+0x7b8/0x1cd0 kernel/sched/core.c:3384
 schedule+0x92/0x1c0 kernel/sched/core.c:3428
 schedule_timeout+0x43e/0xe10 kernel/time/timer.c:1746
 rcu_gp_kthread+0xbf4/0x1ec0 kernel/rcu/tree.c:2255
 kthread+0x319/0x430 kernel/kthread.c:232
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
NMI backtrace for cpu 1
CPU: 1 PID: 7528 Comm: syz-executor862 Not tainted 4.14.159-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x142/0x197 lib/dump_stack.c:58
 nmi_cpu_backtrace.cold+0x57/0x94 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x141/0x189 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
 rcu_dump_cpu_stacks+0x186/0x1d2 kernel/rcu/tree.c:1396
 print_cpu_stall kernel/rcu/tree.c:1542 [inline]
 check_cpu_stall kernel/rcu/tree.c:1610 [inline]
 __rcu_pending kernel/rcu/tree.c:3390 [inline]
 rcu_pending kernel/rcu/tree.c:3452 [inline]
 rcu_check_callbacks.cold+0x43d/0xd0a kernel/rcu/tree.c:2792
 update_process_times+0x31/0x70 kernel/time/timer.c:1590
 tick_sched_handle+0x85/0x160 kernel/time/tick-sched.c:161
 tick_sched_timer+0x43/0x130 kernel/time/tick-sched.c:1219
 __run_hrtimer kernel/time/hrtimer.c:1220 [inline]
 __hrtimer_run_queues+0x270/0xbc0 kernel/time/hrtimer.c:1284
 hrtimer_interrupt+0x1d8/0x5d0 kernel/time/hrtimer.c:1318
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x11c/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
RIP: 0010:fq_flow_add_tail net/sched/sch_fq.c:138 [inline]
RIP: 0010:fq_dequeue+0x7a6/0x1360 net/sched/sch_fq.c:489
RSP: 0018:ffff8880aed06d40 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff10
RAX: ffff8880a6590240 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: 1ffff11010eb9b0a RSI: 0000000000000000 RDI: ffff8880851cbcb8
RBP: ffff8880aed06dc0 R08: 00000043791fc26e R09: ffff88821fff7048
R10: ffff88821fff7050 R11: 0000000000000001 R12: ffff8880851cbc78
R13: ffff8880875cd850 R14: 0000000000000000 R15: ffff8880875cd680
 dequeue_skb net/sched/sch_generic.c:148 [inline]
 qdisc_restart net/sched/sch_generic.c:241 [inline]
 __qdisc_run+0x2b8/0xe00 net/sched/sch_generic.c:257
 __dev_xmit_skb net/core/dev.c:3235 [inline]
 __dev_queue_xmit+0x1571/0x25e0 net/core/dev.c:3493
 dev_queue_xmit+0x18/0x20 net/core/dev.c:3558
 br_dev_queue_push_xmit+0x367/0x530 net/bridge/br_forward.c:55
 br_nf_dev_queue_xmit+0x309/0x1440 net/bridge/br_netfilter_hooks.c:776
 NF_HOOK include/linux/netfilter.h:250 [inline]
 NF_HOOK include/linux/netfilter.h:244 [inline]
 br_nf_post_routing+0xb80/0xf00 net/bridge/br_netfilter_hooks.c:822
 nf_hook_entry_hookfn include/linux/netfilter.h:108 [inline]
 nf_hook_slow+0xaf/0x1b0 net/netfilter/core.c:467
 nf_hook include/linux/netfilter.h:205 [inline]
 NF_HOOK include/linux/netfilter.h:248 [inline]
 br_forward_finish+0x1b7/0x320 net/bridge/br_forward.c:67
 br_nf_hook_thresh+0x25b/0x2e0 net/bridge/br_netfilter_hooks.c:1005
 br_nf_forward_finish+0x264/0x640 net/bridge/br_netfilter_hooks.c:550
 NF_HOOK include/linux/netfilter.h:250 [inline]
 NF_HOOK include/linux/netfilter.h:244 [inline]
 br_nf_forward_ip net/bridge/br_netfilter_hooks.c:617 [inline]
 br_nf_forward_ip+0x5fc/0x1190 net/bridge/br_netfilter_hooks.c:561
 nf_hook_entry_hookfn include/linux/netfilter.h:108 [inline]
 nf_hook_slow+0xaf/0x1b0 net/netfilter/core.c:467
 nf_hook include/linux/netfilter.h:205 [inline]
 NF_HOOK include/linux/netfilter.h:248 [inline]
 __br_forward+0x312/0x9c0 net/bridge/br_forward.c:111
 deliver_clone+0x61/0xc0 net/bridge/br_forward.c:127
 br_flood+0x43c/0x530 net/bridge/br_forward.c:222
 br_handle_frame_finish+0xaf0/0x1800 net/bridge/br_input.c:210
 br_nf_hook_thresh+0x25b/0x2e0 net/bridge/br_netfilter_hooks.c:1005
 br_nf_pre_routing_finish_ipv6+0x621/0xc50 net/bridge/br_netfilter_ipv6.c:210
 NF_HOOK include/linux/netfilter.h:250 [inline]
 br_nf_pre_routing_ipv6+0x419/0x7a0 net/bridge/br_netfilter_ipv6.c:240
 br_nf_pre_routing+0xdd0/0x12c5 net/bridge/br_netfilter_hooks.c:491
 nf_hook_entry_hookfn include/linux/netfilter.h:108 [inline]
 nf_hook_slow+0xaf/0x1b0 net/netfilter/core.c:467
 nf_hook include/linux/netfilter.h:205 [inline]
 NF_HOOK include/linux/netfilter.h:248 [inline]
 br_handle_frame+0x80c/0x1110 net/bridge/br_input.c:348
 __netif_receive_skb_core+0x78d/0x2cb0 net/core/dev.c:4431
 __netif_receive_skb+0x2c/0x1b0 net/core/dev.c:4515
 process_backlog+0x21f/0x730 net/core/dev.c:5197
 napi_poll net/core/dev.c:5598 [inline]
 net_rx_action+0x490/0xf80 net/core/dev.c:5664
 __do_softirq+0x244/0x9a0 kernel/softirq.c:288
 invoke_softirq kernel/softirq.c:368 [inline]
 irq_exit+0x160/0x1b0 kernel/softirq.c:409
 exiting_irq arch/x86/include/asm/apic.h:648 [inline]
 smp_apic_timer_interrupt+0x146/0x5e0 arch/x86/kernel/apic/apic.c:1102
 apic_timer_interrupt+0x96/0xa0 arch/x86/entry/entry_64.S:792
 </IRQ>
RIP: 0010:lock_acquire+0x1f6/0x430 kernel/locking/lockdep.c:3998
RSP: 0018:ffff8880872ef988 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff10
RAX: 1ffffffff0fe2d29 RBX: ffffea0002458e00 RCX: 0000000090259772
RDX: dffffc0000000000 RSI: ffff8880a6590ae8 RDI: 0000000000000282
RBP: ffff8880872ef9a0 R08: 0000000000000000 R09: ffff8880a6590b08
R10: ffff8880a6590ae8 R11: ffff8880a6590240 R12: ffffea0002458e00
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002
 rcu_lock_acquire include/linux/rcupdate.h:242 [inline]
 rcu_read_lock include/linux/rcupdate.h:629 [inline]
 lock_page_memcg+0x39/0x220 mm/memcontrol.c:1669
 page_remove_file_rmap mm/rmap.c:1211 [inline]
 page_remove_rmap+0x19f/0x940 mm/rmap.c:1296
 zap_pte_range mm/memory.c:1342 [inline]
 zap_pmd_range mm/memory.c:1444 [inline]
 zap_pud_range mm/memory.c:1473 [inline]
 zap_p4d_range mm/memory.c:1494 [inline]
 unmap_page_range+0xabd/0x19f0 mm/memory.c:1515
 unmap_single_vma+0x15d/0x2c0 mm/memory.c:1560
 unmap_vmas+0xac/0x170 mm/memory.c:1590
 exit_mmap+0x285/0x4e0 mm/mmap.c:3062
 __mmput kernel/fork.c:930 [inline]
 mmput+0x114/0x440 kernel/fork.c:951
 exit_mm kernel/exit.c:545 [inline]
 do_exit+0x6fc/0x2c80 kernel/exit.c:841
 do_group_exit+0x111/0x330 kernel/exit.c:951
 SYSC_exit_group kernel/exit.c:962 [inline]
 SyS_exit_group+0x1d/0x20 kernel/exit.c:960
 do_syscall_64+0x1e8/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x441488
RSP: 002b:00007fff59704e68 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000441488
RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
RBP: 00000000004c7490 R08: 00000000000000e7 R09: ffffffffffffffd0
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 00000000006daa40 R14: 0000000000000000 R15: 0000000000000000

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/12/20 02:57 linux-4.14.y bfb9e5c03076 36650b4b .config console log report syz C ci2-linux-4-14
* Struck through repros no longer work on HEAD.