syzbot


INFO: rcu detected stall in vhci_release

Status: upstream: reported syz repro on 2024/09/28 11:56
Subsystems: mm
[Documentation on labels]
Reported-by: syzbot+46c3d1706c2d2688baba@syzkaller.appspotmail.com
First crash: 393d, last: 54d
Cause bisection: introduced by (bisect log) :
commit d15121be7485655129101f3960ae6add40204463
Author: Paolo Abeni <pabeni@redhat.com>
Date: Mon May 8 06:17:44 2023 +0000

  Revert "softirq: Let ksoftirqd do its job"

Crash: no output from test machine (log)
Repro: syz .config
  
Fix bisection: fixed by (bisect log) :
commit 6d71a9c6160479899ee744d2c6d6602a191deb1f
Author: Peter Zijlstra <peterz@infradead.org>
Date: Thu Jan 9 10:59:59 2025 +0000

  sched/fair: Fix EEVDF entity placement bug causing scheduling lag

  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [mm?] [bluetooth?] INFO: rcu detected stall in vhci_release 0 (3) 2025/06/04 13:49
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in vhci_release 1 289d 289d 0/3 auto-obsoleted due to no activity on 2024/12/19 05:19
linux-4.14 BUG: soft lockup in vhci_release 1 1532d 1532d 0/1 auto-closed as invalid on 2021/08/14 12:26
Last patch testing requests (7)
Created Duration User Patch Repo Result
2025/05/29 00:54 23m retest repro net-next report log
2025/04/23 17:36 22m retest repro upstream log
2025/03/19 22:57 1h28m retest repro net-next report log
2025/03/19 22:09 2h02m retest repro net-next report log
2025/01/08 23:08 19m retest repro upstream report log
2025/01/08 21:46 19m retest repro net-next report log
2024/10/28 02:20 24m retest repro upstream report log
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2025/06/04 01:37 12h10m bisect fix upstream OK (1) job log
2025/02/12 14:17 2h24m bisect fix upstream OK (0) job log log

Sample crash report:
bridge0: received packet on veth0_to_bridge with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P6116/1:b..l P5961/1:b..l
rcu: 	(detected by 0, t=10503 jiffies, g=11925, q=843 ncpus=2)
task:syz-executor    state:R  running task     stack:20096 pid:5961  tgid:5961  ppid:1      flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5369 [inline]
 __schedule+0x1850/0x4c30 kernel/sched/core.c:6756
 preempt_schedule_irq+0xfb/0x1c0 kernel/sched/core.c:7078
 irqentry_exit+0x5e/0x90 kernel/entry/common.c:354
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:lock_is_held_type+0x13b/0x190
Code: 75 44 48 c7 04 24 00 00 00 00 9c 8f 04 24 f7 04 24 00 02 00 00 75 4c 41 f7 c4 00 02 00 00 74 01 fb 65 48 8b 04 25 28 00 00 00 <48> 3b 44 24 08 75 42 89 d8 48 83 c4 10 5b 41 5c 41 5d 41 5e 41 5f
RSP: 0018:ffffc900040571d0 EFLAGS: 00000206
RAX: 78ee25ec5a88a000 RBX: 0000000000000001 RCX: 0000000080000000
RDX: 0000000000000000 RSI: ffffffff8c0aa960 RDI: ffffffff8c5faee0
RBP: 0000000000000003 R08: ffffffff82115a4e R09: 1ffffffff2857910
R10: dffffc0000000000 R11: fffffbfff2857911 R12: 0000000000000246
R13: ffff888025fe0000 R14: 00000000ffffffff R15: ffffffff8e937ae0
 lookup_page_ext mm/page_ext.c:254 [inline]
 page_ext_get+0x192/0x2a0 mm/page_ext.c:526
 __page_table_check_zero+0xb1/0x350 mm/page_table_check.c:148
 page_table_check_free include/linux/page_table_check.h:41 [inline]
 free_pages_prepare mm/page_alloc.c:1128 [inline]
 free_unref_page+0xd53/0x1010 mm/page_alloc.c:2657
 discard_slab mm/slub.c:2688 [inline]
 __put_partials+0x160/0x1c0 mm/slub.c:3157
 put_cpu_partial+0x17c/0x250 mm/slub.c:3232
 __slab_free+0x290/0x380 mm/slub.c:4483
 qlink_free mm/kasan/quarantine.c:163 [inline]
 qlist_free_all+0x9a/0x140 mm/kasan/quarantine.c:179
 kasan_quarantine_reduce+0x14f/0x170 mm/kasan/quarantine.c:286
 __kasan_kmalloc+0x23/0xb0 mm/kasan/common.c:385
 kasan_kmalloc include/linux/kasan.h:260 [inline]
 __do_kmalloc_node mm/slub.c:4298 [inline]
 __kmalloc_noprof+0x285/0x4c0 mm/slub.c:4310
 kmalloc_noprof include/linux/slab.h:905 [inline]
 kzalloc_noprof include/linux/slab.h:1037 [inline]
 kobject_get_path+0xb8/0x230 lib/kobject.c:161
 kobject_uevent_env+0x2a5/0x8e0 lib/kobject_uevent.c:545
 device_del+0x7db/0x9b0 drivers/base/core.c:3872
 device_unregister+0x20/0xc0 drivers/base/core.c:3895
 hci_conn_cleanup net/bluetooth/hci_conn.c:174 [inline]
 hci_conn_del+0x8c4/0xc40 net/bluetooth/hci_conn.c:1164
 hci_conn_hash_flush+0x258/0x350 net/bluetooth/hci_conn.c:2699
 hci_dev_close_sync+0xa42/0x11c0 net/bluetooth/hci_sync.c:5212
 hci_dev_do_close net/bluetooth/hci_core.c:482 [inline]
 hci_unregister_dev+0x20b/0x510 net/bluetooth/hci_core.c:2697
 vhci_release+0x80/0xd0 drivers/bluetooth/hci_vhci.c:664
 __fput+0x23c/0xa50 fs/file_table.c:450
 task_work_run+0x24f/0x310 kernel/task_work.c:239
 exit_task_work include/linux/task_work.h:43 [inline]
 do_exit+0xa2f/0x28e0 kernel/exit.c:938
 do_group_exit+0x207/0x2c0 kernel/exit.c:1087
 get_signal+0x16b2/0x1750 kernel/signal.c:3017
 arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0xce/0x340 kernel/entry/common.c:218
 do_syscall_64+0x100/0x230 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f4482587c1a
RSP: 002b:00007ffd88e4ef48 EFLAGS: 00000206 ORIG_RAX: 0000000000000036
RAX: 0000000000000000 RBX: 00007ffd88e4efd0 RCX: 00007f4482587c1a
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 0000000000000003 R08: 00000000000002d8 R09: 00007ffd88e4f387
R10: 00007f4482748ea0 R11: 0000000000000206 R12: 00007f4482748e40
R13: 00007ffd88e4ef6c R14: 0000000000000000 R15: 00007f448274b000
 </TASK>
task:sed             state:R  running task     stack:24736 pid:6116  tgid:6116  ppid:6115   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5369 [inline]
 __schedule+0x1850/0x4c30 kernel/sched/core.c:6756
 preempt_schedule_irq+0xfb/0x1c0 kernel/sched/core.c:7078
 irqentry_exit+0x5e/0x90 kernel/entry/common.c:354
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:arch_stack_walk+0x110/0x150 arch/x86/kernel/stacktrace.c:27
Code: bd 70 ff ff ff 00 74 37 48 8d 9d 70 ff ff ff 48 89 df e8 63 b1 09 00 48 85 c0 74 23 4c 89 f7 48 89 c6 4d 89 eb 41 ff d3 66 90 <84> c0 74 11 48 89 df e8 94 b2 09 00 83 bd 70 ff ff ff 00 75 d0 65
RSP: 0000:ffffc900031a6ce0 EFLAGS: 00000282
RAX: 0000000000000001 RBX: ffffc900031a6ce0 RCX: ffffffff917a3000
RDX: dffffc0000000000 RSI: ffffffff81fadacc RDI: ffffc900031a6dcc
RBP: ffffc900031a6d70 R08: 0000000000000002 R09: ffffc900031a6dd0
R10: ffffc900031a6d30 R11: ffffffff818b39e0 R12: ffff88802b7fda00
R13: ffffffff818b39e0 R14: ffffc900031a6dc0 R15: 0000000000000000
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 save_stack+0xfb/0x1f0 mm/page_owner.c:156
 __reset_page_owner+0x76/0x430 mm/page_owner.c:297
 reset_page_owner include/linux/page_owner.h:25 [inline]
 free_pages_prepare mm/page_alloc.c:1127 [inline]
 free_unref_page+0xd3f/0x1010 mm/page_alloc.c:2657
 discard_slab mm/slub.c:2688 [inline]
 __put_partials+0x160/0x1c0 mm/slub.c:3157
 put_cpu_partial+0x17c/0x250 mm/slub.c:3232
 __slab_free+0x290/0x380 mm/slub.c:4483
 qlink_free mm/kasan/quarantine.c:163 [inline]
 qlist_free_all+0x9a/0x140 mm/kasan/quarantine.c:179
 kasan_quarantine_reduce+0x14f/0x170 mm/kasan/quarantine.c:286
 __kasan_slab_alloc+0x23/0x80 mm/kasan/common.c:329
 kasan_slab_alloc include/linux/kasan.h:250 [inline]
 slab_post_alloc_hook mm/slub.c:4119 [inline]
 slab_alloc_node mm/slub.c:4168 [inline]
 kmem_cache_alloc_noprof+0x1d9/0x380 mm/slub.c:4175
 vm_area_dup+0x27/0x290 kernel/fork.c:487
 __split_vma+0x1cb/0xc50 mm/vma.c:434
 vms_gather_munmap_vmas+0x2e6/0x1600 mm/vma.c:1267
 do_vmi_align_munmap+0x3ff/0x6f0 mm/vma.c:1435
 do_vmi_munmap+0x24e/0x2d0 mm/vma.c:1492
 __vm_munmap+0x24c/0x480 mm/mmap.c:1368
 elf_map fs/binfmt_elf.c:389 [inline]
 elf_load+0x2d6/0x700 fs/binfmt_elf.c:414
 load_elf_binary+0x100c/0x2770 fs/binfmt_elf.c:1173
 search_binary_handler fs/exec.c:1748 [inline]
 exec_binprm fs/exec.c:1790 [inline]
 bprm_execve+0xaf5/0x17a0 fs/exec.c:1842
 do_execveat_common+0x55f/0x6f0 fs/exec.c:1949
 do_execve fs/exec.c:2023 [inline]
 __do_sys_execve fs/exec.c:2099 [inline]
 __se_sys_execve fs/exec.c:2094 [inline]
 __x64_sys_execve+0x92/0xb0 fs/exec.c:2094
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7febea9f1ef7
RSP: 002b:00007ffc50e62758 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
RAX: ffffffffffffffda RBX: 0000559883babec0 RCX: 00007febea9f1ef7
RDX: 0000559883babf08 RSI: 0000559883babec0 RDI: 0000559883babf98
RBP: 0000559883babf98 R08: 0000559883babf9d R09: 00007ffc50e67ebb
R10: 00007febeabf0088 R11: 0000000000000246 R12: 0000559883babf08
R13: 00007febeab9f904 R14: 0000559883babf08 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10501 jiffies! g11925 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25232 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5369 [inline]
 __schedule+0x1850/0x4c30 kernel/sched/core.c:6756
 __schedule_loop kernel/sched/core.c:6833 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6848
 schedule_timeout+0x15a/0x290 kernel/time/sleep_timeout.c:99
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2045
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2247
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 UID: 0 PID: 16 Comm: ksoftirqd/0 Not tainted 6.13.0-rc3-syzkaller-00762-g9268abe611b0 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:deref_stack_reg+0x18f/0x210 arch/x86/kernel/unwind_orc.c:406
Code: 31 c0 4c 39 e1 76 33 4c 39 f9 77 2e 4c 89 c7 48 89 d3 e8 f4 07 00 00 49 89 c6 48 8b 6c 24 18 48 89 e8 48 c1 e8 03 80 3c 18 00 <74> 08 48 89 ef e8 27 05 ba 00 4c 89 75 00 b0 01 48 83 c4 20 5b 41
RSP: 0018:ffffc900001567d8 EFLAGS: 00000246
RAX: 1ffff9200002ad28 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: ffffc90000150000 RDI: ffffc90000156d70
RBP: ffffc90000156940 R08: ffffc90000156d70 R09: 0000000000000000
R10: ffffc90000156950 R11: fffff5200002ad2c R12: 1ffff9200002ad21
R13: 1ffff9200002ad22 R14: 0000000000000001 R15: ffffc90000158000
FS:  0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffe348368d8 CR3: 000000000e736000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 unwind_next_frame+0x1799/0x22d0
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:319 [inline]
 __kasan_slab_alloc+0x66/0x80 mm/kasan/common.c:345
 kasan_slab_alloc include/linux/kasan.h:250 [inline]
 slab_post_alloc_hook mm/slub.c:4119 [inline]
 slab_alloc_node mm/slub.c:4168 [inline]
 kmem_cache_alloc_noprof+0x1d9/0x380 mm/slub.c:4175
 skb_clone+0x20c/0x390 net/core/skbuff.c:2084
 deliver_clone net/bridge/br_forward.c:125 [inline]
 maybe_deliver+0x96/0x150 net/bridge/br_forward.c:190
 br_flood+0x2e4/0x680 net/bridge/br_forward.c:237
 br_handle_frame_finish+0x18d2/0x2000 net/bridge/br_input.c:220
 br_nf_hook_thresh+0x472/0x590
 br_nf_pre_routing_finish_ipv6+0xaa0/0xdd0
 NF_HOOK include/linux/netfilter.h:314 [inline]
 br_nf_pre_routing_ipv6+0x379/0x770 net/bridge/br_netfilter_ipv6.c:184
 nf_hook_entry_hookfn include/linux/netfilter.h:154 [inline]
 nf_hook_bridge_pre net/bridge/br_input.c:282 [inline]
 br_handle_frame+0x9f3/0x1530 net/bridge/br_input.c:433
 __netif_receive_skb_core+0x14eb/0x4690 net/core/dev.c:5566
 __netif_receive_skb_one_core net/core/dev.c:5670 [inline]
 __netif_receive_skb+0x12f/0x650 net/core/dev.c:5785
 process_backlog+0x662/0x15b0 net/core/dev.c:6117
 __napi_poll+0xcb/0x490 net/core/dev.c:6883
 napi_poll net/core/dev.c:6952 [inline]
 net_rx_action+0x89b/0x1240 net/core/dev.c:7074
 handle_softirqs+0x2d4/0x9b0 kernel/softirq.c:561
 run_ksoftirqd+0xca/0x130 kernel/softirq.c:950
 smpboot_thread_fn+0x544/0xa30 kernel/smpboot.c:164
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
bridge0: received packet on veth0_to_bridge with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on veth0_to_bridge with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:1b, vlan:0)
bridge0: received packet on bridge_slave_0 with own address as source address (addr:aa:aa:aa:aa:aa:0c, vlan:0)

Crashes (12):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/12/25 21:05 net-next 9268abe611b0 444551c4 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in vhci_release
2025/05/03 16:05 upstream 95d3481af6dc b0714e37 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in vhci_release
2025/03/21 22:50 upstream d07de43e3f05 62330552 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in vhci_release
2024/12/22 14:44 upstream 48f506ad0b68 d7f584ee .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in vhci_release
2024/10/01 05:24 upstream e32cde8d2bd7 bbd4e0a4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in vhci_release
2024/09/24 11:45 upstream abf2050f51fd 89298aad .config console log report syz / log [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in vhci_release
2024/05/28 22:57 upstream e0cce98fe279 34889ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in vhci_release
2025/02/24 05:18 upstream d082ecbc71e9 d34966d1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: rcu detected stall in vhci_release
2024/10/14 02:07 upstream ba01565ced22 084d8178 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: rcu detected stall in vhci_release
2024/07/04 22:42 upstream 795c58e4c7fc dc6bbff0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: rcu detected stall in vhci_release
2025/01/13 14:14 net 76201b597976 6dbc6a9b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in vhci_release
2025/04/09 10:57 linux-next 46086739de22 b133e63a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in vhci_release
* Struck through repros no longer work on HEAD.