syzbot


INFO: rcu detected stall in rt_sigreturn

Status: upstream: reported C repro on 2024/04/26 07:05
Subsystems: kernel
[Documentation on labels]
Reported-by: syzbot+f87fd2bd13a4c9c5af8a@syzkaller.appspotmail.com
First crash: 212d, last: 80d
Cause bisection: failed (error log, bisect log)
  
Fix bisection: fixed by (bisect log) :
commit 5f6bd380c7bdbe10f7b4e8ddcceed60ce0714c6d
Author: Peter Zijlstra <peterz@infradead.org>
Date: Mon May 27 12:06:55 2024 +0000

  sched/rt: Remove default bandwidth control

  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [kernel?] INFO: rcu detected stall in rt_sigreturn 0 (2) 2024/10/04 20:32
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in rt_sigreturn origin:upstream missing-backport C 61 4d00h 438d 0/3 upstream: reported C repro on 2023/09/10 03:48
Last patch testing requests (2)
Created Duration User Patch Repo Result
2024/07/16 00:19 2h13m retest repro upstream report log
2024/05/06 17:51 17m retest repro upstream report log
Fix bisection attempts (3)
Created Duration User Patch Repo Result
2024/10/04 13:45 6h46m bisect fix upstream OK (1) job log
2024/08/29 14:33 1h46m (2) bisect fix upstream OK (0) job log log
2024/07/30 12:10 1h54m bisect fix upstream OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P5586/1:b..l
rcu: 	(detected by 0, t=10503 jiffies, g=15021, q=836 ncpus=2)
task:syz-executor.2  state:R  running task     stack:20288 pid:5586  tgid:5586  ppid:5548   flags:0x00004002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5408 [inline]
 __schedule+0x17e8/0x4a20 kernel/sched/core.c:6745
 preempt_schedule_common+0x84/0xd0 kernel/sched/core.c:6924
 preempt_schedule+0xe1/0xf0 kernel/sched/core.c:6948
 preempt_schedule_thunk+0x1a/0x30 arch/x86/entry/thunk.S:12
 vprintk_emit+0x690/0x770 kernel/printk/printk.c:2346
 _printk+0xd5/0x120 kernel/printk/printk.c:2370
 batadv_check_known_mac_addr+0x2bd/0x410 net/batman-adv/hard-interface.c:528
 batadv_hard_if_event+0x3a5/0x1620 net/batman-adv/hard-interface.c:998
 notifier_call_chain+0x1a1/0x3e0 kernel/notifier.c:93
 call_netdevice_notifiers_extack net/core/dev.c:2030 [inline]
 call_netdevice_notifiers net/core/dev.c:2044 [inline]
 dev_set_mac_address+0x3d9/0x510 net/core/dev.c:9044
 dev_set_mac_address_user+0x31/0x50 net/core/dev.c:9058
 do_setlink+0x8b3/0x41f0 net/core/rtnetlink.c:2855
 __rtnl_newlink net/core/rtnetlink.c:3696 [inline]
 rtnl_newlink+0x180b/0x20a0 net/core/rtnetlink.c:3743
 rtnetlink_rcv_msg+0x89d/0x1180 net/core/rtnetlink.c:6635
 netlink_rcv_skb+0x1e5/0x430 net/netlink/af_netlink.c:2564
 netlink_unicast_kernel net/netlink/af_netlink.c:1335 [inline]
 netlink_unicast+0x7ec/0x980 net/netlink/af_netlink.c:1361
 netlink_sendmsg+0x8db/0xcb0 net/netlink/af_netlink.c:1905
 sock_sendmsg_nosec net/socket.c:730 [inline]
 __sock_sendmsg+0x223/0x270 net/socket.c:745
 __sys_sendto+0x3a4/0x4f0 net/socket.c:2192
 __do_sys_sendto net/socket.c:2204 [inline]
 __se_sys_sendto net/socket.c:2200 [inline]
 __x64_sys_sendto+0xde/0x100 net/socket.c:2200
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fdd0787ed9c
RSP: 002b:00007ffefd40ed20 EFLAGS: 00000293 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00007fdd084e4620 RCX: 00007fdd0787ed9c
RDX: 000000000000002c RSI: 00007fdd084e4670 RDI: 0000000000000003
RBP: 0000000000000000 R08: 00007ffefd40ed74 R09: 000000000000000c
R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000003
R13: 0000000000000000 R14: 00007fdd084e4670 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10555 jiffies! g15021 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:26448 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5408 [inline]
 __schedule+0x17e8/0x4a20 kernel/sched/core.c:6745
 __schedule_loop kernel/sched/core.c:6822 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6837
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2581
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2000
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2202
 kthread+0x2f2/0x390 kernel/kthread.c:389
 ret_from_fork+0x4d/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 5933 Comm: syz-executor.3 Not tainted 6.10.0-rc4-syzkaller-00301-g5f583a3162ff #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
RIP: 0010:native_safe_halt arch/x86/include/asm/irqflags.h:48 [inline]
RIP: 0010:arch_safe_halt arch/x86/include/asm/irqflags.h:86 [inline]
RIP: 0010:kvm_wait+0x2ac/0x2c0 arch/x86/kernel/kvm.c:1060
Code: a0 48 89 df e8 25 8a bb 00 eb 96 e8 7e 8d 45 0a f3 0f 1e fa e8 75 dd 55 00 f4 e9 04 fe ff ff f3 0f 1e fa e8 66 dd 55 00 fb f4 <e9> f4 fd ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 90 90
RSP: 0018:ffffc90000a18620 EFLAGS: 00000246
RAX: ffffffff8140482a RBX: ffff8880b953f614 RCX: ffff88801b7fbc00
RDX: 0000000000000101 RSI: ffffffff8bcabb80 RDI: ffffffff8c1feb80
RBP: ffffc90000a186f0 R08: ffffffff92fa75ef R09: 1ffffffff25f4ebd
R10: dffffc0000000000 R11: fffffbfff25f4ebe R12: 1ffff920001430c8
R13: dffffc0000000000 R14: 0000000000000001 R15: 0000000000000001
FS:  00007f15d11736c0(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f15d1193f80 CR3: 0000000066660000 CR4: 0000000000350ef0
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 pv_wait arch/x86/include/asm/paravirt.h:596 [inline]
 pv_wait_node kernel/locking/qspinlock_paravirt.h:322 [inline]
 __pv_queued_spin_lock_slowpath+0x4a6/0xdc0 kernel/locking/qspinlock.c:473
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:584 [inline]
 queued_spin_lock_slowpath+0x42/0x50 arch/x86/include/asm/qspinlock.h:51
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x272/0x370 kernel/locking/spinlock_debug.c:116
 spin_lock include/linux/spinlock.h:351 [inline]
 mac80211_hwsim_tx_frame_no_nl+0x97f/0x18d0 drivers/net/wireless/virtual/mac80211_hwsim.c:1808
 mac80211_hwsim_tx_frame+0x1cc/0x220 drivers/net/wireless/virtual/mac80211_hwsim.c:2206
 mac80211_hwsim_beacon_tx+0x3bf/0x850 drivers/net/wireless/virtual/mac80211_hwsim.c:2306
 __iterate_interfaces+0x225/0x4c0 net/mac80211/util.c:772
 ieee80211_iterate_active_interfaces_atomic+0xd8/0x170 net/mac80211/util.c:808
 mac80211_hwsim_beacon+0xd4/0x1f0 drivers/net/wireless/virtual/mac80211_hwsim.c:2336
 __run_hrtimer kernel/time/hrtimer.c:1687 [inline]
 __hrtimer_run_queues+0x59d/0xd50 kernel/time/hrtimer.c:1751
 hrtimer_run_softirq+0x19a/0x2c0 kernel/time/hrtimer.c:1768
 handle_softirqs+0x2c6/0x970 kernel/softirq.c:554
 __do_softirq kernel/softirq.c:588 [inline]
 invoke_softirq kernel/softirq.c:428 [inline]
 __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:649
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:lock_release+0x630/0x9f0 kernel/locking/lockdep.c:5778
Code: 3c 3b 00 74 08 4c 89 f7 e8 1d 41 89 00 f6 84 24 91 00 00 00 02 75 77 41 f7 c5 00 02 00 00 74 01 fb 48 c7 44 24 60 0e 36 e0 45 <4b> c7 04 27 00 00 00 00 4b c7 44 27 08 00 00 00 00 65 48 8b 04 25
RSP: 0018:ffffc9000565f940 EFLAGS: 00000206
RAX: 0000000000000001 RBX: 1ffff92000acbf3a RCX: ffffc9000565f903
RDX: 0000000000000000 RSI: ffffffff8bcacd00 RDI: ffffffff8c1feb80
RBP: ffffc9000565fa70 R08: ffffffff8fad4cef R09: 1ffffffff1f5a99d
R10: dffffc0000000000 R11: fffffbfff1f5a99e R12: 1ffff92000acbf34
R13: 0000000000000246 R14: ffffc9000565f9d0 R15: dffffc0000000000
 __copy_from_user include/linux/uaccess.h:97 [inline]
 check_xstate_in_sigframe arch/x86/kernel/fpu/signal.c:35 [inline]
 __fpu_restore_sig arch/x86/kernel/fpu/signal.c:348 [inline]
 fpu__restore_sig+0x1a1/0x11b0 arch/x86/kernel/fpu/signal.c:493
 restore_sigcontext+0x675/0x7d0 arch/x86/kernel/signal_64.c:95
 __do_sys_rt_sigreturn+0x17f/0x270 arch/x86/kernel/signal_64.c:263
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f15d047d0a7
Code: 14 25 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 <0f> 05 48 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89
RSP: 002b:00007f15d1173178 EFLAGS: 00000246
RAX: 00000000000000ca RBX: 00007f15d05b4398 RCX: 00007f15d047d0a9
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 00007f15d05b4398
RBP: 00007f15d05b4390 R08: 00007f15d11736c0 R09: 00007f15d11736c0
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f15d05b439c
R13: 000000000000006e R14: 00007ffd15f09fc0 R15: 00007ffd15f0a0a8
 </TASK>
sched: RT throttling activated

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/06/23 16:26 upstream 5f583a3162ff edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in rt_sigreturn
2024/04/22 06:57 upstream 48cf398f15fc af24b050 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in rt_sigreturn
2024/06/16 16:05 linux-next a957267fa7e9 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in rt_sigreturn
* Struck through repros no longer work on HEAD.