syzbot


INFO: rcu detected stall in sys_socket (10)

Status: upstream: reported C repro on 2023/11/30 15:24
Subsystems: net apparmor
[Documentation on labels]
Reported-by: syzbot+de8e83db70e8beedd556@syzkaller.appspotmail.com
First crash: 150d, last: 2d11h
Cause bisection: introduced by (bisect log) :
commit 5a781ccbd19e4664babcbe4b4ead7aa2b9283d22
Author: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Date: Sat Sep 29 00:59:43 2018 +0000

  tc: Add support for configuring the taprio scheduler

Crash: BUG: soft lockup in do_idle (log)
Repro: C syz .config
  
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [net?] INFO: rcu detected stall in sys_socket (10) 6 (8) 2023/12/05 02:10
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_socket (4) fs 1 1094d 1094d 0/26 auto-closed as invalid on 2021/07/27 17:27
linux-5.15 INFO: rcu detected stall in sys_socket 1 206d 206d 0/3 auto-obsoleted due to no activity on 2024/01/11 06:48
upstream INFO: rcu detected stall in sys_socket (5) net 2 932d 957d 0/26 auto-closed as invalid on 2022/01/06 01:08
upstream INFO: rcu detected stall in sys_socket kernel 11 1605d 1606d 0/26 closed as invalid on 2019/12/04 14:04
linux-4.19 INFO: rcu detected stall in sys_socket 1 735d 735d 0/1 auto-closed as invalid on 2022/08/20 07:08
upstream INFO: rcu detected stall in sys_socket (6) cgroups mm 2 591d 635d 0/26 auto-obsoleted due to no activity on 2022/12/12 22:48
upstream INFO: rcu detected stall in sys_socket (7) kernel 2 450d 483d 0/26 auto-obsoleted due to no activity on 2023/05/02 14:50
upstream INFO: rcu detected stall in sys_socket (9) kasan mm 2 268d 277d 0/26 closed as invalid on 2023/09/07 14:25
upstream INFO: rcu detected stall in sys_socket (2) kernel 3 1571d 1571d 0/26 closed as invalid on 2020/01/08 05:23
linux-5.15 INFO: rcu detected stall in sys_socket (2) origin:upstream C 2 16d 55d 0/3 upstream: reported C repro on 2024/03/02 17:55
upstream INFO: rcu detected stall in sys_socket (3) kernel 4 1570d 1570d 0/26 closed as invalid on 2020/01/09 08:13
android-5-15 BUG: soft lockup in sys_socket origin:lts C 11 2d19h 16d 0/2 upstream: reported C repro on 2024/04/10 16:23
Last patch testing requests (3)
Created Duration User Patch Repo Result
2024/02/07 18:23 1h26m retest repro linux-next report log
2023/12/25 00:55 25m retest repro upstream report log
2023/12/05 02:10 36m eadavis@qq.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 18d46e76d7c2 report log
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2024/03/11 22:11 2h48m bisect fix upstream job log (0) log
2024/01/10 13:15 2h59m bisect fix upstream job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=a20c/1/0x4000000000000000 softirq=5979/5980 fqs=2
rcu: 	(detected by 0, t=10502 jiffies, g=5653, q=134 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 5112 Comm: syz-executor228 Not tainted 6.8.0-syzkaller-08951-gfe46a7dd189e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
RIP: 0010:mark_lock+0x2b/0x350 kernel/locking/lockdep.c:4647
Code: 41 57 41 56 41 55 41 54 53 48 83 ec 10 49 89 f7 48 89 3c 24 49 bd 00 00 00 00 00 fc ff df 83 fa 08 75 27 49 8d 5f 20 48 89 d8 <48> c1 e8 03 42 0f b6 04 28 84 c0 0f 85 8a 02 00 00 31 ed f6 43 02
RSP: 0018:ffffc90000a08908 EFLAGS: 00000046
RAX: ffff88807c7a4720 RBX: ffff88807c7a4720 RCX: 0000000000000000
RDX: 0000000000000008 RSI: ffff88807c7a4700 RDI: ffff88807c7a3c00
RBP: ffff88807c7a4700 R08: ffffffff92cae507 R09: 1ffffffff2595ca0
R10: dffffc0000000000 R11: fffffbfff2595ca1 R12: 0000000000000001
R13: dffffc0000000000 R14: 1ffff1100f8f48e4 R15: ffff88807c7a4700
FS:  000055556bbe5480(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 0000000025576000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 mark_usage kernel/locking/lockdep.c:4599 [inline]
 __lock_acquire+0xc0a/0x1fd0 kernel/locking/lockdep.c:5091
 lock_acquire+0x1e4/0x530 kernel/locking/lockdep.c:5754
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
 debug_object_deactivate+0x158/0x390 lib/debugobjects.c:763
 debug_hrtimer_deactivate kernel/time/hrtimer.c:428 [inline]
 debug_deactivate+0x1b/0x200 kernel/time/hrtimer.c:484
 __run_hrtimer kernel/time/hrtimer.c:1660 [inline]
 __hrtimer_run_queues+0x30f/0xd00 kernel/time/hrtimer.c:1756
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1818
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1032 [inline]
 __sysvec_apic_timer_interrupt+0x107/0x3a0 arch/x86/kernel/apic/apic.c:1049
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1043 [inline]
 sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1043
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__sanitizer_cov_trace_switch+0xe/0x120 kernel/kcov.c:321
Code: 0f 1f 84 00 00 00 00 00 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 41 57 41 56 41 54 53 4c 8b 16 <48> 8b 46 08 48 83 c0 f8 48 c1 c0 3d 48 83 f8 02 7f 1f 48 85 c0 74
RSP: 0018:ffffc90004537b78 EFLAGS: 00000286
RAX: 1ffff1100335280c RBX: 00000000534f434b RCX: dffffc0000000000
RDX: ffff8880182fd888 RSI: ffffffff8e790fd0 RDI: 00000000534f434b
RBP: ffffc90004537ca0 R08: ffffffff84570a3d R09: 0000000000000000
R10: 0000000000000005 R11: ffffffff845708b0 R12: ffff888019a94060
R13: ffff88807c7a4498 R14: 1ffff1100f8f4893 R15: ffff8880182fd888
 smack_d_instantiate+0x342/0xa50 security/smack/smack_lsm.c:3478
 security_d_instantiate+0x9f/0x100 security/security.c:3909
 d_instantiate+0x55/0xa0 fs/dcache.c:1878
 alloc_path_pseudo fs/file_table.c:334 [inline]
 alloc_file_pseudo+0x19e/0x290 fs/file_table.c:346
 sock_alloc_file+0xb8/0x290 net/socket.c:469
 sock_map_fd net/socket.c:494 [inline]
 __sys_socket+0x1dd/0x3c0 net/socket.c:1715
 __do_sys_socket net/socket.c:1720 [inline]
 __se_sys_socket net/socket.c:1718 [inline]
 __x64_sys_socket+0x7a/0x90 net/socket.c:1718
 do_syscall_64+0xfb/0x240
 entry_SYSCALL_64_after_hwframe+0x6d/0x75
RIP: 0033:0x7f78b4d849a9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 d1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd02510af8 EFLAGS: 00000246 ORIG_RAX: 0000000000000029
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f78b4d849a9
RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000010
RBP: 00000000000f4240 R08: 00007ffd02510b60 R09: 00007ffd02510b60
R10: 00007ffd02510b60 R11: 0000000000000246 R12: 00007ffd02510b60
R13: 0000000000098985 R14: 00007ffd02510b2c R15: 0000000000000003
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.959 msecs
rcu: rcu_preempt kthread starved for 10498 jiffies! g5653 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25496 pid:16    tgid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0x1781/0x49d0 kernel/sched/core.c:6736
 __schedule_loop kernel/sched/core.c:6813 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6828
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2572
 rcu_gp_fqs_loop+0x2df/0x1370 kernel/rcu/tree.c:1663
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1862
 kthread+0x2f0/0x390 kernel/kthread.c:388
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 2475 Comm: kworker/u8:8 Not tainted 6.8.0-syzkaller-08951-gfe46a7dd189e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline]
RIP: 0010:smp_call_function_many_cond+0x1850/0x2960 kernel/smp.c:855
Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 d9 d5 0b 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 84 d1 0b 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 68 d1
RSP: 0018:ffffc90009d276e0 EFLAGS: 00000293
RAX: ffffffff818922e8 RBX: 1ffff110172a8801 RCX: ffff888029c4da00
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc90009d278e0 R08: ffffffff818922b7 R09: 1ffffffff2595ca0
R10: dffffc0000000000 R11: fffffbfff2595ca1 R12: dffffc0000000000
R13: ffff8880b9544008 R14: ffff8880b943f440 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055556bbe5da8 CR3: 000000000df32000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1023
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2086 [inline]
 text_poke_bp_batch+0x352/0xb30 arch/x86/kernel/alternative.c:2296
 text_poke_flush arch/x86/kernel/alternative.c:2487 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2494
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826
 process_one_work kernel/workqueue.c:3254 [inline]
 process_scheduled_works+0xa00/0x1770 kernel/workqueue.c:3335
 worker_thread+0x86d/0xd70 kernel/workqueue.c:3416
 kthread+0x2f0/0x390 kernel/kthread.c:388
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
 </TASK>

Crashes (9):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/04/13 20:12 upstream fe46a7dd189e c8349e48 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_socket
2023/11/29 02:01 upstream 18d46e76d7c2 1adfb6f6 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_socket
2024/01/24 14:50 linux-next 8bf1262c53f5 1e153dc8 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_socket
2024/04/24 19:23 upstream 9d1ddab261f3 8bdc0f22 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_socket
2024/04/19 22:03 upstream 3cdb45594619 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_socket
2024/04/08 13:32 upstream fe46a7dd189e ca620dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_socket
2024/04/08 11:44 upstream fe46a7dd189e ca620dd8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_socket
2024/04/21 23:26 linux-next 7b4f2bc91c15 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_socket
2023/12/11 00:38 linux-next 8e00ce02066e 28b24332 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_socket
* Struck through repros no longer work on HEAD.