syzbot


INFO: rcu detected stall in sys_io_uring_enter (2)

Status: upstream: reported on 2024/09/20 05:20
Subsystems: io-uring
[Documentation on labels]
Reported-by: syzbot+5fca234bd7eb378ff78e@syzkaller.appspotmail.com
Fix commit: eac2ca2d682f io_uring: check if we need to reschedule during overflow flush
Patched on: [ci-qemu-gce-upstream-auto ci-qemu-upstream ci-qemu-upstream-386 ci-qemu2-arm32 ci-qemu2-arm64 ci-qemu2-arm64-compat ci-qemu2-arm64-mte ci-qemu2-riscv64 ci-snapshot-upstream-root ci-upstream-bpf-kasan-gce ci-upstream-bpf-next-kasan-gce ci-upstream-gce-arm64 ci-upstream-gce-leak ci-upstream-kasan-badwrites-root ci-upstream-kasan-gce ci-upstream-kasan-gce-386 ci-upstream-kasan-gce-root ci-upstream-kasan-gce-selinux-root ci-upstream-kasan-gce-smack-root ci-upstream-kmsan-gce-386-root ci-upstream-kmsan-gce-root ci-upstream-linux-next-kasan-gce-root ci-upstream-net-kasan-gce ci-upstream-net-this-kasan-gce ci2-upstream-fs ci2-upstream-kcsan-gce ci2-upstream-usb], missing on: [ci-qemu-native-arm64-kvm]
First crash: 154d, last: 2d00h
Discussions (6)
Title Replies (including bot) Last reply
[PATCH AUTOSEL 6.1 22/42] io_uring: check if we need to reschedule during overflow flush 1 (1) 2024/10/04 18:26
[PATCH AUTOSEL 6.6 29/58] io_uring: check if we need to reschedule during overflow flush 1 (1) 2024/10/04 18:24
[PATCH AUTOSEL 6.10 31/70] io_uring: check if we need to reschedule during overflow flush 1 (1) 2024/10/04 18:20
[PATCH AUTOSEL 6.11 32/76] io_uring: check if we need to reschedule during overflow flush 1 (1) 2024/10/04 18:16
[PATCH] io_uring: check if we need to reschedule during overflow flush 1 (1) 2024/09/20 08:54
[syzbot] [io-uring?] INFO: rcu detected stall in sys_io_uring_enter (2) 1 (2) 2024/09/20 08:51
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_io_uring_enter bpf 2 1057d 1066d 0/28 closed as invalid on 2022/02/08 10:00

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=2e84/1/0x4000000000000000 softirq=69789/69792 fqs=1
rcu: 	(detected by 1, t=10506 jiffies, g=62185, q=803 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 15138 Comm: syz.3.4303 Not tainted 6.12.0-rc1-syzkaller-00042-gf23aa4c0761a #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:lookup_chain_cache kernel/locking/lockdep.c:3798 [inline]
RIP: 0010:lookup_chain_cache_add kernel/locking/lockdep.c:3817 [inline]
RIP: 0010:validate_chain+0x1e6/0x5920 kernel/locking/lockdep.c:3872
Code: 89 df e8 cd 6b 8a 00 48 89 5c 24 48 48 8b 1b 48 85 db 74 48 48 83 c3 f8 74 42 4c 8d 7b 18 4c 89 f8 48 c1 e8 03 42 80 3c 20 00 <74> 08 4c 89 ff e8 a0 6b 8a 00 49 8b 07 4c 39 f0 0f 84 83 03 00 00
RSP: 0018:ffffc90000007600 EFLAGS: 00000046
RAX: 1ffffffff28527b7 RBX: ffffffff94293da0 RCX: ffffffff8170fcce
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff9428f800
RBP: ffffc90000007900 R08: ffffffff9428f807 R09: 1ffffffff2851f00
R10: dffffc0000000000 R11: fffffbfff2851f01 R12: dffffc0000000000
R13: ffff888028a1c728 R14: 9feb8235be99f827 R15: ffffffff94293db8
FS:  00007f3dde4a26c0(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2db1eff8 CR3: 0000000021f0e000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __lock_acquire+0x1384/0x2050 kernel/locking/lockdep.c:5202
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd5/0x120 kernel/locking/spinlock.c:162
 debug_object_activate+0x16d/0x510 lib/debugobjects.c:708
 debug_hrtimer_activate kernel/time/hrtimer.c:423 [inline]
 debug_activate kernel/time/hrtimer.c:478 [inline]
 enqueue_hrtimer+0x30/0x3c0 kernel/time/hrtimer.c:1085
 __run_hrtimer kernel/time/hrtimer.c:1708 [inline]
 __hrtimer_run_queues+0x6cb/0xd50 kernel/time/hrtimer.c:1755
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1817
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1026 [inline]
 __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1043
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1037 [inline]
 sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1037
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xd8/0x140 kernel/locking/spinlock.c:194
Code: 9c 8f 44 24 20 42 80 3c 23 00 74 08 4c 89 f7 e8 7e 0a 35 f6 f6 44 24 21 02 75 52 41 f7 c7 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> a3 d4 9c f5 65 8b 05 a4 78 3d 74 85 c0 74 43 48 c7 04 24 0e 36
RSP: 0018:ffffc9000cdb7aa0 EFLAGS: 00000206
RAX: cbe5f13cb0501900 RBX: 1ffff920019b6f58 RCX: ffffffff9a396903
RDX: dffffc0000000000 RSI: ffffffff8c0acac0 RDI: 0000000000000001
RBP: ffffc9000cdb7b38 R08: ffffffff901bb9ef R09: 1ffffffff203773d
R10: dffffc0000000000 R11: fffffbfff203773e R12: dffffc0000000000
R13: 1ffff920019b6f54 R14: ffffc9000cdb7ac0 R15: 0000000000000246
 spin_unlock_irqrestore include/linux/spinlock.h:406 [inline]
 prepare_to_wait_exclusive+0xd0/0x230 kernel/sched/wait.c:256
 io_cqring_wait io_uring/io_uring.c:2584 [inline]
 __do_sys_io_uring_enter io_uring/io_uring.c:3384 [inline]
 __se_sys_io_uring_enter+0x17d5/0x2bb0 io_uring/io_uring.c:3276
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f3ddd77dff9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f3dde4a2038 EFLAGS: 00000246 ORIG_RAX: 00000000000001aa
RAX: ffffffffffffffda RBX: 00007f3ddd935f80 RCX: 00007f3ddd77dff9
RDX: 0000000000400000 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007f3ddd7f0296 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f3ddd935f80 R15: 00007ffce4a34d88
 </TASK>
rcu: rcu_preempt kthread starved for 10500 jiffies! g62185 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:25888 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5315 [inline]
 __schedule+0x1843/0x4ae0 kernel/sched/core.c:6675
 __schedule_loop kernel/sched/core.c:6752 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6767
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2615
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2045
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2247
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 UID: 0 PID: 52 Comm: kworker/u8:3 Not tainted 6.12.0-rc1-syzkaller-00042-gf23aa4c0761a #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:340 [inline]
RIP: 0010:smp_call_function_many_cond+0x19f3/0x2ca0 kernel/smp.c:884
Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 46 f9 0b 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 f1 f4 0b 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 d5 f4
RSP: 0018:ffffc90000bc76e0 EFLAGS: 00000293
RAX: ffffffff8188e68b RBX: 1ffff110170c8ca5 RCX: ffff88801d693c00
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc90000bc78e0 R08: ffffffff8188e65a R09: 1ffffffff2851f00
R10: dffffc0000000000 R11: fffffbfff2851f01 R12: dffffc0000000000
R13: ffff8880b8646528 R14: ffff8880b873fc40 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000084 CR3: 000000000e734000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1051
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2085 [inline]
 text_poke_bp_batch+0x352/0xb30 arch/x86/kernel/alternative.c:2295
 text_poke_flush arch/x86/kernel/alternative.c:2486 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2493
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 toggle_allocation_gate+0xbc/0x260 mm/kfence/core.c:849
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (34):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/10/02 23:59 upstream f23aa4c0761a a4c7fd36 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/28 19:32 upstream 3efc57369a0c ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/28 04:12 upstream ad46e8f95e93 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/20 06:25 upstream 2004cef11ea0 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/16 05:06 upstream 98f7e32f20d2 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/14 07:01 upstream b7718454f937 ff60e2ca .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/11 19:38 upstream 7c6a3a65ace7 9326a104 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/09/04 00:10 upstream 88fac17500f4 9d47f20a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/28 22:42 upstream 928f79a188aa 940f38c1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/25 02:22 upstream 48fb4b3d9b43 d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/23 17:44 upstream 3d5f968a177d d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/23 00:58 upstream aa0743a22936 295a4b50 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/16 18:17 upstream 670c12ce09a8 76120936 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/16 18:17 upstream 670c12ce09a8 76120936 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/03 04:34 upstream 948752d2e010 53683cf2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in sys_io_uring_enter
2024/07/11 18:02 upstream 9d9a2f29aefd c699c2eb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/05 09:19 upstream 661e504db04c 2a40360c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/05/03 10:55 upstream f03359bca01b 375d4445 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_io_uring_enter
2024/08/13 11:05 upstream d74da846046a 7b0f4b46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: rcu detected stall in sys_io_uring_enter
2024/07/28 03:12 linux-next 668d33c9ff92 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/23 08:54 linux-next 41c196e567fb 708c10c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/19 01:52 linux-next 73399b58e5e5 ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/08 18:52 linux-next 0b58e108042b cde64f7d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/08 14:08 linux-next 0b58e108042b cde64f7d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/08 04:48 linux-next 0b58e108042b bc4ebbb5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/04 05:49 linux-next 0b58e108042b 409d975c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/07/02 20:14 linux-next 82e4255305c5 8373af66 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/26 16:06 linux-next 62c97045b8f7 880c1ca1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/26 00:35 linux-next 62c97045b8f7 da0bd5ca .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/22 06:19 linux-next f76698bd9a8c edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/16 21:52 linux-next a957267fa7e9 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/14 10:33 linux-next a957267fa7e9 a9616ff5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/13 21:52 linux-next a957267fa7e9 a9616ff5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
2024/06/10 08:11 linux-next d35b2284e966 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_io_uring_enter
* Struck through repros no longer work on HEAD.