syzbot


INFO: rcu detected stall in shmem_fault (6)

Status: upstream: reported on 2024/09/19 14:28
Subsystems: mm
[Documentation on labels]
Reported-by: syzbot+4145b11cdf925264bff4@syzkaller.appspotmail.com
First crash: 169d, last: 3d06h
Discussions (2)
Title Replies (including bot) Last reply
[PATCH] mm: remove the newlines, which are added for unknown reasons and interfere with bug analysis 6 (6) 2024/10/07 11:31
[syzbot] [cgroups?] [mm?] INFO: rcu detected stall in shmem_fault (6) 0 (1) 2024/09/19 14:28
Similar bugs (8)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in shmem_fault (4) kernel 1 654d 654d 0/28 auto-obsoleted due to no activity on 2023/04/08 07:17
linux-5.15 INFO: rcu detected stall in shmem_fault (2) 5 15d 133d 0/3 upstream: reported on 2024/06/03 20:13
upstream INFO: rcu detected stall in shmem_fault mm 13 2037d 2197d 0/28 closed as dup on 2019/01/02 16:58
linux-6.1 INFO: rcu detected stall in shmem_fault 7 65d 130d 0/3 upstream: reported on 2024/06/07 10:17
upstream INFO: rcu detected stall in shmem_fault (5) cgroups mm 3 424d 444d 23/28 fixed on 2023/10/12 12:48
linux-5.15 INFO: rcu detected stall in shmem_fault 1 409d 409d 0/3 auto-obsoleted due to no activity on 2023/12/11 11:17
upstream INFO: rcu detected stall in shmem_fault (3) fs mm 4 903d 903d 0/28 auto-closed as invalid on 2022/06/25 08:10
upstream INFO: rcu detected stall in shmem_fault (2) cgroups mm 1 1777d 1777d 0/28 closed as invalid on 2019/12/04 14:04

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=b434/1/0x4000000000000000 softirq=33284/33285 fqs=1
rcu: 	(detected by 0, t=10506 jiffies, g=37093, q=235 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 10981 Comm: syz.3.2489 Not tainted 6.12.0-rc2-syzkaller-00260-g9e4c6c1ad9a1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:__lock_acquire+0xc40/0x2050 kernel/locking/lockdep.c:5172
Code: 00 41 8b 07 a9 00 00 18 00 0f 84 d3 04 00 00 4c 89 e7 48 89 ee ba 08 00 00 00 e8 eb 64 00 00 45 31 f6 85 c0 0f 84 d0 07 00 00 <48> c7 c0 40 99 39 9a 48 c1 e8 03 48 bf 00 00 00 00 00 fc ff df 0f
RSP: 0018:ffffc90000a18a30 EFLAGS: 00000002
RAX: 0000000000000001 RBX: 1ffff1100f9f216a RCX: ffffffff8170ce1a
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffffff942929a0
RBP: ffff88807cf90b30 R08: ffffffff942929a7 R09: 1ffffffff2852534
R10: dffffc0000000000 R11: fffffbfff2852535 R12: ffff88807cf90000
R13: ffff88807cf90ad8 R14: 0000000000000000 R15: ffff88807cf90b50
FS:  00007f0a039626c0(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000110c2d19ed CR3: 0000000079840000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5825
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:351 [inline]
 advance_sched+0xab/0xca0 net/sched/sch_taprio.c:924
 __run_hrtimer kernel/time/hrtimer.c:1691 [inline]
 __hrtimer_run_queues+0x59b/0xd50 kernel/time/hrtimer.c:1755
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1817
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1026 [inline]
 __sysvec_apic_timer_interrupt+0x110/0x3f0 arch/x86/kernel/apic/apic.c:1043
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1037 [inline]
 sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1037
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0xd8/0x140 kernel/locking/spinlock.c:194
Code: 9c 8f 44 24 20 42 80 3c 23 00 74 08 4c 89 f7 e8 8e a4 34 f6 f6 44 24 21 02 75 52 41 f7 c7 00 02 00 00 74 01 fb bf 01 00 00 00 <e8> c3 6d 9c f5 65 8b 05 e4 07 3d 74 85 c0 74 43 48 c7 04 24 0e 36
RSP: 0018:ffffc900094a69a0 EFLAGS: 00000206
RAX: 324caa1485d38e00 RBX: 1ffff92001294d38 RCX: ffffffff8170ce1a
RDX: dffffc0000000000 RSI: ffffffff8c0acac0 RDI: 0000000000000001
RBP: ffffc900094a6a30 R08: ffffffff94292807 R09: 1ffffffff2852500
R10: dffffc0000000000 R11: fffffbfff2852501 R12: dffffc0000000000
R13: 1ffff92001294d34 R14: ffffc900094a69c0 R15: 0000000000000246
 spin_unlock_irqrestore include/linux/spinlock.h:406 [inline]
 rmqueue_bulk mm/page_alloc.c:2307 [inline]
 __rmqueue_pcplist+0x1dc7/0x2650 mm/page_alloc.c:2983
 rmqueue_pcplist mm/page_alloc.c:3025 [inline]
 rmqueue mm/page_alloc.c:3056 [inline]
 get_page_from_freelist+0x870/0x3180 mm/page_alloc.c:3454
 __alloc_pages_noprof+0x292/0x710 mm/page_alloc.c:4733
 alloc_pages_mpol_noprof+0x3e8/0x680 mm/mempolicy.c:2265
 folio_alloc_mpol_noprof+0x36/0x50 mm/mempolicy.c:2283
 shmem_alloc_folio mm/shmem.c:1774 [inline]
 shmem_alloc_and_add_folio+0x49b/0x13d0 mm/shmem.c:1813
 shmem_get_folio_gfp+0x5a9/0x20a0 mm/shmem.c:2335
 shmem_fault+0x253/0x6f0 mm/shmem.c:2536
 __do_fault+0x135/0x460 mm/memory.c:4876
 do_read_fault mm/memory.c:5282 [inline]
 do_fault mm/memory.c:5416 [inline]
 do_pte_missing mm/memory.c:3965 [inline]
 handle_pte_fault+0x2d1c/0x6800 mm/memory.c:5751
 __handle_mm_fault mm/memory.c:5894 [inline]
 handle_mm_fault+0x1053/0x1ad0 mm/memory.c:6062
 faultin_page mm/gup.c:1187 [inline]
 __get_user_pages+0x1b16/0x48d0 mm/gup.c:1485
 populate_vma_page_range+0x264/0x330 mm/gup.c:1923
 __mm_populate+0x27a/0x460 mm/gup.c:2026
 mm_populate include/linux/mm.h:3371 [inline]
 vm_mmap_pgoff+0x2c3/0x3d0 mm/util.c:593
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f0a02b7dff9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f0a03962038 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
RAX: ffffffffffffffda RBX: 00007f0a02d36058 RCX: 00007f0a02b7dff9
RDX: 0000000000000003 RSI: 0000000000b36000 RDI: 0000000020000000
RBP: 00007f0a02bf0296 R08: ffffffffffffffff R09: 0000000000000000
R10: 0000000000008031 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000001 R14: 00007f0a02d36058 R15: 00007fff34707548
 </TASK>
rcu: rcu_preempt kthread starved for 10500 jiffies! g37093 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:24024 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5322 [inline]
 __schedule+0x1843/0x4ae0 kernel/sched/core.c:6682
 __schedule_loop kernel/sched/core.c:6759 [inline]
 schedule+0x14b/0x320 kernel/sched/core.c:6774
 schedule_timeout+0x1be/0x310 kernel/time/timer.c:2615
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:2045
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:2247
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 UID: 0 PID: 12 Comm: kworker/u8:1 Not tainted 6.12.0-rc2-syzkaller-00260-g9e4c6c1ad9a1 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:340 [inline]
RIP: 0010:smp_call_function_many_cond+0x19f3/0x2ca0 kernel/smp.c:884
Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 56 f3 0b 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 01 ef 0b 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 e5 ee
RSP: 0018:ffffc900001176e0 EFLAGS: 00000293
RAX: ffffffff8188f81b RBX: 1ffff110170e8911 RCX: ffff88801bef5a00
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc900001178e0 R08: ffffffff8188f7ea R09: 1ffffffff2852500
R10: dffffc0000000000 R11: fffffbfff2852501 R12: dffffc0000000000
R13: ffff8880b8744888 R14: ffff8880b863fc40 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe805b052d8 CR3: 000000000e734000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1051
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2085 [inline]
 text_poke_bp_batch+0x352/0xb30 arch/x86/kernel/alternative.c:2295
 text_poke_flush arch/x86/kernel/alternative.c:2486 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2493
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 toggle_allocation_gate+0xbc/0x260 mm/kfence/core.c:849
 process_one_work kernel/workqueue.c:3229 [inline]
 process_scheduled_works+0xa63/0x1850 kernel/workqueue.c:3310
 worker_thread+0x870/0xd30 kernel/workqueue.c:3391
 kthread+0x2f0/0x390 kernel/kthread.c:389
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>

Crashes (39):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/10/11 21:38 upstream 9e4c6c1ad9a1 cd942402 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/09/29 23:34 upstream e7ed34365879 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/09/27 04:29 upstream 075dbe9f6e3c 9314348a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/09/15 14:18 upstream 0babf683783d 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/09/10 04:46 upstream bc83b4d1f086 dd9c7e97 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/09/01 16:15 upstream 431c1646e1f8 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/09/01 07:33 upstream e8784b0aef62 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/08/18 10:20 upstream df6cbc62cc9b dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/08/18 09:10 upstream df6cbc62cc9b dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/08/15 02:12 upstream 9d5906799f7d e4bacdaf .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/08/10 14:55 upstream afdab700f65e 6f4edef4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/08/05 08:18 upstream a5dbd76a8942 1786a2a8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/08/01 11:57 upstream 21b136cc63d2 1e9c4cf3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/07/30 07:16 upstream 94ede2a3e913 5187fc86 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/07/10 12:02 upstream 34afb82a3c67 e7213be3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/07/08 17:36 upstream 256abd8e550c cde64f7d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/07/04 08:41 upstream 795c58e4c7fc 409d975c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/06/30 10:36 upstream 8282d5af7be8 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/06/27 11:32 upstream 24ca36a562d6 5c045c04 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/23 03:14 upstream 5f583a3162ff edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/20 09:53 upstream 2ccbdf43d5e7 41b7e219 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in shmem_fault
2024/06/15 01:24 upstream 2ccbdf43d5e7 8d849073 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/08 10:58 upstream 96e09b8f8166 82c05ab8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/06/07 01:21 upstream 8a92980606e3 121701b6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/05/28 11:58 upstream 2bfcfd584ff5 f550015e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/05/22 22:33 upstream 8f6a15f095a6 4d098039 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in shmem_fault
2024/04/29 06:22 upstream e67572cd2204 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in shmem_fault
2024/05/10 20:11 net a2c78977950d f7c35481 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in shmem_fault
2024/10/12 06:32 net-next d677aebd663d 084d8178 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/10/08 10:31 net-next 1405981bbba0 402f1df0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/10/07 10:12 net-next 8b641b5e4c78 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/10/03 19:27 net-next 7c2f1c2690a5 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/10/02 21:34 net-next 44badc908f2c a4c7fd36 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/09/28 02:47 net-next c824deb1a897 2b1784d6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in shmem_fault
2024/07/22 14:40 linux-next 41c196e567fb f063dfd9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/07/05 17:22 linux-next 0b58e108042b 2a40360c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/17 09:43 linux-next a957267fa7e9 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/11 20:27 linux-next a957267fa7e9 c7d7d815 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in shmem_fault
2024/06/05 09:21 linux-next 234cb065ad82 e1e2c66e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in shmem_fault
* Struck through repros no longer work on HEAD.