syzbot


INFO: rcu detected stall in validate_mm (3)

Status: upstream: reported C repro on 2024/05/12 09:19
Subsystems: mm
[Documentation on labels]
Reported-by: syzbot+a941018a091f1a1f9546@syzkaller.appspotmail.com
First crash: 196d, last: 23d
Cause bisection: failed (error log, bisect log)
  
Discussions (2)
Title Replies (including bot) Last reply
[RFC] Signaling overhead on RT tasks is causing RCU stall 3 (3) 2024/07/09 07:43
[syzbot] [mm?] INFO: rcu detected stall in validate_mm (3) 2 (5) 2024/07/03 08:24
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in validate_mm (2) mm 2 380d 390d 0/28 auto-obsoleted due to no activity on 2024/02/04 15:00
linux-6.1 INFO: rcu detected stall in validate_mm 6 43d 109d 0/3 upstream: reported on 2024/08/03 11:55
Last patch testing requests (4)
Created Duration User Patch Repo Result
2024/09/07 08:31 15m retest repro upstream report log
2024/07/03 08:24 15m radoslaw.zielonek@gmail.com patch upstream report log
2024/07/03 07:32 21m radoslaw.zielonek@gmail.com patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git dccb07f2914cdab2ac3a5b6c98406f765acab803 report log
2024/06/13 00:49 15m retest repro upstream report log
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/09/26 10:12 1h12m bisect fix upstream OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P17678/1:b..l
rcu: 	(detected by 1, t=10502 jiffies, g=36541, q=38 ncpus=2)
task:syz-executor952 state:R  running task     stack:28968 pid:17678 tgid:17678 ppid:5114   flags:0x00000002
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0xf15/0x5d00 kernel/sched/core.c:6746
 preempt_schedule_irq+0x51/0x90 kernel/sched/core.c:7068
 irqentry_exit+0x36/0x90 kernel/entry/common.c:354
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:bytes_is_nonzero mm/kasan/generic.c:88 [inline]
RIP: 0010:memory_is_nonzero mm/kasan/generic.c:122 [inline]
RIP: 0010:memory_is_poisoned_n mm/kasan/generic.c:129 [inline]
RIP: 0010:memory_is_poisoned mm/kasan/generic.c:161 [inline]
RIP: 0010:check_region_inline mm/kasan/generic.c:180 [inline]
RIP: 0010:kasan_check_range+0xc7/0x1a0 mm/kasan/generic.c:189
Code: 83 c0 08 48 39 d0 0f 84 be 00 00 00 48 83 38 00 74 ed 48 8d 50 08 eb 0d 48 83 c0 01 48 39 c2 0f 84 8d 00 00 00 80 38 00 74 ee <48> 89 c2 b8 01 00 00 00 48 85 d2 74 1e 41 83 e2 07 49 39 d1 75 0a
RSP: 0018:ffffc900031ef850 EFLAGS: 00000202
RAX: fffffbfff2949b78 RBX: fffffbfff2949b79 RCX: ffffffff8ac92249
RDX: fffffbfff2949b79 RSI: 0000000000000004 RDI: ffffffff94a4dbc0
RBP: fffffbfff2949b78 R08: 0000000000000001 R09: fffffbfff2949b78
R10: ffffffff94a4dbc3 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000001 R14: 0000000000000300 R15: 0000000000000000
 instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
 atomic_inc include/linux/atomic/atomic-instrumented.h:435 [inline]
 mt_validate_nulls+0x5e9/0x9e0 lib/maple_tree.c:7550
 mt_validate+0x3148/0x4390 lib/maple_tree.c:7599
 validate_mm+0x9c/0x4b0 mm/mmap.c:288
 mmap_region+0x1478/0x2760 mm/mmap.c:2934
 do_mmap+0x8ae/0xf10 mm/mmap.c:1385
 vm_mmap_pgoff+0x1ab/0x3c0 mm/util.c:573
 ksys_mmap_pgoff+0x7d/0x5b0 mm/mmap.c:1431
 __do_sys_mmap arch/x86/kernel/sys_x86_64.c:86 [inline]
 __se_sys_mmap arch/x86/kernel/sys_x86_64.c:79 [inline]
 __x64_sys_mmap+0x125/0x190 arch/x86/kernel/sys_x86_64.c:79
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f305228c143
RSP: 002b:00007ffdd7b4fc18 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
RAX: ffffffffffffffda RBX: fffffffffffff000 RCX: 00007f305228c143
RDX: 0000000000000000 RSI: 0000000000021000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 00000000ffffffff R09: 0000000000000000
R10: 0000000000020022 R11: 0000000000000246 R12: 00007ffdd7b4fe70
R13: ffffffffffffffc0 R14: 0000000000001000 R15: 0000000000000000
 </TASK>
rcu: rcu_preempt kthread starved for 10533 jiffies! g36541 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28736 pid:16    tgid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5409 [inline]
 __schedule+0xf15/0x5d00 kernel/sched/core.c:6746
 __schedule_loop kernel/sched/core.c:6823 [inline]
 schedule+0xe7/0x350 kernel/sched/core.c:6838
 schedule_timeout+0x136/0x2a0 kernel/time/timer.c:2582
 rcu_gp_fqs_loop+0x1eb/0xb00 kernel/rcu/tree.c:1663
 rcu_gp_kthread+0x271/0x380 kernel/rcu/tree.c:1862
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 17676 Comm: syz-executor952 Not tainted 6.9.0-rc7-syzkaller-00012-gdccb07f2914c #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:160 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x29/0x50 kernel/locking/spinlock.c:202
Code: 90 f3 0f 1e fa 53 48 8b 74 24 08 48 89 fb 48 83 c7 18 e8 6a 98 8c f6 48 89 df e8 c2 14 8d f6 e8 ed 98 b5 f6 fb bf 01 00 00 00 <e8> b2 4f 7e f6 65 8b 05 b3 88 24 75 85 c0 74 06 5b c3 cc cc cc cc
RSP: 0018:ffffc9000321fcf0 EFLAGS: 00000202
RAX: 0000000003959e61 RBX: ffff88801c3d0940 RCX: 1ffffffff1f3e279
RDX: 0000000000000000 RSI: ffffffff8b0cae00 RDI: 0000000000000001
RBP: ffff88801c3d0d40 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff8f9f5657 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000021 R14: ffff88801c3d0940 R15: ffff88801c3d0940
FS:  00007f305221e6c0(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f305221de40 CR3: 000000002dcec000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 spin_unlock_irq include/linux/spinlock.h:401 [inline]
 get_signal+0x1e3e/0x2710 kernel/signal.c:2914
 arch_do_signal_or_restart+0x90/0x7e0 arch/x86/kernel/signal.c:310
 exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0x14a/0x2a0 kernel/entry/common.c:218
 do_syscall_64+0xdc/0x260 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f305228c107
Code: 14 25 28 00 00 00 75 05 48 83 c4 28 c3 e8 31 1b 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 <0f> 05 48 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89
RSP: 002b:00007f305221e238 EFLAGS: 00000246
RAX: 00000000000000ca RBX: 00007f305230f318 RCX: 00007f305228c109
RDX: 0000000000000000 RSI: 0000000000000080 RDI: 00007f305230f318
RBP: 00007f305230f310 R08: 00007f305221e6c0 R09: 00007f305221e6c0
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f30522dc278
R13: 000000000000006e R14: 00007ffdd7b4fb90 R15: 00007ffdd7b4fc78
 </TASK>
sched: RT throttling activated

Crashes (30):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/08 09:12 upstream dccb07f2914c 4cf3f9b3 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/08/24 07:02 upstream d2bafcf224f3 d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/08/21 20:03 upstream b311c1b497e5 db5852f9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/08/18 02:01 upstream df6cbc62cc9b dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/08/14 01:05 upstream 6b4aa469f049 bde81f6f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/08/13 10:50 upstream d74da846046a 7b0f4b46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/08/08 06:02 upstream 6a0e38264012 de12cf65 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/08/08 04:43 upstream 6a0e38264012 7b2f2f35 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/08/06 08:48 upstream b446a2dae984 e1bdb00a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/08/06 07:12 upstream b446a2dae984 e1bdb00a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/08/02 02:07 upstream c0ecd6388360 1e9c4cf3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/07/22 22:29 upstream 933069701c1b f063dfd9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/07/11 12:36 upstream a19ea421490d c699c2eb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/07/06 22:03 upstream 1dd28064d416 bc4ebbb5 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/07/04 21:49 upstream 795c58e4c7fc dc6bbff0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/06/28 08:18 upstream 6d6444ba8205 b62c7d46 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/06/27 13:55 upstream 24ca36a562d6 5c045c04 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/06/24 08:05 upstream f2661062f16b edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/06/22 12:13 upstream 35bb670d65fc edc5149a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in validate_mm
2024/06/20 15:13 upstream 2ccbdf43d5e7 dac2aa43 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/06/18 07:53 upstream 2ccbdf43d5e7 1f11cfd7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/05/29 13:25 upstream e0cce98fe279 34889ee3 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/05/22 07:16 upstream 8f6a15f095a6 1014eca7 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in validate_mm
2024/05/12 19:48 upstream ba16c1cf11c9 9026e142 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/10/28 11:15 net e31a8219fbfc 65e8686b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in validate_mm
2024/10/09 23:22 https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing 4a9fe2a8ac53 56fb2cb7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-usb INFO: rcu detected stall in validate_mm
2024/10/05 07:33 https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing 4a9fe2a8ac53 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-usb INFO: rcu detected stall in validate_mm
2024/07/24 19:02 linux-next 9ec6ec93f2c1 b24754ac .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/06/25 05:01 linux-next 62c97045b8f7 215eef4a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in validate_mm
2024/06/15 17:11 linux-next a957267fa7e9 f429ab00 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in validate_mm
* Struck through repros no longer work on HEAD.