KCSAN: data-race in event_sched_in / perf_event_release_kernel

Status: moderation: reported on 2026/11/20 00:05
Subsystems: perf
Labels: race:benign prio:low
[Documentation on labels]
Reported-by: syzbot+ca170c97cd5de8125b23@syzkaller.appspotmail.com
First crash: 2d21h, last: 2h40m
✨ AI-generated root cause summary:

In kernel/events/core.c, perf_event_release_kernel() performs an unmarked read of event->state while holding only ctx->mutex, racing with event_sched_in() -> perf_event_set_state() updating event->state via WRITE_ONCE() under ctx->lock during context switch. Because ctx->mutex serializes all transitions to PERF_EVENT_STATE_REVOKED and PERF_EVENT_STATE_DEAD, any concurrent ctx->lock transitions (such as PERF_EVENT_STATE_INACTIVE to PERF_EVENT_STATE_ACTIVE) remain strictly greater than PERF_EVENT_STATE_REVOKED. As a result, perf_event_release_kernel() still consistently takes the perf_remove_from_context() branch, making this a benign KCSAN data race caused by a missing READ_ONCE().

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 4405c18c95d2:

Root Cause Analysis

The KCSAN report is triggered by an unmarked read of struct perf_event::state in perf_event_release_kernel() racing with a marked write (WRITE_ONCE()) to event->state in perf_event_set_state() during context-switch scheduling (event_sched_in()).

Locking and Execution Flow

  1. Writer path (CPU 1, event_sched_in): During a context switch (finish_task_switch() -> perf_event_task_sched_in() -> __perf_event_task_sched_in() -> perf_event_context_sched_in()), the scheduler acquires the raw spinlock ctx->lock via perf_ctx_lock(cpuctx, ctx) (it cannot acquire ctx->mutex in atomic/IRQ-disabled context). In event_sched_in() (kernel/events/core.c:2849), it calls perf_event_set_state(event, PERF_EVENT_STATE_ACTIVE) (kernel/events/core.c:790), which executes:

    WRITE_ONCE(event->state, state);
    

    transitioning event->state from PERF_EVENT_STATE_INACTIVE (0) to PERF_EVENT_STATE_ACTIVE (1).

  2. Reader path (CPU 0, perf_event_release_kernel): When closing the perf event file descriptor (close_range() -> perf_release() -> perf_event_release_kernel()), perf_event_release_kernel() acquires ctx->mutex via perf_event_ctx_lock(event) (kernel/events/core.c:5976), without holding ctx->lock. At kernel/events/core.c:5990, it performs an unmarked read of event->state:

    if (event->state > PERF_EVENT_STATE_REVOKED) {
        perf_remove_from_context(event, DETACH_GROUP|DETACH_DEAD);
    } else {
        event->state = PERF_EVENT_STATE_DEAD;
    }
    

Why the Race Occurs and State Invariants

  • Synchronization mismatch: event->state is mutated under ctx->lock during scheduling operations (event_sched_in(), event_sched_out(), merge_sched_in()), while perf_event_release_kernel() checks event->state > PERF_EVENT_STATE_REVOKED holding only ctx->mutex.
  • Invariant preserved: Transitions into PERF_EVENT_STATE_REVOKED (-4) or PERF_EVENT_STATE_DEAD (-5) only occur while holding ctx->mutex (pmu_detach_event() -> __pmu_detach_event() -> perf_event_exit_event(..., DETACH_REVOKE) and perf_event_release_kernel()). Once REVOKED, the event is detached from ctx and can no longer be visited by event_sched_in(). Conversely, when event->state > PERF_EVENT_STATE_REVOKED, concurrent ctx->lock-only transitions can only move event->state among PERF_EVENT_STATE_INACTIVE (0), PERF_EVENT_STATE_ACTIVE (1), and PERF_EVENT_STATE_ERROR (-2), all of which are strictly greater than PERF_EVENT_STATE_REVOKED (-4).
  • As a result, perf_event_release_kernel() always takes the perf_remove_from_context(event, DETACH_GROUP|DETACH_DEAD) branch, which then acquires ctx->lock (directly or via event_function_call()) and properly serializes with event_sched_in().

Fix Direction

Use READ_ONCE(event->state) at kernel/events/core.c:5990 (consistent with other lockless/ctx->mutex-only readers of event->state in kernel/events/core.c such as perf_event_read() and perf_event_stop()), preventing KCSAN warnings and potential compiler load-tearing/reloading.

▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
c765c2c2 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in event_sched_in / perf_event_release_kernel 2026/11/19 21:13 2026/11/19 21:13 2026/11/19 21:15 229865fb
affd7ecd assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in event_sched_in / perf_event_release_kernel 2026/11/19 21:11 2026/11/19 21:11 2026/11/19 21:13 229865fb

Sample crash report:
==================================================================
BUG: KCSAN: data-race in event_sched_in / perf_event_release_kernel

write (marked) to 0xffff88811b7e2028 of 4 bytes by task 6809 on cpu 1:
 perf_event_set_state kernel/events/core.c:790 [inline]
 event_sched_in+0x360/0xc70 kernel/events/core.c:2849
 group_sched_in kernel/events/core.c:2896 [inline]
 merge_sched_in+0x252/0xbe0 kernel/events/core.c:4160
 visit_groups_merge+0x9cb/0xf70 kernel/events/core.c:4091
 pmu_groups_sched_in kernel/events/core.c:4196 [inline]
 __pmu_ctx_sched_in kernel/events/core.c:-1 [inline]
 ctx_sched_in+0x3ba/0x410 kernel/events/core.c:4275
 perf_event_sched_in kernel/events/core.c:-1 [inline]
 perf_event_context_sched_in kernel/events/core.c:4322 [inline]
 __perf_event_task_sched_in+0x69e/0xb50 kernel/events/core.c:4351
 perf_event_task_sched_in include/linux/perf_event.h:1645 [inline]
 finish_task_switch+0x2bd/0x340 kernel/sched/core.c:5430
 context_switch kernel/sched/core.c:5587 [inline]
 __schedule+0x796/0xd30 kernel/sched/core.c:7386
 preempt_schedule_irq+0x2c/0x50 kernel/sched/core.c:7666
 irqentry_exit_cond_resched kernel/entry/common.c:145 [inline]
 irqentry_exit_to_kernel_mode_preempt+0x63/0x80 include/linux/irq-entry-common.h:453
 irqentry_exit_to_kernel_mode include/linux/irq-entry-common.h:543 [inline]
 irqentry_exit+0x99/0x640 kernel/entry/common.c:154
 sysvec_apic_timer_interrupt+0x44/0x80 arch/x86/kernel/apic/apic.c:1062
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:674
 syscall_enter_from_user_mode_work include/linux/entry-common.h:142 [inline]
 do_syscall_64+0x86/0x3a0 arch/x86/entry/syscall_64.c:86
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

read to 0xffff88811b7e2028 of 4 bytes by task 6808 on cpu 0:
 perf_event_release_kernel+0x23f/0x550 kernel/events/core.c:5990
 perf_release+0x1f/0x30 kernel/events/core.c:6066
 __fput+0x292/0x660 fs/file_table.c:512
 fput_close_sync+0xd5/0x1d0 fs/file_table.c:618
 filp_close_sync+0x27/0x40 fs/open.c:1574
 __range_close fs/file.c:919 [inline]
 __do_sys_close_range fs/file.c:1003 [inline]
 __se_sys_close_range+0x765/0x880 fs/file.c:954
 __x64_sys_close_range+0x43/0x50 fs/file.c:954
 x64_sys_call+0x2198/0x2550 arch/x86/include/generated/asm/syscalls_64.h:437
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

value changed: 0x00000000 -> 0x00000001

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 6808 Comm: syz.1.662 Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
==================================================================

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/19 22:10 upstream 8dac6df92a2e 229865fb .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in event_sched_in / perf_event_release_kernel
2026/11/19 09:28 upstream 4405c18c95d2 f138241c .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in event_sched_in / perf_event_release_kernel
2026/11/18 04:15 upstream 63fdb511c27a 122f94d5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in event_sched_in / perf_event_release_kernel
2026/11/17 12:45 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in event_sched_in / perf_event_release_kernel
2026/11/17 03:38 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in event_sched_in / perf_event_release_kernel
* Struck through repros no longer work on HEAD.