syzbot
KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit
Status:
moderation: reported on 2026/11/19 00:04
Subsystems:
trace
Labels:
prio:low
race:benign
[Documentation on labels]
Reported-by: syzbot+2f56c6bdef6124061cdf@syzkaller.appspotmail.com
First crash: 1d22h, last: 1d13h
✨ AI-generated root cause summary:
The KCSAN data-race report between ring_buffer_lock_reserve() and ring_buffer_unlock_commit() is caused by two compounding factors: (1) kernel/trace/Makefile accidentally placed 'KCSAN_SANITIZE := n' inside an 'ifdef CONFIG_FUNCTION_TRACER' block, which left kernel/trace/ instrumented with KCSAN in configurations where trace events are active but CONFIG_FUNCTION_TRACER is disabled; and (2) ring_buffer_lock_reserve() and ring_buffer_unlock_commit() run with preemption disabled but hardware interrupts enabled, accessing cpu_buffer->current_context in trace_recursive_lock() and trace_recursive_unlock() using plain C operations without READ_ONCE()/WRITE_ONCE() annotations. When KCSAN placed a watchpoint delay on current_context during trace_recursive_unlock(), a local APIC timer interrupt fired on the same CPU and called trace_recursive_lock(), triggering the data-race report.
▶
▼
✨ Full AI root cause analysis
Technical Root Cause Analysis
Involved Entities
- Source Files:
- Functions:
- Structures and Fields:
struct ring_buffer_per_cpu::current_context
- Context bits:
RB_CTX_NORMAL, RB_CTX_SOFTIRQ, RB_CTX_IRQ, RB_CTX_NMI, RB_CTX_TRANSITION
Mechanism and Root Cause
The reported KCSAN data race between ring_buffer_lock_reserve() and ring_buffer_unlock_commit() is the result of two interacting defects:
-
Unintended KCSAN Instrumentation in kernel/trace/Makefile:
In commit f5d2313bd3c5 ("kcsan, trace: Make KCSAN compatible with tracing"), the intention was to disable KCSAN across the entire tracing subsystem to avoid recursion and watchdog delays during core trace path execution:
/* kernel/trace/Makefile */
ifdef CONFIG_FUNCTION_TRACER
# Avoid recursion due to instrumentation.
KCSAN_SANITIZE := n
Because KCSAN_SANITIZE := n was placed inside ifdef CONFIG_FUNCTION_TRACER, any kernel configuration where general tracing and trace events are enabled (CONFIG_TRACING=y, CONFIG_RING_BUFFER=y) but function tracing is disabled (# CONFIG_FUNCTION_TRACER is not set) leaves all object files in kernel/trace/ (including ring_buffer.o) instrumented by KCSAN. When instrumented, KCSAN sets watchpoints and injects microsecond delays (udelay() / delay_tsc()) during tracing operations.
-
Unmarked Interrupt-Concurrent Accesses in kernel/trace/ring_buffer.c:
Both ring_buffer_lock_reserve() and ring_buffer_unlock_commit() protect ring buffer modifications by disabling preemption via preempt_disable_notrace(), but they intentionally leave hardware interrupts enabled. To detect and prevent re-entrant tracing loops from nested interrupt levels (normal task -> softirq -> hardirq -> NMI), the ring buffer maintains a per-CPU bitmask cpu_buffer->current_context.
In trace_recursive_unlock():
static __always_inline void
trace_recursive_unlock(struct ring_buffer_per_cpu *cpu_buffer)
{
cpu_buffer->current_context &=
cpu_buffer->current_context - (1 << cpu_buffer->nest);
}
And in trace_recursive_lock():
static __always_inline bool
trace_recursive_lock(struct ring_buffer_per_cpu *cpu_buffer)
{
unsigned int val = cpu_buffer->current_context;
...
val |= (1 << (bit + cpu_buffer->nest));
cpu_buffer->current_context = val;
return false;
}
These accesses to cpu_buffer->current_context are plain C memory operations rather than volatile or atomic reads/writes (READ_ONCE() / WRITE_ONCE()).
How the Bug is Triggered
- Task context on CPU 0 calls
kfree(), which invokes the trace_kfree() tracepoint.
trace_kfree() calls ring_buffer_unlock_commit() -> trace_recursive_unlock().
- In
trace_recursive_unlock(), because kernel/trace/ is compiled with KCSAN instrumentation, KCSAN intercepts the write/read-write to cpu_buffer->current_context and sets up a watchpoint, inserting a delay loop (delay_tsc()).
- While the task is stalled inside
delay_tsc() with interrupts enabled, an APIC timer interrupt (sysvec_apic_timer_interrupt) fires on CPU 0.
- The timer interrupt handler executes
trace_local_timer_entry(), which calls ring_buffer_lock_reserve() -> trace_recursive_lock().
trace_recursive_lock() performs a plain read from cpu_buffer->current_context (unsigned int val = cpu_buffer->current_context;).
- KCSAN detects that
0xffff88810007a838 was concurrently read by the interrupt handler while a watchpoint was pending for the write by the interrupted task on the same CPU, triggering the data-race report.
Fix Considerations
- In
kernel/trace/Makefile, move KCSAN_SANITIZE := n outside the ifdef CONFIG_FUNCTION_TRACER block so that the tracer subsystem is uniformly exempt from KCSAN instrumentation regardless of whether CONFIG_FUNCTION_TRACER is enabled.
- In
kernel/trace/ring_buffer.c, use READ_ONCE() and WRITE_ONCE() (or appropriate single-load expressions) when reading and writing cpu_buffer->current_context in trace_recursive_lock(), trace_recursive_unlock(), and ring_buffer_swap_cpu() to prevent compiler load tearing or duplicate fetches across nested interrupt boundaries.
Sample crash report:
* Struck through repros no longer work on HEAD.