Technical Mechanism & Root Cause Analysis
Background and Invariants
When split page-table locks (ALLOC_SPLIT_PTLOCKS) are enabled, each page-table page allocates its associated spinlock (ptdesc->ptl) from the page->ptl slab cache (page_ptl_cachep) via ptlock_alloc() / ptlock_init().
Because readers traversing page tables (e.g., via pte_offset_map(), pte_offset_map_ro_nolock(), and pte_offset_map_rw_nolock()) hold rcu_read_lock() rather than mmap_write_lock, page-table freeing mechanisms that run without mmap_write_lock must guarantee that page-table pages and their split locks are protected by RCU and remain valid throughout any concurrent RCU read-side critical section. For instance, THP page-table retraction (khugepaged.c) uses pte_free_defer(), which defers pte_free() and pagetable_pte_dtor() until after an RCU grace period via call_rcu().
The Defect
With CONFIG_PT_RECLAIM enabled, madvise(MADV_DONTNEED) (madvise_dontneed_single_vma() in mm/madvise.c) sets details.reclaim_pt = true while holding only mmap_read_lock. When zap_pte_range() in mm/memory.c clears all entries in a PMD range:
zap_pte_range() calls try_get_and_clear_pmd() to disconnect the PMD and then invokes free_pte(mm, start, tlb, pmdval) (mm/pt_reclaim.c:28).
free_pte() calls pte_free_tlb(tlb, pmd_pgtable(pmdval), addr).
- On x86,
pte_free_tlb() expands to __pte_free_tlb() -> ___pte_free_tlb() (arch/x86/mm/pgtable.c:61).
- While
___pte_free_tlb() enqueues the page-table page itself for deferred RCU reclamation via paravirt_tlb_remove_table(tlb, pte) -> tlb_remove_table_rcu(), it calls pagetable_pte_dtor(page_ptdesc(pte)) synchronously before enqueuing.
pagetable_pte_dtor() calls ptlock_free(ptdesc) (include/linux/mm.h:3009), which immediately executes kmem_cache_free(page_ptl_cachep, ptdesc->ptl) (mm/memory.c:7035). The page->ptl slab cache does not use SLAB_TYPESAFE_BY_RCU.
Race Condition Trigger
Concurrently, userfaultfd_move() -> move_pages() -> move_pages_pte() (mm/userfaultfd.c) executes under mmap_read_lock:
move_pages_pte() maps the source PTE page via pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, &src_ptl) (mm/userfaultfd.c:1181). This enters an RCU read-side critical section (rcu_read_lock()) and retrieves src_ptl (ptdesc->ptl).
move_pages_pte() acquires and drops src_ptl to check orig_src_pte:
spin_lock(src_ptl);
orig_src_pte = ptep_get(src_pte);
spin_unlock(src_ptl);
- While
src_ptl is temporarily released (or even before its initial acquisition) and before move_pages_pte() exits its RCU read-side critical section, the concurrent madvise(MADV_DONTNEED) worker executes free_pte() and synchronously frees src_ptl via pagetable_pte_dtor().
move_pages_pte() reaches line 1248 to re-lock the source page table:
spin_lock(src_ptl);
src_ptl points to memory in page_ptl_cachep that has already been poisoned and freed by kmem_cache_free(). When spin_lock(src_ptl) calls into lockdep (__lock_acquire()), it reads the freed memory, causing KASAN to detect a slab-use-after-free read.