Root Cause Analysis

Involved Components

• Source File: net/xfrm/xfrm_iptfs.c (called into net/xfrm/xfrm_output.c and include/net/dst.h)
• Functions: iptfs_output_collect(), iptfs_enqueue(), iptfs_delay_timer(), iptfs_output_queued(), xfrm_output_one(), skb_dst_force()
• Data Structures / Fields: struct sk_buff::_skb_refdst, struct dst_entry, struct xfrm_iptfs_data::queue, struct xfrm_iptfs_data::iptfs_timer, SKB_DST_NOREF

Technical Mechanism

1. IP-TFS Ingress and Output Collection: In IPsec IP-TFS (AggFrag / IP Tunnel Framing, RFC 9347), iptfs_output_collect() is assigned as the dst->output callback for the XFRM bundle (dst1->output = xfrm[i]->mode_cbs->output; in net/xfrm/xfrm_policy.c). Outgoing packets routed through the IP-TFS state enter iptfs_output_collect() synchronously from the local network stack under an active RCU read-side critical section (e.g., via dst_output() in ip_output() or ip6_output()).

2. Unrefcounted Destination Entries (SKB_DST_NOREF): Packets arriving at dst->output may carry an unrefcounted destination entry where skb->_skb_refdst has the SKB_DST_NOREF bit set. Under Linux networking invariants, as long as skb->_skb_refdst has SKB_DST_NOREF, the pointed-to struct dst_entry is only guaranteed to remain valid for the duration of the current RCU read-side critical section. Whenever an SKB escapes an RCU-protected region—such as when stored on an internal queue across asynchronous boundaries or timers—the subsystem must invoke skb_dst_force(skb) while still inside the RCU read-side critical section to safely convert the noref pointer into a true reference (dst_hold_safe(dst)).

3. Missing skb_dst_force() in iptfs_output_collect(): In iptfs_output_collect(), after breaking apart GSO packets, the code loops over the segments and directly enqueues them via iptfs_enqueue():

   /* Enqueue to send in tunnel */
   ok = iptfs_enqueue(xtfs, skb);
   

   Neither iptfs_output_collect() nor iptfs_enqueue() calls skb_dst_force(skb). The function then arms the aggregation delay hrtimer xtfs->iptfs_timer and returns 0 to the caller. At this point, the transmitting thread exits its RCU read-side critical section, leaving the queued SKBs in xtfs->queue referencing unpinned dst_entry structures.

4. Deferred Execution and Warning Trigger: When xtfs->iptfs_timer expires, iptfs_delay_timer() executes in softirq context (hrtimer_run_softirq), where rcu_read_lock() is not held. It dequeues the packets and calls iptfs_output_queued() -> xfrm_output(NULL, skb) -> xfrm_output_resume() -> xfrm_output_one().

   In net/xfrm/xfrm_output.c:549:

   skb_dst_force(skb);
   

   skb_dst_force() (defined in include/net/dst.h:321) checks:

   if (skb_dst_is_noref(skb)) {
       struct dst_entry *dst = skb_dst(skb);
   
       WARN_ON(!rcu_read_lock_held());
       if (!dst_hold_safe(dst))
           dst = NULL;
       ...
   }
   

   Because skb->_skb_refdst still carries SKB_DST_NOREF, execution reaches line 326. Since softirq execution does not hold rcu_read_lock(), !rcu_read_lock_held() evaluates to true and fires the warning splat. Furthermore, if an RCU grace period has elapsed while the packet was queued in xtfs->queue (for example, due to a route change, netdevice down, or policy deletion), the destination entry may already have been freed, causing dst_hold_safe() to touch freed memory.