Technical Analysis of the Root Cause
Overview and Context
The reported bug is a lockdep warning (WARNING: syz... still has locks held!) triggered at process exit (do_exit() -> debug_check_no_locks_held()), reporting that &q->q_usage_counter(queue) (q->q_lockdep_map) acquired by blk_mq_freeze_queue_nomemsave() at block/blk-mq.c:206 was never released by task syz.8.2692/13204.
The task was exiting from syscall ioctl(fd, DM_TABLE_LOAD, ...) (command 0xc138fd09) which returned -ENXIO (RAX = 0xfffffffffffffffa).
Affected Files, Structs, and Functions
- Files:
block/blk-mq.c, block/blk.h
- Structs:
struct request_queue
- Fields:
q->mq_freeze_lock, q->mq_freeze_depth, q->mq_freeze_owner, q->mq_freeze_queue_dying, q->mq_freeze_disk_dead, q->q_lockdep_map, q->io_lockdep_map
- Functions:
blk_freeze_queue_start(), __blk_freeze_queue_start(), blk_freeze_set_owner(), blk_freeze_acquire_lock(), blk_mq_unfreeze_queue_nomemrestore(), __blk_mq_unfreeze_queue(), blk_unfreeze_release_lock()
Mechanism of the Defect
1. Asymmetric Locking in Queue Freeze / Unfreeze
In block/blk-mq.c, blk_freeze_queue_start() calls __blk_freeze_queue_start() followed by blk_freeze_acquire_lock():
void blk_freeze_queue_start(struct request_queue *q)
{
if (__blk_freeze_queue_start(q, current))
blk_freeze_acquire_lock(q);
}
Under q->mq_freeze_lock, blk_freeze_set_owner() checks whether !q->mq_freeze_depth and records state:
if (!q->mq_freeze_depth) {
q->mq_freeze_owner = owner;
q->mq_freeze_owner_depth = 1;
q->mq_freeze_disk_dead = !q->disk ||
test_bit(GD_DEAD, &q->disk->state) ||
!blk_queue_registered(q);
q->mq_freeze_queue_dying = blk_queue_dying(q);
return true;
}
When blk_freeze_acquire_lock(q) runs (block/blk.h):
static inline void blk_freeze_acquire_lock(struct request_queue *q)
{
if (!q->mq_freeze_disk_dead)
rwsem_acquire(&q->io_lockdep_map, 0, 1, _RET_IP_);
if (!q->mq_freeze_queue_dying)
rwsem_acquire(&q->q_lockdep_map, 0, 1, _RET_IP_);
}
If !blk_queue_registered(q) is true (e.g. on an allocated device queue before add_disk() completes registration, such as during dm_setup_md_queue()) and blk_queue_dying(q) is false, q->mq_freeze_disk_dead is set to true and q->mq_freeze_queue_dying is false. Thus, only q->q_lockdep_map is acquired.
2. Race Condition on Queue Unfreeze
In blk_mq_unfreeze_queue_nomemrestore(q):
void blk_mq_unfreeze_queue_nomemrestore(struct request_queue *q)
{
if (__blk_mq_unfreeze_queue(q, false))
blk_unfreeze_release_lock(q);
}
Inside __blk_mq_unfreeze_queue(q, false):
mutex_lock(&q->mq_freeze_lock);
...
q->mq_freeze_depth--;
...
unfreeze = blk_unfreeze_check_owner(q);
mutex_unlock(&q->mq_freeze_lock);
return unfreeze;
q->mq_freeze_lock is released as soon as q->mq_freeze_depth drops to 0. Outside q->mq_freeze_lock, blk_unfreeze_release_lock(q) is called:
static inline void blk_unfreeze_release_lock(struct request_queue *q)
{
if (!q->mq_freeze_queue_dying)
rwsem_release(&q->q_lockdep_map, _RET_IP_);
if (!q->mq_freeze_disk_dead)
rwsem_release(&q->io_lockdep_map, _RET_IP_);
}
Because blk_unfreeze_release_lock(q) runs after q->mq_freeze_lock has been unlocked and q->mq_freeze_depth is already 0:
- Thread 1 (
DM_TABLE_LOAD) unfreezes md->queue. __blk_mq_unfreeze_queue() decrements q->mq_freeze_depth to 0, clears q->mq_freeze_owner, and releases q->mq_freeze_lock.
- Before Thread 1 can execute
blk_unfreeze_release_lock(q) (e.g. due to preemption under PREEMPT(full) or SMP concurrency), Thread 2 (DM_DEV_REMOVE or DM_REMOVE_ALL) runs concurrently.
- Thread 2 removes the device from the hash cell (
__hash_remove() -> dm_set_mdptr(hc->md, NULL)), drops _hash_lock, and calls dm_destroy(md) -> __dm_destroy(md, true).
- In
__dm_destroy(), blk_mark_disk_dead(md->disk) is called (block/genhd.c:689):
- Sets
QUEUE_FLAG_DYING on q.
- Calls
__blk_mark_disk_dead() -> blk_queue_start_drain(q) -> __blk_freeze_queue_start(q, current).
- In
__blk_freeze_queue_start(), Thread 2 acquires q->mq_freeze_lock. Since q->mq_freeze_depth == 0, blk_freeze_set_owner() executes and updates:
q->mq_freeze_queue_dying = blk_queue_dying(q); /* now true! */
- Thread 1 resumes and executes
blk_unfreeze_release_lock(q). It reads q->mq_freeze_queue_dying, which is now true.
- Because
!q->mq_freeze_queue_dying evaluates to false, Thread 1 skips rwsem_release(&q->q_lockdep_map, _RET_IP_).
- Thread 1 finishes its operation and proceeds to stage the new table in
drivers/md/dm-ioctl.c:table_load(). It finds dm_get_mdptr(md) == NULL (cleared by Thread 2) and returns -ENXIO.
- Upon returning toward userspace, Thread 1 receives a signal, aborts the syscall, and calls
do_group_exit() -> do_exit(). In debug_check_no_locks_held(), lockdep detects that &q->q_usage_counter(queue) acquired during the earlier freeze remains held, triggering the warning.
Summary of the Invariant Violation
The lockdep release logic in blk_unfreeze_release_lock() relies on mutable queue-global flags (q->mq_freeze_queue_dying and q->mq_freeze_disk_dead) read outside of q->mq_freeze_lock after q->mq_freeze_depth has already been reduced to zero. A concurrent caller starting a new freeze can overwrite these flags before the unfreezing caller releases the lockdep map, resulting in skipped lock releases and leaked lockdep tracking state.