syzbot
kernel BUG in nvmet_fc_remove_port
Status:
upstream: reported C repro on 2026/10/27 12:08
Subsystems:
nvme
Labels:
prio:normal
[Documentation on labels]
Reported-by: syzbot+df0058a8d0bdce7c4059@syzkaller.appspotmail.com
First crash: 4d13h, last: 4d11h
✨ AI-generated root cause summary:
In the NVMe target subsystem, port attribute store handlers in drivers/nvme/target/configfs.c (such as nvmet_addr_trtype_store) do not acquire nvmet_config_sem and only check port->enabled. Because port->enabled is not set to true until after ops->add_port() finishes in nvmet_enable_port(), a concurrent write to addr_trtype can race and switch port->disc_addr.trtype (e.g., from TCP to FC). When disabling the port, nvmet_disable_port() in drivers/nvme/target/core.c indexes nvmet_transports using port->disc_addr.trtype instead of using port->tr_ops, causing type confusion on port->priv (treating struct nvmet_tcp_port as struct nvmet_fc_port_entry) and resulting in memory corruption, refcount underflow, and use-after-free.
▶
▼
✨ Full AI root cause analysis
Root Cause Analysis
1. Functions, Structs, and Files Involved
drivers/nvme/target/configfs.c:
nvmet_addr_trtype_store(): Configfs .store callback for /config/nvmet/ports/<id>/addr_trtype.
nvmet_is_port_enabled(): Helper that checks p->enabled to prevent modifying port attributes while the port is enabled.
nvmet_port_subsys_allow_link(): Creates a symlink between a subsystem and a port; calls nvmet_enable_port() if the port's subsystems list is empty.
nvmet_port_subsys_drop_link(): Removes the symlink; calls nvmet_disable_port() when the subsystems list becomes empty.
drivers/nvme/target/core.c:
nvmet_enable_port(): Enables a port using ops = nvmet_transports[port->disc_addr.trtype], calls ops->add_port(port), and at the end sets port->enabled = true and port->tr_ops = ops.
nvmet_disable_port(): Disables a port, but looks up ops = nvmet_transports[port->disc_addr.trtype] instead of using port->tr_ops.
drivers/nvme/target/tcp.c:
nvmet_tcp_add_port(): Allocates and initializes struct nvmet_tcp_port, sets port->sock, and assigns nport->priv = port.
drivers/nvme/target/fc.c:
nvmet_fc_remove_port(): Expects port->priv to be struct nvmet_fc_port_entry *pe and calls nvmet_fc_portentry_unbind(pe).
nvmet_fc_portentry_unbind(): Calls nvmet_fc_tgtport_put(pe->tgtport), sets pe->tgtport->pe = NULL, and performs list_del(&pe->pe_list).
2. Root Cause & Trigger Mechanism
- When linking a subsystem to a port via configfs (
nvmet_port_subsys_allow_link()), the semaphore nvmet_config_sem is acquired and nvmet_enable_port(port) is invoked.
- In
nvmet_enable_port(), ops is looked up from nvmet_transports[port->disc_addr.trtype] (here, TCP). Then ops->add_port(port) (nvmet_tcp_add_port()) is executed. Inside nvmet_tcp_add_port(), a struct nvmet_tcp_port is allocated and assigned to port->priv.
- Note that
port->enabled is only set to true at the very end of nvmet_enable_port(), after ops->add_port(port) completes.
- Meanwhile, port attribute
.store handlers in drivers/nvme/target/configfs.c (such as nvmet_addr_trtype_store()) do not acquire nvmet_config_sem and rely solely on nvmet_is_port_enabled(port) (which tests port->enabled).
- A concurrent write to
addr_trtype (e.g. writing "fc\n") races with nvmet_enable_port(), sees port->enabled == false, and mutates port->disc_addr.trtype to NVMF_TRTYPE_FC.
- When the port is later disabled via
nvmet_port_subsys_drop_link() -> nvmet_disable_port(), nvmet_disable_port() does not use port->tr_ops (which was saved as TCP), but instead retrieves the transport operations using ops = nvmet_transports[port->disc_addr.trtype]. Since port->disc_addr.trtype was changed to NVMF_TRTYPE_FC, it invokes nvmet_fc_remove_port().
nvmet_fc_remove_port() casts port->priv to struct nvmet_fc_port_entry *pe, leading to severe type confusion:
- At offset 0 in
struct nvmet_fc_port_entry, pe->tgtport is read. In struct nvmet_tcp_port, offset 0 contains sock (struct socket *).
nvmet_fc_portentry_unbind(pe) calls nvmet_fc_tgtport_put(pe->tgtport), which calls kref_put(&tgtport->ref, ...) on the memory inside the struct socket structure at the offset of tgtport->ref. Because that value is not a valid kref, refcount underflow is triggered.
- It also executes
pe->tgtport->pe = NULL (writing NULL into struct socket) and list_del(&pe->pe_list) (operating on pointers at offset 32 of struct nvmet_tcp_port, inside its accept_work structure).
- Finally,
kfree(pe) frees the struct nvmet_tcp_port directly without proper TCP socket cleanup.
3. Production Runtime Consequences & Schema Values
- MemoryCorruption (
true): Type confusion causes invalid pointer writes (pe->tgtport->pe = NULL), linked list corruption (list_del), memory corruption within the socket object, and invalid/use-after-free conditions (kfree(pe) freeing a live struct nvmet_tcp_port).
- InfoLeak (
none): The bug corrupts memory and crashes the kernel during teardown; no uninitialized or out-of-bounds kernel memory is copied or exposed to userspace.
- RawDenialOfService (
true): In a production kernel without debug sanitizers, writing through wild pointers and performing list_del on arbitrary data results in kernel panics, page faults on non-canonical/invalid addresses, or severe data/socket structure corruption.
- IsNetCandidate (
true): The bug directly involves the NVMe-oF TCP transport subsystem, socket allocation, and listening port setup (sock_create, kernel_bind, kernel_listen).
- IsKVMCandidate (
false): No hypervisor, KVM, or virtualization interfaces are involved.
Sample crash report:
nvmet_tcp: enabling port 1 (127.0.0.1:4420)
refcount_t: underflow; use-after-free.
------------[ cut here ]------------
kernel BUG at lib/refcount.c:37!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
CPU: 0 UID: 0 PID: 6113 Comm: syz.0.43 Not tainted syzkaller #0 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:refcount_warn_saturate+0xd7/0x110 lib/refcount.c:37
Code: 67 48 0f b9 3a e8 b9 e8 d6 fc 5b 5d e9 c2 d6 b5 06 77 19 83 fd 02 74 3e e8 a6 e8 d6 fc 48 c7 c7 c0 08 81 8c e8 ba 43 eb fb 90 <0f> 0b e8 92 e8 d6 fc 48 8d 3d 6b 27 3d 0d 67 48 0f b9 3a e8 81 e8
RSP: 0018:ffffc900039d7c00 EFLAGS: 00010086
RAX: 0000000000000026 RBX: ffff8880549e4678 RCX: 0000000000000000
RDX: 0000000000000026 RSI: ffffffff81ebe399 RDI: fffff5200073af71
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: 1ffff9200073af85
R13: 0000000000000000 R14: 0000000000000000 R15: ffff8880549e4678
FS: 0000555566238500(0000) GS:ffff8880d449e000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8d01471220 CR3: 000000002d14f000 CR4: 0000000000352ef0
Call Trace:
<TASK>
__refcount_sub_and_test include/linux/refcount.h:400 [inline]
__refcount_dec_and_test include/linux/refcount.h:432 [inline]
refcount_dec_and_test include/linux/refcount.h:450 [inline]
kref_put include/linux/kref.h:64 [inline]
nvmet_fc_tgtport_put drivers/nvme/target/fc.c:1464 [inline]
nvmet_fc_portentry_unbind drivers/nvme/target/fc.c:1278 [inline]
nvmet_fc_remove_port+0x4c2/0x5f0 drivers/nvme/target/fc.c:2930
nvmet_disable_port+0x13c/0x200 drivers/nvme/target/core.c:400
nvmet_port_subsys_drop_link+0x26b/0x310 drivers/nvme/target/configfs.c:1189
configfs_unlink+0x3cc/0x800 fs/configfs/symlink.c:258
vfs_unlink+0x2f0/0xbd0 fs/namei.c:5661
filename_unlinkat+0x408/0x730 fs/namei.c:5731
__do_sys_unlinkat fs/namei.c:5760 [inline]
__se_sys_unlinkat fs/namei.c:5752 [inline]
__x64_sys_unlinkat+0xc0/0x130 fs/namei.c:5752
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f8d0159f559
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffef6d7e878 EFLAGS: 00000246 ORIG_RAX: 0000000000000107
RAX: ffffffffffffffda RBX: 00007f8d01825fa0 RCX: 00007f8d0159f559
RDX: 0000000000000000 RSI: 0000200000000780 RDI: ffffffffffffff9c
RBP: 00007f8d01636b20 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f8d01825fac R14: 00007f8d01825fa0 R15: 00007f8d01825fa0
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:refcount_warn_saturate+0xd7/0x110 lib/refcount.c:37
Code: 67 48 0f b9 3a e8 b9 e8 d6 fc 5b 5d e9 c2 d6 b5 06 77 19 83 fd 02 74 3e e8 a6 e8 d6 fc 48 c7 c7 c0 08 81 8c e8 ba 43 eb fb 90 <0f> 0b e8 92 e8 d6 fc 48 8d 3d 6b 27 3d 0d 67 48 0f b9 3a e8 81 e8
RSP: 0018:ffffc900039d7c00 EFLAGS: 00010086
RAX: 0000000000000026 RBX: ffff8880549e4678 RCX: 0000000000000000
RDX: 0000000000000026 RSI: ffffffff81ebe399 RDI: fffff5200073af71
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000001 R12: 1ffff9200073af85
R13: 0000000000000000 R14: 0000000000000000 R15: ffff8880549e4678
FS: 0000555566238500(0000) GS:ffff8880d449e000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8d01471220 CR3: 000000002d14f000 CR4: 0000000000352ef0
* Struck through repros no longer work on HEAD.