| 0/0 |
2026/08/08 03:06 |
flow |
patching |
1h43m
Results: map[AckedBy:[] Fixes:map[AuthorEmail:fdmanana@suse.com AuthorName:Filipe Manana Hash:79bd37120b149532af5b21953643ed74af69654f Title:btrfs: rework chunk allocation to avoid exhaustion of the system chunk array] KernelBranch:master KernelCommit:075b74841bd0065a3bda3440873c747938e69b68 KernelRepo:git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git PatchDescription:btrfs: check system chunk space before removing device extents
During a chunk relocation/balance operation, a transaction abort with
-ENOSPC can occur in btrfs_remove_chunk(). This happens because the
filesystem runs out of system space while trying to remove a chunk item
from the chunk tree, and it cannot allocate a new system chunk.
The crash trace is as follows:
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
</TASK>
The sequence of events leading to the abort is as follows. First,
btrfs_remove_chunk() removes the device extents from the device tree by
calling btrfs_remove_dev_extents(). It then calls check_system_chunk() to
reserve system space for removing the chunk item from the chunk tree. If
the system space is full, check_system_chunk() attempts to allocate a new
system chunk. To allocate a new chunk, find_free_dev_extent() is called.
However, find_free_dev_extent() searches the commit root of the device tree
to prevent reusing space freed in the current transaction. Because the
device extents were just freed in the current transaction by
btrfs_remove_dev_extents(), find_free_dev_extent() does not see this freed
space. If the device is otherwise full, it returns -ENOSPC.
check_system_chunk() ignores the failure to allocate a system chunk and
does not reserve any space. Finally, remove_chunk_item() is called, which
fails with -ENOSPC because it needs to COW a node in the chunk tree but no
system space was reserved. Since btrfs_remove_dev_extents() has already
modified the device tree, btrfs_remove_chunk() cannot fail gracefully and
is forced to abort the transaction.
To fix this, we must ensure that we have enough system space before
modifying the device tree. We change check_system_chunk() and
reserve_chunk_space() to return an int error code. In btrfs_remove_chunk(),
we call check_system_chunk() early. If it fails, we can fail gracefully and
return -ENOSPC before any irreversible tree modifications occur. If it
succeeds, we immediately release the reserved metadata space using
btrfs_trans_release_chunk_metadata(). This is necessary because
btrfs_remove_dev_extents() can trigger nested chunk allocations which
unconditionally release all chunk metadata reservations. We then proceed to
call check_system_chunk() a second time later in the function to make the
actual reservation, which is now guaranteed to succeed since we ensured the
system space_info has enough free space. PatchDiff:diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
Recipients:[map[Email:clm@fb.com Name:Chris Mason To:true] map[Email:dsterba@suse.com Name:David Sterba To:true] map[Email:linux-btrfs@vger.kernel.org Name: To:true] map[Email:linux-kernel@vger.kernel.org Name: To:false] map[Email:fdmanana@suse.com Name:Filipe Manana To:true]] ReportedBy:[] ReviewedBy:[] SuggestedBy:[] TestedBy:[]]
|
| 1/1 |
2026/08/08 03:06 |
action |
base-commit-picker |
0m
Results: map[KernelBranch:master KernelCommit:075b74841bd0065a3bda3440873c747938e69b68 KernelRepo:git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git]
|
| 2/1 |
2026/08/08 03:06 |
action |
syz-repro-to-c-repro |
0m
Results: map[SimplifiedCRepro:// autogenerated by syzkaller (https://github.com/google/syzkaller)
#define _GNU_SOURCE
#include <endian.h>
#include <errno.h>
#include <fcntl.h>
#include <setjmp.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <linux/loop.h>
#ifndef __NR_memfd_create
#define __NR_memfd_create 319
#endif
static unsigned long long procid;
//% This code is derived from puff.{c,h}, found in the zlib development. The
//% original files come with the following copyright notice:
//% Copyright (C) 2002-2013 Mark Adler, all rights reserved
//% version 2.3, 21 Jan 2013
//% This software is provided 'as-is', without any express or implied
//% warranty. In no event will the author be held liable for any damages
//% arising from the use of this software.
//% Permission is granted to anyone to use this software for any purpose,
//% including commercial applications, and to alter it and redistribute it
//% freely, subject to the following restrictions:
//% 1. The origin of this software must not be misrepresented; you must not
//% claim that you wrote the original software. If you use this software
//% in a product, an acknowledgment in the product documentation would be
//% appreciated but is not required.
//% 2. Altered source versions must be plainly marked as such, and must not be
//% misrepresented as being the original software.
//% 3. This notice may not be removed or altered from any source distribution.
//% Mark Adler madler@alumni.caltech.edu
//% BEGIN CODE DERIVED FROM puff.{c,h}
#define MAXBITS 15
#define MAXLCODES 286
#define MAXDCODES 30
#define MAXCODES (MAXLCODES + MAXDCODES)
#define FIXLCODES 288
struct puff_state {
unsigned char* out;
unsigned long outlen;
unsigned long outcnt;
const unsigned char* in;
unsigned long inlen;
unsigned long incnt;
int bitbuf;
int bitcnt;
jmp_buf env;
};
static int puff_bits(struct puff_state* s, int need)
{
long val = s->bitbuf;
while (s->bitcnt < need) {
if (s->incnt == s->inlen)
longjmp(s->env, 1);
val |= (long)(s->in[s->incnt++]) << s->bitcnt;
s->bitcnt += 8;
}
s->bitbuf = (int)(val >> need);
s->bitcnt -= need;
return (int)(val & ((1L << need) - 1));
}
static int puff_stored(struct puff_state* s)
{
s->bitbuf = 0;
s->bitcnt = 0;
if (s->incnt + 4 > s->inlen)
return 2;
unsigned len = s->in[s->incnt++];
len |= s->in[s->incnt++] << 8;
if (s->in[s->incnt++] != (~len & 0xff) ||
s->in[s->incnt++] != ((~len >> 8) & 0xff))
return -2;
if (s->incnt + len > s->inlen)
return 2;
if (s->outcnt + len > s->outlen)
return 1;
for (; len--; s->outcnt++, s->incnt++) {
if (s->in[s->incnt])
s->out[s->outcnt] = s->in[s->incnt];
}
return 0;
}
struct puff_huffman {
short* count;
short* symbol;
};
static int puff_decode(struct puff_state* s, const struct puff_huffman* h)
{
int first = 0;
int index = 0;
int bitbuf = s->bitbuf;
int left = s->bitcnt;
int code = first = index = 0;
int len = 1;
short* next = h->count + 1;
while (1) {
while (left--) {
code |= bitbuf & 1;
bitbuf >>= 1;
int count = *next++;
if (code - count < first) {
s->bitbuf = bitbuf;
s->bitcnt = (s->bitcnt - len) & 7;
return h->symbol[index + (code - first)];
}
index += count;
first += count;
first <<= 1;
code <<= 1;
len++;
}
left = (MAXBITS + 1) - len;
if (left == 0)
break;
if (s->incnt == s->inlen)
longjmp(s->env, 1);
bitbuf = s->in[s->incnt++];
if (left > 8)
left = 8;
}
return -10;
}
static int puff_construct(struct puff_huffman* h, const short* length, int n)
{
int len;
for (len = 0; len <= MAXBITS; len++)
h->count[len] = 0;
int symbol;
for (symbol = 0; symbol < n; symbol++)
(h->count[length[symbol]])++;
if (h->count[0] == n)
return 0;
int left = 1;
for (len = 1; len <= MAXBITS; len++) {
left <<= 1;
left -= h->count[len];
if (left < 0)
return left;
}
short offs[MAXBITS + 1];
offs[1] = 0;
for (len = 1; len < MAXBITS; len++)
offs[len + 1] = offs[len] + h->count[len];
for (symbol = 0; symbol < n; symbol++)
if (length[symbol] != 0)
h->symbol[offs[length[symbol]]++] = symbol;
return left;
}
static int puff_codes(struct puff_state* s,
const struct puff_huffman* lencode,
const struct puff_huffman* distcode)
{
static const short lens[29] = {
3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31,
35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258};
static const short lext[29] = {
0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2,
3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0};
static const short dists[30] = {
1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193,
257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145,
8193, 12289, 16385, 24577};
static const short dext[30] = {
0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6,
7, 7, 8, 8, 9, 9, 10, 10, 11, 11,
12, 12, 13, 13};
int symbol;
do {
symbol = puff_decode(s, lencode);
if (symbol < 0)
return symbol;
if (symbol < 256) {
if (s->outcnt == s->outlen)
return 1;
if (symbol)
s->out[s->outcnt] = symbol;
s->outcnt++;
} else if (symbol > 256) {
symbol -= 257;
if (symbol >= 29)
return -10;
int len = lens[symbol] + puff_bits(s, lext[symbol]);
symbol = puff_decode(s, distcode);
if (symbol < 0)
return symbol;
unsigned dist = dists[symbol] + puff_bits(s, dext[symbol]);
if (dist > s->outcnt)
return -11;
if (s->outcnt + len > s->outlen)
return 1;
while (len--) {
if (dist <= s->outcnt && s->out[s->outcnt - dist])
s->out[s->outcnt] = s->out[s->outcnt - dist];
s->outcnt++;
}
}
} while (symbol != 256);
return 0;
}
static int puff_fixed(struct puff_state* s)
{
static int virgin = 1;
static short lencnt[MAXBITS + 1], lensym[FIXLCODES];
static short distcnt[MAXBITS + 1], distsym[MAXDCODES];
static struct puff_huffman lencode, distcode;
if (virgin) {
lencode.count = lencnt;
lencode.symbol = lensym;
distcode.count = distcnt;
distcode.symbol = distsym;
short lengths[FIXLCODES];
int symbol;
for (symbol = 0; symbol < 144; symbol++)
lengths[symbol] = 8;
for (; symbol < 256; symbol++)
lengths[symbol] = 9;
for (; symbol < 280; symbol++)
lengths[symbol] = 7;
for (; symbol < FIXLCODES; symbol++)
lengths[symbol] = 8;
puff_construct(&lencode, lengths, FIXLCODES);
for (symbol = 0; symbol < MAXDCODES; symbol++)
lengths[symbol] = 5;
puff_construct(&distcode, lengths, MAXDCODES);
virgin = 0;
}
return puff_codes(s, &lencode, &distcode);
}
static int puff_dynamic(struct puff_state* s)
{
static const short order[19] =
{16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15};
int nlen = puff_bits(s, 5) + 257;
int ndist = puff_bits(s, 5) + 1;
int ncode = puff_bits(s, 4) + 4;
if (nlen > MAXLCODES || ndist > MAXDCODES)
return -3;
short lengths[MAXCODES];
int index;
for (index = 0; index < ncode; index++)
lengths[order[index]] = puff_bits(s, 3);
for (; index < 19; index++)
lengths[order[index]] = 0;
short lencnt[MAXBITS + 1], lensym[MAXLCODES];
struct puff_huffman lencode = {lencnt, lensym};
int err = puff_construct(&lencode, lengths, 19);
if (err != 0)
return -4;
index = 0;
while (index < nlen + ndist) {
int symbol;
int len;
symbol = puff_decode(s, &lencode);
if (symbol < 0)
return symbol;
if (symbol < 16)
lengths[index++] = symbol;
else {
len = 0;
if (symbol == 16) {
if (index == 0)
return -5;
len = lengths[index - 1];
symbol = 3 + puff_bits(s, 2);
} else if (symbol == 17)
symbol = 3 + puff_bits(s, 3);
else
symbol = 11 + puff_bits(s, 7);
if (index + symbol > nlen + ndist)
return -6;
while (symbol--)
lengths[index++] = len;
}
}
if (lengths[256] == 0)
return -9;
err = puff_construct(&lencode, lengths, nlen);
if (err && (err < 0 || nlen != lencode.count[0] + lencode.count[1]))
return -7;
short distcnt[MAXBITS + 1], distsym[MAXDCODES];
struct puff_huffman distcode = {distcnt, distsym};
err = puff_construct(&distcode, lengths + nlen, ndist);
if (err && (err < 0 || ndist != distcode.count[0] + distcode.count[1]))
return -8;
return puff_codes(s, &lencode, &distcode);
}
static int puff(
unsigned char* dest,
unsigned long* destlen,
const unsigned char* source,
unsigned long sourcelen)
{
struct puff_state s = {
.out = dest,
.outlen = *destlen,
.outcnt = 0,
.in = source,
.inlen = sourcelen,
.incnt = 0,
.bitbuf = 0,
.bitcnt = 0,
};
int err;
if (setjmp(s.env) != 0)
err = 2;
else {
int last;
do {
last = puff_bits(&s, 1);
int type = puff_bits(&s, 2);
err = type == 0 ? puff_stored(&s) : (type == 1 ? puff_fixed(&s) : (type == 2 ? puff_dynamic(&s) : -1));
if (err != 0)
break;
} while (!last);
}
*destlen = s.outcnt;
return err;
}
//% END CODE DERIVED FROM puff.{c,h}
#define ZLIB_HEADER_WIDTH 2
static int puff_zlib_to_file(const unsigned char* source, unsigned long sourcelen, int dest_fd)
{
if (sourcelen < ZLIB_HEADER_WIDTH)
return 0;
source += ZLIB_HEADER_WIDTH;
sourcelen -= ZLIB_HEADER_WIDTH;
const unsigned long max_destlen = 132 << 20;
void* ret = mmap(0, max_destlen, PROT_WRITE | PROT_READ, MAP_PRIVATE | MAP_ANON, -1, 0);
if (ret == MAP_FAILED)
return -1;
unsigned char* dest = (unsigned char*)ret;
unsigned long destlen = max_destlen;
int err = puff(dest, &destlen, source, sourcelen);
if (err) {
munmap(dest, max_destlen);
errno = -err;
return -1;
}
if (write(dest_fd, dest, destlen) != (ssize_t)destlen) {
munmap(dest, max_destlen);
return -1;
}
return munmap(dest, max_destlen);
}
static int setup_loop_device(unsigned char* data, unsigned long size, const char* loopname, int* loopfd_p)
{
int err = 0, loopfd = -1;
int memfd = syscall(__NR_memfd_create, "syzkaller", 0);
if (memfd == -1) {
err = errno;
goto error;
}
if (puff_zlib_to_file(data, size, memfd)) {
err = errno;
goto error_close_memfd;
}
loopfd = open(loopname, O_RDWR);
if (loopfd == -1) {
err = errno;
goto error_close_memfd;
}
if (ioctl(loopfd, LOOP_SET_FD, memfd)) {
if (errno != EBUSY) {
err = errno;
goto error_close_loop;
}
ioctl(loopfd, LOOP_CLR_FD, 0);
usleep(1000);
if (ioctl(loopfd, LOOP_SET_FD, memfd)) {
err = errno;
goto error_close_loop;
}
}
close(memfd);
*loopfd_p = loopfd;
return 0;
error_close_loop:
close(loopfd);
error_close_memfd:
close(memfd);
error:
errno = err;
return -1;
}
static void reset_loop_device(const char* loopname)
{
int loopfd = open(loopname, O_RDWR);
if (loopfd == -1) {
return;
}
if (ioctl(loopfd, LOOP_CLR_FD, 0)) {
}
close(loopfd);
}
static long syz_mount_image(
volatile long fsarg,
volatile long dir,
volatile long flags,
volatile long optsarg,
volatile long change_dir,
volatile unsigned long size,
volatile long image)
{
unsigned char* data = (unsigned char*)image;
int res = -1, err = 0, need_loop_device = !!size;
char* mount_opts = (char*)optsarg;
char* target = (char*)dir;
char* fs = (char*)fsarg;
char* source = NULL;
char loopname[64];
if (need_loop_device) {
int loopfd;
memset(loopname, 0, sizeof(loopname));
snprintf(loopname, sizeof(loopname), "/dev/loop%llu", procid);
if (setup_loop_device(data, size, loopname, &loopfd) == -1)
return -1;
close(loopfd);
source = loopname;
}
mkdir(target, 0777);
char opts[256];
memset(opts, 0, sizeof(opts));
if (strlen(mount_opts) > (sizeof(opts) - 32)) {
}
strncpy(opts, mount_opts, sizeof(opts) - 32);
if (strcmp(fs, "iso9660") == 0) {
flags |= MS_RDONLY;
} else if (strncmp(fs, "ext", 3) == 0) {
bool has_remount_ro = false;
char* remount_ro_start = strstr(opts, "errors=remount-ro");
if (remount_ro_start != NULL) {
char after = *(remount_ro_start + strlen("errors=remount-ro"));
char before = remount_ro_start == opts ? '\0' : *(remount_ro_start - 1);
has_remount_ro = ((before == '\0' || before == ',') && (after == '\0' || after == ','));
}
if (strstr(opts, "errors=panic") || !has_remount_ro)
strcat(opts, ",errors=continue");
} else if (strcmp(fs, "xfs") == 0) {
strcat(opts, ",nouuid");
} else if (strncmp(fs, "gfs2", 4) == 0 && (strstr(opts, "errors=panic") || strstr(opts, "debug"))) {
strcat(opts, ",errors=withdraw");
}
res = mount(source, target, fs, flags, opts);
if (res == -1) {
err = errno;
goto error_clear_loop;
}
res = open(target, O_RDONLY | O_DIRECTORY);
if (res == -1) {
err = errno;
goto error_clear_loop;
}
if (change_dir) {
res = chdir(target);
if (res == -1) {
err = errno;
}
}
error_clear_loop:
if (need_loop_device)
reset_loop_device(loopname);
errno = err;
return res;
}
uint64_t r[2] = {0xffffffffffffffff, 0xffffffffffffffff};
int main(void)
{
syscall(__NR_mmap, /*addr=*/0x1ffffffff000ul, /*len=*/0x1000, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200000000000ul, /*len=*/0x1000000, /*prot=PROT_WRITE|PROT_READ|PROT_EXEC*/7ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200001000000ul, /*len=*/0x1000, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
const char* reason;
(void)reason;
intptr_t res = 0;
if (write(1, "executing program\n", sizeof("executing program\n") - 1)) {}
// syz_mount_image$btrfs arguments: [
// fs: ptr[in, buffer] {
// buffer: {62 74 72 66 73 00} (length 0x6)
// }
// dir: ptr[in, buffer] {
// buffer: {2e 2f 66 69 6c 65 30 00} (length 0x8)
// }
// flags: mount_flags = 0x0 (8 bytes)
// opts: ptr[in, fs_options[btrfs_options]] {
// fs_options[btrfs_options] {
// elems: array[fs_opt_elem[btrfs_options]] {
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// noacl: buffer: {6e 6f 61 63 6c} (length 0x5)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// subvolid: fs_opt["subvolid", fmt[hex, int64]] {
// name: buffer: {73 75 62 76 6f 6c 69 64} (length 0x8)
// eq: const = 0x3d (1 bytes)
// val: int64 = 0x0 (18 bytes)
// }
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// ssd_spread: buffer: {73 73 64 5f 73 70 72 65 61 64} (length 0xa)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// space_cache_v2: buffer: {73 70 61 63 65 5f 63 61 63 68 65 3d 76 32} (length 0xe)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// discard: buffer: {64 69 73 63 61 72 64} (length 0x7)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// enospc_debug: buffer: {65 6e 6f 73 70 63 5f 64 65 62 75 67} (length 0xc)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// space_cache_v2: buffer: {73 70 61 63 65 5f 63 61 63 68 65 3d 76 32} (length 0xe)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// noflushoncommit: buffer: {6e 6f 66 6c 75 73 68 6f 6e 63 6f 6d 6d 69 74} (length 0xf)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// ssd_spread: buffer: {73 73 64 5f 73 70 72 65 61 64} (length 0xa)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// datasum: buffer: {64 61 74 61 73 75 6d} (length 0x7)
// }
// comma: const = 0x2c (1 bytes)
// }
// }
// common: array[fs_opt_elem[fs_options_common]] {
// }
// null: const = 0x0 (1 bytes)
// }
// }
// chdir: int8 = 0x1 (1 bytes)
// size: len = 0x55a8 (8 bytes)
// img: ptr[in, buffer] {
// buffer: (compressed buffer with length 0x55a8)
// }
// ]
// returns fd_dir
memcpy((void*)0x2000000055c0, "btrfs\000", 6);
memcpy((void*)0x200000005600, "./file0\000", 8);
memcpy((void*)0x2000000013c0, "noacl", 5);
*(uint8_t*)0x2000000013c5 = 0x2c;
memcpy((void*)0x2000000013c6, "subvolid", 8);
*(uint8_t*)0x2000000013ce = 0x3d;
sprintf((char*)0x2000000013cf, "0x%016llx", (long long)0);
*(uint8_t*)0x2000000013e1 = 0x2c;
memcpy((void*)0x2000000013e2, "ssd_spread", 10);
*(uint8_t*)0x2000000013ec = 0x2c;
memcpy((void*)0x2000000013ed, "space_cache=v2", 14);
*(uint8_t*)0x2000000013fb = 0x2c;
memcpy((void*)0x2000000013fc, "discard", 7);
*(uint8_t*)0x200000001403 = 0x2c;
memcpy((void*)0x200000001404, "enospc_debug", 12);
*(uint8_t*)0x200000001410 = 0x2c;
memcpy((void*)0x200000001411, "space_cache=v2", 14);
*(uint8_t*)0x20000000141f = 0x2c;
memcpy((void*)0x200000001420, "noflushoncommit", 15);
*(uint8_t*)0x20000000142f = 0x2c;
memcpy((void*)0x200000001430, "ssd_spread", 10);
*(uint8_t*)0x20000000143a = 0x2c;
memcpy((void*)0x20000000143b, "datasum", 7);
*(uint8_t*)0x200000001442 = 0x2c;
*(uint8_t*)0x200000001443 = 0;
memcpy((void*)0x200000005680, "... [truncated large byte array] ...", 21928);
syz_mount_image(/*fs=*/0x2000000055c0, /*dir=*/0x200000005600, /*flags=*/0, /*opts=*/0x2000000013c0, /*chdir=*/1, /*size=*/0x55a8, /*img=*/0x200000005680);
// open arguments: [
// file: ptr[in, buffer] {
// buffer: {2e 2f 66 69 6c 65 30 00} (length 0x8)
// }
// flags: open_flags = 0x0 (8 bytes)
// mode: open_mode = 0x0 (8 bytes)
// ]
// returns fd
memcpy((void*)0x200000000080, "./file0\000", 8);
res = syscall(__NR_open, /*file=*/0x200000000080ul, /*flags=*/0ul, /*mode=*/0ul);
if (res != -1)
r[0] = res;
// ioctl$BTRFS_IOC_BALANCE_V2 arguments: [
// fd: fd (resource)
// cmd: const = 0xc4009420 (4 bytes)
// arg: ptr[inout, btrfs_ioctl_balance_args] {
// btrfs_ioctl_balance_args {
// flags: btrfs_ioctl_balance_args_flags = 0xa (8 bytes)
// state: btrfs_ioctl_balance_args_states = 0x0 (8 bytes)
// data: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// struct: btrfs_balance_args_u_s1 {
// usage_min: int32 = 0x0 (4 bytes)
// usage_max: int32 = 0x0 (4 bytes)
// }
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// struct: btrfs_balance_args_u_s1 {
// usage_min: int32 = 0x0 (4 bytes)
// usage_max: int32 = 0x0 (4 bytes)
// }
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// meta: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// sys: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// stat: btrfs_balance_progress {
// expected: int64 = 0x0 (8 bytes)
// considered: int64 = 0x0 (8 bytes)
// completed: int64 = 0x0 (8 bytes)
// }
// unused: buffer: {00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00} (length 0x240)
// }
// }
// ]
*(uint64_t*)0x200000001200 = 0xa;
*(uint64_t*)0x200000001208 = 0;
*(uint64_t*)0x200000001210 = 0;
*(uint32_t*)0x200000001218 = 0;
*(uint32_t*)0x20000000121c = 0;
*(uint64_t*)0x200000001220 = 0;
*(uint64_t*)0x200000001228 = 0;
*(uint64_t*)0x200000001230 = 0;
*(uint64_t*)0x200000001238 = 0;
*(uint64_t*)0x200000001240 = 0;
*(uint64_t*)0x200000001248 = 0;
*(uint64_t*)0x200000001250 = 0;
*(uint32_t*)0x200000001258 = 0;
*(uint32_t*)0x20000000125c = 0;
*(uint32_t*)0x200000001260 = 0;
*(uint32_t*)0x200000001264 = 0;
*(uint64_t*)0x200000001268 = 0;
*(uint64_t*)0x200000001270 = 0;
*(uint64_t*)0x200000001278 = 0;
*(uint64_t*)0x200000001280 = 0;
*(uint64_t*)0x200000001288 = 0;
*(uint64_t*)0x200000001290 = 0;
*(uint64_t*)0x200000001298 = 0;
*(uint64_t*)0x2000000012a0 = 0;
*(uint64_t*)0x2000000012a8 = 0;
*(uint64_t*)0x2000000012b0 = 0;
*(uint64_t*)0x2000000012b8 = 0;
*(uint64_t*)0x2000000012c0 = 0;
*(uint64_t*)0x2000000012c8 = 0;
*(uint64_t*)0x2000000012d0 = 0;
*(uint64_t*)0x2000000012d8 = 0;
*(uint64_t*)0x2000000012e0 = 0;
*(uint32_t*)0x2000000012e8 = 0;
*(uint32_t*)0x2000000012ec = 0;
*(uint64_t*)0x2000000012f0 = 0;
*(uint64_t*)0x2000000012f8 = 0;
*(uint64_t*)0x200000001300 = 0;
*(uint64_t*)0x200000001308 = 0;
*(uint64_t*)0x200000001310 = 0;
*(uint64_t*)0x200000001318 = 0;
*(uint64_t*)0x200000001320 = 0;
*(uint64_t*)0x200000001328 = 0;
*(uint64_t*)0x200000001330 = 0;
*(uint64_t*)0x200000001338 = 0;
*(uint64_t*)0x200000001340 = 0;
*(uint64_t*)0x200000001348 = 0;
*(uint64_t*)0x200000001350 = 0;
*(uint64_t*)0x200000001358 = 0;
*(uint64_t*)0x200000001360 = 0;
*(uint64_t*)0x200000001368 = 0;
*(uint32_t*)0x200000001370 = 0;
*(uint32_t*)0x200000001374 = 0;
*(uint64_t*)0x200000001378 = 0;
*(uint64_t*)0x200000001380 = 0;
*(uint64_t*)0x200000001388 = 0;
*(uint64_t*)0x200000001390 = 0;
*(uint64_t*)0x200000001398 = 0;
*(uint64_t*)0x2000000013a0 = 0;
*(uint64_t*)0x2000000013a8 = 0;
*(uint64_t*)0x2000000013b0 = 0;
*(uint64_t*)0x2000000013b8 = 0;
memset((void*)0x2000000013c0, 0, 576);
syscall(__NR_ioctl, /*fd=*/r[0], /*cmd=*/0xc4009420, /*arg=*/0x200000001200ul);
// open arguments: [
// file: ptr[in, buffer] {
// buffer: {2e 00} (length 0x2)
// }
// flags: open_flags = 0x0 (8 bytes)
// mode: open_mode = 0x0 (8 bytes)
// ]
// returns fd
memcpy((void*)0x200000000000, ".\000", 2);
res = syscall(__NR_open, /*file=*/0x200000000000ul, /*flags=*/0ul, /*mode=*/0ul);
if (res != -1)
r[1] = res;
// ioctl$EXT4_IOC_GROUP_ADD arguments: [
// fd: fd (resource)
// cmd: const = 0xc0185879 (4 bytes)
// arg: ptr[in, ext4_new_group_input] {
// ext4_new_group_input {
// group: int32 = 0x0 (4 bytes)
// pad = 0x0 (4 bytes)
// block_bitmap: int64 = 0x100004000 (8 bytes)
// inode_bitmap: int64 = 0x0 (8 bytes)
// inode_table: int64 = 0x0 (8 bytes)
// blocks_count: int32 = 0x0 (4 bytes)
// reserved_blocks: int16 = 0x0 (2 bytes)
// unused: const = 0x0 (2 bytes)
// }
// }
// ]
*(uint32_t*)0x200000000380 = 0;
*(uint64_t*)0x200000000388 = 0x100004000;
*(uint64_t*)0x200000000390 = 0;
*(uint64_t*)0x200000000398 = 0;
*(uint32_t*)0x2000000003a0 = 0;
*(uint16_t*)0x2000000003a4 = 0;
*(uint16_t*)0x2000000003a6 = 0;
syscall(__NR_ioctl, /*fd=*/r[1], /*cmd=*/0xc0185879, /*arg=*/0x200000000380ul);
return 0;
}
]
|
| 3/1 |
2026/08/08 03:06 |
action |
kernel-checkouter |
0m
Results: map[KernelSrc:/app/workdir/cache/src/fda57abbaa89268f1be2087d652398a1a507dfa0]
|
| 4/1 |
2026/08/08 03:06 |
action |
kernel-builder |
13m
Results: map[KernelObj:/app/workdir/cache/build/83ca9bf7d6b3e7673ebd2d376b4f723958eb1859]
|
| 5/1 |
2026/08/08 03:20 |
action |
crash-reproducer |
1m
Results: map[OtherCrashReports:<nil> ReproducedBugTitle:WARNING in btrfs_remove_chunk ReproducedCrashReport:BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
ReproducedFaultInjection:]
|
| 6/1 |
2026/08/08 03:21 |
action |
codesearch-prepare |
8m
Results: map[Index:codesearch-index]
|
| 7/1 |
2026/08/08 03:30 |
agent |
debugger |
33m
Model:
Instruction:
You are an experienced Linux kernel developer tasked with debugging a kernel crash root cause.
You need to provide a detailed explanation of the root cause for another developer to be
able to write a fix for the bug based on your explanation. Include all relevant details
into the response: function/struct/field/etc names, code snippets, line numbers,
macro/enum values, etc.
Don't make assumptions about the kernel source code (it may be different from what you assume it is).
Extensively use the provided code access tools (codesearch-*, git-*, grepper, etc)
to examine the actual source code, and confirm any assumptions.
Prefer calling several tools at the same time to save round-trips.
Prompt:
The crash is:
BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
The following C code is a draft of the vulnerable syscall sequence. Keep in mind that
it may lack the precise threading, sandboxing, and some arguments of a working reproducer:
// autogenerated by syzkaller (https://github.com/google/syzkaller)
#define _GNU_SOURCE
#include <endian.h>
#include <errno.h>
#include <fcntl.h>
#include <setjmp.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <linux/loop.h>
#ifndef __NR_memfd_create
#define __NR_memfd_create 319
#endif
static unsigned long long procid;
//% This code is derived from puff.{c,h}, found in the zlib development. The
//% original files come with the following copyright notice:
//% Copyright (C) 2002-2013 Mark Adler, all rights reserved
//% version 2.3, 21 Jan 2013
//% This software is provided 'as-is', without any express or implied
//% warranty. In no event will the author be held liable for any damages
//% arising from the use of this software.
//% Permission is granted to anyone to use this software for any purpose,
//% including commercial applications, and to alter it and redistribute it
//% freely, subject to the following restrictions:
//% 1. The origin of this software must not be misrepresented; you must not
//% claim that you wrote the original software. If you use this software
//% in a product, an acknowledgment in the product documentation would be
//% appreciated but is not required.
//% 2. Altered source versions must be plainly marked as such, and must not be
//% misrepresented as being the original software.
//% 3. This notice may not be removed or altered from any source distribution.
//% Mark Adler madler@alumni.caltech.edu
//% BEGIN CODE DERIVED FROM puff.{c,h}
#define MAXBITS 15
#define MAXLCODES 286
#define MAXDCODES 30
#define MAXCODES (MAXLCODES + MAXDCODES)
#define FIXLCODES 288
struct puff_state {
unsigned char* out;
unsigned long outlen;
unsigned long outcnt;
const unsigned char* in;
unsigned long inlen;
unsigned long incnt;
int bitbuf;
int bitcnt;
jmp_buf env;
};
static int puff_bits(struct puff_state* s, int need)
{
long val = s->bitbuf;
while (s->bitcnt < need) {
if (s->incnt == s->inlen)
longjmp(s->env, 1);
val |= (long)(s->in[s->incnt++]) << s->bitcnt;
s->bitcnt += 8;
}
s->bitbuf = (int)(val >> need);
s->bitcnt -= need;
return (int)(val & ((1L << need) - 1));
}
static int puff_stored(struct puff_state* s)
{
s->bitbuf = 0;
s->bitcnt = 0;
if (s->incnt + 4 > s->inlen)
return 2;
unsigned len = s->in[s->incnt++];
len |= s->in[s->incnt++] << 8;
if (s->in[s->incnt++] != (~len & 0xff) ||
s->in[s->incnt++] != ((~len >> 8) & 0xff))
return -2;
if (s->incnt + len > s->inlen)
return 2;
if (s->outcnt + len > s->outlen)
return 1;
for (; len--; s->outcnt++, s->incnt++) {
if (s->in[s->incnt])
s->out[s->outcnt] = s->in[s->incnt];
}
return 0;
}
struct puff_huffman {
short* count;
short* symbol;
};
static int puff_decode(struct puff_state* s, const struct puff_huffman* h)
{
int first = 0;
int index = 0;
int bitbuf = s->bitbuf;
int left = s->bitcnt;
int code = first = index = 0;
int len = 1;
short* next = h->count + 1;
while (1) {
while (left--) {
code |= bitbuf & 1;
bitbuf >>= 1;
int count = *next++;
if (code - count < first) {
s->bitbuf = bitbuf;
s->bitcnt = (s->bitcnt - len) & 7;
return h->symbol[index + (code - first)];
}
index += count;
first += count;
first <<= 1;
code <<= 1;
len++;
}
left = (MAXBITS + 1) - len;
if (left == 0)
break;
if (s->incnt == s->inlen)
longjmp(s->env, 1);
bitbuf = s->in[s->incnt++];
if (left > 8)
left = 8;
}
return -10;
}
static int puff_construct(struct puff_huffman* h, const short* length, int n)
{
int len;
for (len = 0; len <= MAXBITS; len++)
h->count[len] = 0;
int symbol;
for (symbol = 0; symbol < n; symbol++)
(h->count[length[symbol]])++;
if (h->count[0] == n)
return 0;
int left = 1;
for (len = 1; len <= MAXBITS; len++) {
left <<= 1;
left -= h->count[len];
if (left < 0)
return left;
}
short offs[MAXBITS + 1];
offs[1] = 0;
for (len = 1; len < MAXBITS; len++)
offs[len + 1] = offs[len] + h->count[len];
for (symbol = 0; symbol < n; symbol++)
if (length[symbol] != 0)
h->symbol[offs[length[symbol]]++] = symbol;
return left;
}
static int puff_codes(struct puff_state* s,
const struct puff_huffman* lencode,
const struct puff_huffman* distcode)
{
static const short lens[29] = {
3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31,
35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258};
static const short lext[29] = {
0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2,
3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0};
static const short dists[30] = {
1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193,
257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145,
8193, 12289, 16385, 24577};
static const short dext[30] = {
0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6,
7, 7, 8, 8, 9, 9, 10, 10, 11, 11,
12, 12, 13, 13};
int symbol;
do {
symbol = puff_decode(s, lencode);
if (symbol < 0)
return symbol;
if (symbol < 256) {
if (s->outcnt == s->outlen)
return 1;
if (symbol)
s->out[s->outcnt] = symbol;
s->outcnt++;
} else if (symbol > 256) {
symbol -= 257;
if (symbol >= 29)
return -10;
int len = lens[symbol] + puff_bits(s, lext[symbol]);
symbol = puff_decode(s, distcode);
if (symbol < 0)
return symbol;
unsigned dist = dists[symbol] + puff_bits(s, dext[symbol]);
if (dist > s->outcnt)
return -11;
if (s->outcnt + len > s->outlen)
return 1;
while (len--) {
if (dist <= s->outcnt && s->out[s->outcnt - dist])
s->out[s->outcnt] = s->out[s->outcnt - dist];
s->outcnt++;
}
}
} while (symbol != 256);
return 0;
}
static int puff_fixed(struct puff_state* s)
{
static int virgin = 1;
static short lencnt[MAXBITS + 1], lensym[FIXLCODES];
static short distcnt[MAXBITS + 1], distsym[MAXDCODES];
static struct puff_huffman lencode, distcode;
if (virgin) {
lencode.count = lencnt;
lencode.symbol = lensym;
distcode.count = distcnt;
distcode.symbol = distsym;
short lengths[FIXLCODES];
int symbol;
for (symbol = 0; symbol < 144; symbol++)
lengths[symbol] = 8;
for (; symbol < 256; symbol++)
lengths[symbol] = 9;
for (; symbol < 280; symbol++)
lengths[symbol] = 7;
for (; symbol < FIXLCODES; symbol++)
lengths[symbol] = 8;
puff_construct(&lencode, lengths, FIXLCODES);
for (symbol = 0; symbol < MAXDCODES; symbol++)
lengths[symbol] = 5;
puff_construct(&distcode, lengths, MAXDCODES);
virgin = 0;
}
return puff_codes(s, &lencode, &distcode);
}
static int puff_dynamic(struct puff_state* s)
{
static const short order[19] =
{16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15};
int nlen = puff_bits(s, 5) + 257;
int ndist = puff_bits(s, 5) + 1;
int ncode = puff_bits(s, 4) + 4;
if (nlen > MAXLCODES || ndist > MAXDCODES)
return -3;
short lengths[MAXCODES];
int index;
for (index = 0; index < ncode; index++)
lengths[order[index]] = puff_bits(s, 3);
for (; index < 19; index++)
lengths[order[index]] = 0;
short lencnt[MAXBITS + 1], lensym[MAXLCODES];
struct puff_huffman lencode = {lencnt, lensym};
int err = puff_construct(&lencode, lengths, 19);
if (err != 0)
return -4;
index = 0;
while (index < nlen + ndist) {
int symbol;
int len;
symbol = puff_decode(s, &lencode);
if (symbol < 0)
return symbol;
if (symbol < 16)
lengths[index++] = symbol;
else {
len = 0;
if (symbol == 16) {
if (index == 0)
return -5;
len = lengths[index - 1];
symbol = 3 + puff_bits(s, 2);
} else if (symbol == 17)
symbol = 3 + puff_bits(s, 3);
else
symbol = 11 + puff_bits(s, 7);
if (index + symbol > nlen + ndist)
return -6;
while (symbol--)
lengths[index++] = len;
}
}
if (lengths[256] == 0)
return -9;
err = puff_construct(&lencode, lengths, nlen);
if (err && (err < 0 || nlen != lencode.count[0] + lencode.count[1]))
return -7;
short distcnt[MAXBITS + 1], distsym[MAXDCODES];
struct puff_huffman distcode = {distcnt, distsym};
err = puff_construct(&distcode, lengths + nlen, ndist);
if (err && (err < 0 || ndist != distcode.count[0] + distcode.count[1]))
return -8;
return puff_codes(s, &lencode, &distcode);
}
static int puff(
unsigned char* dest,
unsigned long* destlen,
const unsigned char* source,
unsigned long sourcelen)
{
struct puff_state s = {
.out = dest,
.outlen = *destlen,
.outcnt = 0,
.in = source,
.inlen = sourcelen,
.incnt = 0,
.bitbuf = 0,
.bitcnt = 0,
};
int err;
if (setjmp(s.env) != 0)
err = 2;
else {
int last;
do {
last = puff_bits(&s, 1);
int type = puff_bits(&s, 2);
err = type == 0 ? puff_stored(&s) : (type == 1 ? puff_fixed(&s) : (type == 2 ? puff_dynamic(&s) : -1));
if (err != 0)
break;
} while (!last);
}
*destlen = s.outcnt;
return err;
}
//% END CODE DERIVED FROM puff.{c,h}
#define ZLIB_HEADER_WIDTH 2
static int puff_zlib_to_file(const unsigned char* source, unsigned long sourcelen, int dest_fd)
{
if (sourcelen < ZLIB_HEADER_WIDTH)
return 0;
source += ZLIB_HEADER_WIDTH;
sourcelen -= ZLIB_HEADER_WIDTH;
const unsigned long max_destlen = 132 << 20;
void* ret = mmap(0, max_destlen, PROT_WRITE | PROT_READ, MAP_PRIVATE | MAP_ANON, -1, 0);
if (ret == MAP_FAILED)
return -1;
unsigned char* dest = (unsigned char*)ret;
unsigned long destlen = max_destlen;
int err = puff(dest, &destlen, source, sourcelen);
if (err) {
munmap(dest, max_destlen);
errno = -err;
return -1;
}
if (write(dest_fd, dest, destlen) != (ssize_t)destlen) {
munmap(dest, max_destlen);
return -1;
}
return munmap(dest, max_destlen);
}
static int setup_loop_device(unsigned char* data, unsigned long size, const char* loopname, int* loopfd_p)
{
int err = 0, loopfd = -1;
int memfd = syscall(__NR_memfd_create, "syzkaller", 0);
if (memfd == -1) {
err = errno;
goto error;
}
if (puff_zlib_to_file(data, size, memfd)) {
err = errno;
goto error_close_memfd;
}
loopfd = open(loopname, O_RDWR);
if (loopfd == -1) {
err = errno;
goto error_close_memfd;
}
if (ioctl(loopfd, LOOP_SET_FD, memfd)) {
if (errno != EBUSY) {
err = errno;
goto error_close_loop;
}
ioctl(loopfd, LOOP_CLR_FD, 0);
usleep(1000);
if (ioctl(loopfd, LOOP_SET_FD, memfd)) {
err = errno;
goto error_close_loop;
}
}
close(memfd);
*loopfd_p = loopfd;
return 0;
error_close_loop:
close(loopfd);
error_close_memfd:
close(memfd);
error:
errno = err;
return -1;
}
static void reset_loop_device(const char* loopname)
{
int loopfd = open(loopname, O_RDWR);
if (loopfd == -1) {
return;
}
if (ioctl(loopfd, LOOP_CLR_FD, 0)) {
}
close(loopfd);
}
static long syz_mount_image(
volatile long fsarg,
volatile long dir,
volatile long flags,
volatile long optsarg,
volatile long change_dir,
volatile unsigned long size,
volatile long image)
{
unsigned char* data = (unsigned char*)image;
int res = -1, err = 0, need_loop_device = !!size;
char* mount_opts = (char*)optsarg;
char* target = (char*)dir;
char* fs = (char*)fsarg;
char* source = NULL;
char loopname[64];
if (need_loop_device) {
int loopfd;
memset(loopname, 0, sizeof(loopname));
snprintf(loopname, sizeof(loopname), "/dev/loop%llu", procid);
if (setup_loop_device(data, size, loopname, &loopfd) == -1)
return -1;
close(loopfd);
source = loopname;
}
mkdir(target, 0777);
char opts[256];
memset(opts, 0, sizeof(opts));
if (strlen(mount_opts) > (sizeof(opts) - 32)) {
}
strncpy(opts, mount_opts, sizeof(opts) - 32);
if (strcmp(fs, "iso9660") == 0) {
flags |= MS_RDONLY;
} else if (strncmp(fs, "ext", 3) == 0) {
bool has_remount_ro = false;
char* remount_ro_start = strstr(opts, "errors=remount-ro");
if (remount_ro_start != NULL) {
char after = *(remount_ro_start + strlen("errors=remount-ro"));
char before = remount_ro_start == opts ? '\0' : *(remount_ro_start - 1);
has_remount_ro = ((before == '\0' || before == ',') && (after == '\0' || after == ','));
}
if (strstr(opts, "errors=panic") || !has_remount_ro)
strcat(opts, ",errors=continue");
} else if (strcmp(fs, "xfs") == 0) {
strcat(opts, ",nouuid");
} else if (strncmp(fs, "gfs2", 4) == 0 && (strstr(opts, "errors=panic") || strstr(opts, "debug"))) {
strcat(opts, ",errors=withdraw");
}
res = mount(source, target, fs, flags, opts);
if (res == -1) {
err = errno;
goto error_clear_loop;
}
res = open(target, O_RDONLY | O_DIRECTORY);
if (res == -1) {
err = errno;
goto error_clear_loop;
}
if (change_dir) {
res = chdir(target);
if (res == -1) {
err = errno;
}
}
error_clear_loop:
if (need_loop_device)
reset_loop_device(loopname);
errno = err;
return res;
}
uint64_t r[2] = {0xffffffffffffffff, 0xffffffffffffffff};
int main(void)
{
syscall(__NR_mmap, /*addr=*/0x1ffffffff000ul, /*len=*/0x1000, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200000000000ul, /*len=*/0x1000000, /*prot=PROT_WRITE|PROT_READ|PROT_EXEC*/7ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
syscall(__NR_mmap, /*addr=*/0x200001000000ul, /*len=*/0x1000, /*prot=*/0ul, /*flags=MAP_FIXED|MAP_ANONYMOUS|MAP_PRIVATE*/0x32ul, /*fd=*/(intptr_t)-1, /*offset=*/0ul);
const char* reason;
(void)reason;
intptr_t res = 0;
if (write(1, "executing program\n", sizeof("executing program\n") - 1)) {}
// syz_mount_image$btrfs arguments: [
// fs: ptr[in, buffer] {
// buffer: {62 74 72 66 73 00} (length 0x6)
// }
// dir: ptr[in, buffer] {
// buffer: {2e 2f 66 69 6c 65 30 00} (length 0x8)
// }
// flags: mount_flags = 0x0 (8 bytes)
// opts: ptr[in, fs_options[btrfs_options]] {
// fs_options[btrfs_options] {
// elems: array[fs_opt_elem[btrfs_options]] {
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// noacl: buffer: {6e 6f 61 63 6c} (length 0x5)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// subvolid: fs_opt["subvolid", fmt[hex, int64]] {
// name: buffer: {73 75 62 76 6f 6c 69 64} (length 0x8)
// eq: const = 0x3d (1 bytes)
// val: int64 = 0x0 (18 bytes)
// }
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// ssd_spread: buffer: {73 73 64 5f 73 70 72 65 61 64} (length 0xa)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// space_cache_v2: buffer: {73 70 61 63 65 5f 63 61 63 68 65 3d 76 32} (length 0xe)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// discard: buffer: {64 69 73 63 61 72 64} (length 0x7)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// enospc_debug: buffer: {65 6e 6f 73 70 63 5f 64 65 62 75 67} (length 0xc)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// space_cache_v2: buffer: {73 70 61 63 65 5f 63 61 63 68 65 3d 76 32} (length 0xe)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// noflushoncommit: buffer: {6e 6f 66 6c 75 73 68 6f 6e 63 6f 6d 6d 69 74} (length 0xf)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// ssd_spread: buffer: {73 73 64 5f 73 70 72 65 61 64} (length 0xa)
// }
// comma: const = 0x2c (1 bytes)
// }
// fs_opt_elem[btrfs_options] {
// elem: union btrfs_options {
// datasum: buffer: {64 61 74 61 73 75 6d} (length 0x7)
// }
// comma: const = 0x2c (1 bytes)
// }
// }
// common: array[fs_opt_elem[fs_options_common]] {
// }
// null: const = 0x0 (1 bytes)
// }
// }
// chdir: int8 = 0x1 (1 bytes)
// size: len = 0x55a8 (8 bytes)
// img: ptr[in, buffer] {
// buffer: (compressed buffer with length 0x55a8)
// }
// ]
// returns fd_dir
memcpy((void*)0x2000000055c0, "btrfs\000", 6);
memcpy((void*)0x200000005600, "./file0\000", 8);
memcpy((void*)0x2000000013c0, "noacl", 5);
*(uint8_t*)0x2000000013c5 = 0x2c;
memcpy((void*)0x2000000013c6, "subvolid", 8);
*(uint8_t*)0x2000000013ce = 0x3d;
sprintf((char*)0x2000000013cf, "0x%016llx", (long long)0);
*(uint8_t*)0x2000000013e1 = 0x2c;
memcpy((void*)0x2000000013e2, "ssd_spread", 10);
*(uint8_t*)0x2000000013ec = 0x2c;
memcpy((void*)0x2000000013ed, "space_cache=v2", 14);
*(uint8_t*)0x2000000013fb = 0x2c;
memcpy((void*)0x2000000013fc, "discard", 7);
*(uint8_t*)0x200000001403 = 0x2c;
memcpy((void*)0x200000001404, "enospc_debug", 12);
*(uint8_t*)0x200000001410 = 0x2c;
memcpy((void*)0x200000001411, "space_cache=v2", 14);
*(uint8_t*)0x20000000141f = 0x2c;
memcpy((void*)0x200000001420, "noflushoncommit", 15);
*(uint8_t*)0x20000000142f = 0x2c;
memcpy((void*)0x200000001430, "ssd_spread", 10);
*(uint8_t*)0x20000000143a = 0x2c;
memcpy((void*)0x20000000143b, "datasum", 7);
*(uint8_t*)0x200000001442 = 0x2c;
*(uint8_t*)0x200000001443 = 0;
memcpy((void*)0x200000005680, "... [truncated large byte array] ...", 21928);
syz_mount_image(/*fs=*/0x2000000055c0, /*dir=*/0x200000005600, /*flags=*/0, /*opts=*/0x2000000013c0, /*chdir=*/1, /*size=*/0x55a8, /*img=*/0x200000005680);
// open arguments: [
// file: ptr[in, buffer] {
// buffer: {2e 2f 66 69 6c 65 30 00} (length 0x8)
// }
// flags: open_flags = 0x0 (8 bytes)
// mode: open_mode = 0x0 (8 bytes)
// ]
// returns fd
memcpy((void*)0x200000000080, "./file0\000", 8);
res = syscall(__NR_open, /*file=*/0x200000000080ul, /*flags=*/0ul, /*mode=*/0ul);
if (res != -1)
r[0] = res;
// ioctl$BTRFS_IOC_BALANCE_V2 arguments: [
// fd: fd (resource)
// cmd: const = 0xc4009420 (4 bytes)
// arg: ptr[inout, btrfs_ioctl_balance_args] {
// btrfs_ioctl_balance_args {
// flags: btrfs_ioctl_balance_args_flags = 0xa (8 bytes)
// state: btrfs_ioctl_balance_args_states = 0x0 (8 bytes)
// data: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// struct: btrfs_balance_args_u_s1 {
// usage_min: int32 = 0x0 (4 bytes)
// usage_max: int32 = 0x0 (4 bytes)
// }
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// struct: btrfs_balance_args_u_s1 {
// usage_min: int32 = 0x0 (4 bytes)
// usage_max: int32 = 0x0 (4 bytes)
// }
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// meta: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// sys: btrfs_balance_args {
// profiles: int64 = 0x0 (8 bytes)
// union1: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// devid: devid (resource)
// pstart: int64 = 0x0 (8 bytes)
// pend: int64 = 0x0 (8 bytes)
// vstart: int64 = 0x0 (8 bytes)
// vend: int64 = 0x0 (8 bytes)
// target: int64 = 0x0 (8 bytes)
// flags: btrfs_balance_args_flags = 0x0 (8 bytes)
// union2: union btrfs_balance_args_u {
// usage: int64 = 0x0 (8 bytes)
// }
// stripes_min: int32 = 0x0 (4 bytes)
// stripes_max: int32 = 0x0 (4 bytes)
// unused: array[int64] {
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// int64 = 0x0 (8 bytes)
// }
// }
// stat: btrfs_balance_progress {
// expected: int64 = 0x0 (8 bytes)
// considered: int64 = 0x0 (8 bytes)
// completed: int64 = 0x0 (8 bytes)
// }
// unused: buffer: {00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00} (length 0x240)
// }
// }
// ]
*(uint64_t*)0x200000001200 = 0xa;
*(uint64_t*)0x200000001208 = 0;
*(uint64_t*)0x200000001210 = 0;
*(uint32_t*)0x200000001218 = 0;
*(uint32_t*)0x20000000121c = 0;
*(uint64_t*)0x200000001220 = 0;
*(uint64_t*)0x200000001228 = 0;
*(uint64_t*)0x200000001230 = 0;
*(uint64_t*)0x200000001238 = 0;
*(uint64_t*)0x200000001240 = 0;
*(uint64_t*)0x200000001248 = 0;
*(uint64_t*)0x200000001250 = 0;
*(uint32_t*)0x200000001258 = 0;
*(uint32_t*)0x20000000125c = 0;
*(uint32_t*)0x200000001260 = 0;
*(uint32_t*)0x200000001264 = 0;
*(uint64_t*)0x200000001268 = 0;
*(uint64_t*)0x200000001270 = 0;
*(uint64_t*)0x200000001278 = 0;
*(uint64_t*)0x200000001280 = 0;
*(uint64_t*)0x200000001288 = 0;
*(uint64_t*)0x200000001290 = 0;
*(uint64_t*)0x200000001298 = 0;
*(uint64_t*)0x2000000012a0 = 0;
*(uint64_t*)0x2000000012a8 = 0;
*(uint64_t*)0x2000000012b0 = 0;
*(uint64_t*)0x2000000012b8 = 0;
*(uint64_t*)0x2000000012c0 = 0;
*(uint64_t*)0x2000000012c8 = 0;
*(uint64_t*)0x2000000012d0 = 0;
*(uint64_t*)0x2000000012d8 = 0;
*(uint64_t*)0x2000000012e0 = 0;
*(uint32_t*)0x2000000012e8 = 0;
*(uint32_t*)0x2000000012ec = 0;
*(uint64_t*)0x2000000012f0 = 0;
*(uint64_t*)0x2000000012f8 = 0;
*(uint64_t*)0x200000001300 = 0;
*(uint64_t*)0x200000001308 = 0;
*(uint64_t*)0x200000001310 = 0;
*(uint64_t*)0x200000001318 = 0;
*(uint64_t*)0x200000001320 = 0;
*(uint64_t*)0x200000001328 = 0;
*(uint64_t*)0x200000001330 = 0;
*(uint64_t*)0x200000001338 = 0;
*(uint64_t*)0x200000001340 = 0;
*(uint64_t*)0x200000001348 = 0;
*(uint64_t*)0x200000001350 = 0;
*(uint64_t*)0x200000001358 = 0;
*(uint64_t*)0x200000001360 = 0;
*(uint64_t*)0x200000001368 = 0;
*(uint32_t*)0x200000001370 = 0;
*(uint32_t*)0x200000001374 = 0;
*(uint64_t*)0x200000001378 = 0;
*(uint64_t*)0x200000001380 = 0;
*(uint64_t*)0x200000001388 = 0;
*(uint64_t*)0x200000001390 = 0;
*(uint64_t*)0x200000001398 = 0;
*(uint64_t*)0x2000000013a0 = 0;
*(uint64_t*)0x2000000013a8 = 0;
*(uint64_t*)0x2000000013b0 = 0;
*(uint64_t*)0x2000000013b8 = 0;
memset((void*)0x2000000013c0, 0, 576);
syscall(__NR_ioctl, /*fd=*/r[0], /*cmd=*/0xc4009420, /*arg=*/0x200000001200ul);
// open arguments: [
// file: ptr[in, buffer] {
// buffer: {2e 00} (length 0x2)
// }
// flags: open_flags = 0x0 (8 bytes)
// mode: open_mode = 0x0 (8 bytes)
// ]
// returns fd
memcpy((void*)0x200000000000, ".\000", 2);
res = syscall(__NR_open, /*file=*/0x200000000000ul, /*flags=*/0ul, /*mode=*/0ul);
if (res != -1)
r[1] = res;
// ioctl$EXT4_IOC_GROUP_ADD arguments: [
// fd: fd (resource)
// cmd: const = 0xc0185879 (4 bytes)
// arg: ptr[in, ext4_new_group_input] {
// ext4_new_group_input {
// group: int32 = 0x0 (4 bytes)
// pad = 0x0 (4 bytes)
// block_bitmap: int64 = 0x100004000 (8 bytes)
// inode_bitmap: int64 = 0x0 (8 bytes)
// inode_table: int64 = 0x0 (8 bytes)
// blocks_count: int32 = 0x0 (4 bytes)
// reserved_blocks: int16 = 0x0 (2 bytes)
// unused: const = 0x0 (2 bytes)
// }
// }
// ]
*(uint32_t*)0x200000000380 = 0;
*(uint64_t*)0x200000000388 = 0x100004000;
*(uint64_t*)0x200000000390 = 0;
*(uint64_t*)0x200000000398 = 0;
*(uint32_t*)0x2000000003a0 = 0;
*(uint16_t*)0x2000000003a4 = 0;
*(uint16_t*)0x2000000003a6 = 0;
syscall(__NR_ioctl, /*fd=*/r[1], /*cmd=*/0xc0185879, /*arg=*/0x200000000380ul);
return 0;
}
Reply: An analysis of the crash reveals that it is caused by a transaction abort (`-ENOSPC`) in `btrfs_remove_chunk` during a chunk relocation/balance operation. The abort happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
### Root Cause
1. **Order of Operations**: During chunk removal, `btrfs_remove_chunk` first removes the device extents from the device tree by calling `btrfs_remove_dev_extents`.
2. **System Space Reservation**: It then calls `check_system_chunk` to reserve system space for removing the chunk item from the chunk tree. If the system space is full, `check_system_chunk` attempts to allocate a new system chunk.
3. **Commit Root Search**: To allocate a new chunk, `find_free_dev_extent` is called. However, `find_free_dev_extent` searches the *commit root* of the device tree to prevent reusing space freed in the current transaction (which could lead to corruption if the transaction aborts).
4. **Invisible Freed Space**: Because the device extents were just freed in the *current* transaction by `btrfs_remove_dev_extents`, `find_free_dev_extent` does not see this freed space. If the device is otherwise full, it returns `-ENOSPC`.
5. **Ignored Failure**: `check_system_chunk` ignores the failure to allocate a system chunk and does not reserve any space in `trans->chunk_bytes_reserved`.
6. **Transaction Abort**: `remove_chunk_item` is then called, which fails with `-ENOSPC` because it needs to COW a node in the chunk tree but no system space was reserved. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully (returning `-ENOSPC` would leave the filesystem in an inconsistent state with the chunk item present but device extents missing). Thus, it is forced to abort the transaction.
### The Fix
We must ensure that we have enough system space *before* we modify the device tree. However, we cannot simply move the `check_system_chunk` call before `btrfs_remove_dev_extents` and keep the reservation, because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations (`btrfs_trans_release_chunk_metadata`).
Instead, we can call `check_system_chunk` early to *ensure* the system `space_info` has enough free space (allocating a system chunk if necessary), and then immediately release the `chunk_block_rsv` reservation. If it fails to allocate a system chunk, we can fail gracefully by returning `-ENOSPC` before any trees are modified. Then, after `btrfs_remove_dev_extents`, we call `check_system_chunk` again to make the actual reservation for `remove_chunk_item`.
Here is the proposed fix for `fs/btrfs/volumes.c` in `btrfs_remove_chunk`:
```c
map = em->map_lookup;
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
/*
* Take the device list mutex to prevent races with the final phase of
* a device replace operation...
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
// ... existing btrfs_remove_dev_extents logic ...
}
mutex_unlock(&fs_devices->device_list_mutex);
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
### Why this works:
1. **Graceful Failure**: By checking `trans->chunk_bytes_reserved`, we can detect if `check_system_chunk` failed to allocate a system chunk. If it failed, we return `-ENOSPC` *before* any trees are modified, allowing the balance operation to fail cleanly instead of crashing the kernel.
2. **No Metadata Leaks**: We immediately release the early reservation using `btrfs_trans_release_chunk_metadata(trans)`.
3. **Safe from Nested Allocations**: Because we release the reservation, we don't care if `btrfs_remove_dev_extents` triggers a nested chunk allocation that clears `trans->chunk_bytes_reserved`.
4. **Guaranteed Space**: The early `check_system_chunk` ensures the system `space_info` has enough free space (by allocating a 32MB system chunk if needed). When we call `check_system_chunk` the second time, it will see the free space in the `space_info` and successfully reserve the few KB needed for `remove_chunk_item`.
|
| 8/2 |
2026/08/08 03:30 |
llm |
debugger |
0m
Model:
Tokens:
input: 19252
output: 38
thoughts: 83
Thoughts: Here's my analysis of the situation:
**Investigating a btrfs Chunk Removal Crash**
Okay, so I've got a crash reported within `btrfs_remove_chunk`. Specifically, the error is occurring at `fs/btrfs/volumes.c:3526`. The backtrace indicates the warning is originating from `btrfs_remove_chunk+0xc9b/0x1070`. My initial course of action is to dive right into that code. I need to examine `fs/btrfs/volumes.c`, focusing on the area around line 3526. Let's see what's happening there during this chunk removal process. I'm hoping I can glean enough context from the surrounding code to determine the root cause of the warning and, ultimately, the crash. I'll be looking for potential race conditions, null pointer dereferences, or any other logic flaws that could lead to this issue during the chunk removal.
|
| 9/2 |
2026/08/08 03:30 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3500 LineCount:50]
Results: map[Contents:3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
]
|
| 10/2 |
2026/08/08 03:30 |
llm |
debugger |
0m
Model:
Tokens:
input: 20412
output: 40
thoughts: 77
Thoughts: Here's my summary, as the expert:
**Investigating a Transaction Abort**
Okay, so I've got a transaction abort warning, and the kernel's pointing me directly to `btrfs_abort_transaction(trans, ret)` on line 3526. That's a good starting point, saves me some time. Now, the astute thing to do is *not* assume that's a simple function call. Let's remember the kernel's tricks! Right, it's probably a macro. Yep, confirms it, `btrfs_abort_transaction` expands to a warning and a call to `__btrfs_abort_transaction`. I'm going to take a look at the macro expansion next to be sure there's nothing unexpected. Then, I can start looking at where that specific call to `btrfs_abort_transaction` originates, and understand why the transaction is being aborted and what `ret` represents. Understanding the calling context is key here.
|
| 11/2 |
2026/08/08 03:30 |
tool |
codesearch-definition-source |
0m
Error:
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_abort_transaction]
Results: map[SourceCode: SourceFile:]
|
| 12/2 |
2026/08/08 03:30 |
llm |
debugger |
0m
Model:
Tokens:
input: 20765
output: 23
thoughts: 14
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience and written in the first person:
**Investigating a Btrfs Transaction Abort**
Okay, so I'm looking at something involving a `btrfs_abort_transaction`. That function name alone tells me a lot. A transaction abort implies something went wrong, and the filesystem has decided to roll back changes to maintain consistency. My immediate thought is, "What triggered this?" I need to understand the *why* behind the abort. I'm going to start by grepping the codebase for that function name. This is going to give me my initial context. I'm hoping to find the call sites, error codes being passed, and any related logging that might provide clues. I anticipate seeing various potential error paths, and I'll be looking for commonalities or patterns that might narrow down the possible root causes. This is going to be a deep dive.
|
| 13/2 |
2026/08/08 03:30 |
tool |
grepper |
0m
Args: map[Expression:#define btrfs_abort_transaction]
Results: map[Output:fs/btrfs/transaction.h=260=do { \
--
fs/btrfs/transaction.h-273- */
fs/btrfs/transaction.h:274:#define btrfs_abort_transaction(trans, error) \
fs/btrfs/transaction.h-275-do { \
]
|
| 14/2 |
2026/08/08 03:30 |
llm |
debugger |
0m
Model:
Tokens:
input: 21096
output: 37
thoughts: 22
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience:
**Diving into Transaction Handling**
Okay, let's zero in on `fs/btrfs/transaction.h`, specifically around line 274. I need to refresh my memory on how Btrfs handles transactions at a fundamental level. I'm guessing that line 274 is likely dealing with a key data structure or function related to commit groups or the transaction descriptor itself. Perhaps it's defining the flags or fields within the transaction struct that dictates rollback behaviour, commit order, or even the locking scheme employed. It could also involve definitions for tracking extent modifications within a transaction, or maybe even related to the metadata updates. I'll need to see if it's related to the commit cycle, and if there are any subtle hints about performance optimizations in that section that are worth investigating. Time to dig in and see what nuances the code reveals.
|
| 15/2 |
2026/08/08 03:31 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/transaction.h FirstLine:260 LineCount:30]
Results: map[Contents: 260: do { \
261: (void)sizeof(char[-!(__builtin_constant_p(error) ? (error) < 0 : 1)]); \
262: if (IS_ENABLED(CONFIG_BTRFS_DEBUG)) { \
263: if (sizeof(char[(error) < 0]) != 1) \
264: DEBUG_WARN("error >= 0 passed to btrfs_abort_transaction()"); \
265: } \
266: } while(0)
267:
268: /*
269: * Call btrfs_abort_transaction() as early as possible when an error condition
270: * is detected, that way the exact stack trace is reported for some errors.
271: *
272: * Error number must be negative as it encodes wheather it's the first abort.
273: */
274: #define btrfs_abort_transaction(trans, error) \
275: do { \
276: int __error = (error); \
277: \
278: VERIFY_NEGATIVE_ERROR(error); \
279: /* Report first abort since mount */ \
280: if (!test_and_set_bit(BTRFS_FS_STATE_TRANS_ABORTED, \
281: &((trans)->fs_info->fs_state))) { \
282: WARN_ON(btrfs_abort_should_print_stack(__error)); \
283: __error = -__error; \
284: } \
285: __btrfs_abort_transaction((trans), __func__, \
286: __LINE__, __error); \
287: } while (0)
288:
289: int btrfs_end_transaction(struct btrfs_trans_handle *trans);
]
|
| 16/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 21899
output: 40
thoughts: 90
Thoughts: Okay, let me break this down.
**Diving into the Btrfs Chunk Creation Failure**
Right, that `WARN_ON(btrfs_abort_should_print_stack(__error));` call… that's our canary. It's flagging the problem, specifically an aborted transaction, and it's happening during `btrfs_remove_chunk`. Interesting. This points to something going sideways earlier in the process.
The root cause appears to be the `btrfs_create_chunk` function, because that's what returned the error that triggered the abort. So, *that's* the critical piece I need to understand. Let's dig into what `btrfs_create_chunk` is actually doing and why it might be failing in this specific context. What could it be? What kind of errors are coming back? Time to dive into the code...
|
| 17/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_create_chunk]
Results: map[SourceCode:6121: struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,
6122: struct btrfs_space_info *space_info,
6123: u64 type)
6124: {
6125: struct btrfs_fs_info *info = trans->fs_info;
6126: struct btrfs_fs_devices *fs_devices = info->fs_devices;
6127: struct btrfs_device_info AUTO_KFREE(devices_info);
6128: struct alloc_chunk_ctl ctl;
6129: int ret;
6130:
6131: lockdep_assert_held(&info->chunk_mutex);
6132:
6133: if (unlikely(!alloc_profile_is_valid(type, 0))) {
6134: DEBUG_WARN("invalid alloc profile for type %llu", type);
6135: return ERR_PTR(-EINVAL);
6136: }
6137:
6138: if (list_empty(&fs_devices->alloc_list)) {
6139: if (btrfs_test_opt(info, ENOSPC_DEBUG))
6140: btrfs_debug(info, "%s: no writable device", __func__);
6141: return ERR_PTR(-ENOSPC);
6142: }
6143:
6144: if (unlikely(!(type & BTRFS_BLOCK_GROUP_TYPE_MASK))) {
6145: btrfs_err(info, "invalid chunk type 0x%llx requested", type);
6146: DEBUG_WARN();
6147: return ERR_PTR(-EINVAL);
6148: }
6149:
6150: ctl.start = find_next_chunk(info);
6151: ctl.type = type;
6152: ctl.space_info = space_info;
6153: init_alloc_chunk_ctl(fs_devices, &ctl);
6154:
6155: devices_info = kzalloc_objs(*devices_info, fs_devices->rw_devices, GFP_NOFS);
6156: if (!devices_info)
6157: return ERR_PTR(-ENOMEM);
6158:
6159: ret = gather_device_info(fs_devices, &ctl, devices_info);
6160: if (ret < 0)
6161: return ERR_PTR(ret);
6162:
6163: ret = decide_stripe_size(fs_devices, &ctl, devices_info);
6164: if (ret < 0)
6165: return ERR_PTR(ret);
6166:
6167: return create_chunk(trans, &ctl, devices_info);
6168: }
SourceFile:fs/btrfs/volumes.c]
|
| 18/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 23038
output: 38
thoughts: 66
Thoughts: Here's my summary, as though I'm thinking through this:
**Analyzing the Btrfs Chunk Removal Crash**
Okay, so the kernel panic gives us a pretty specific location, which is great. It's in the `btrfs_remove_chunk` function, and the stack trace points directly to line 3526 within `fs/btrfs/volumes.c`. Time to dive into the code! I need to understand what's happening *around* that line. Specifically, I need to see the function's context. I'll check what conditions could have led to this failure. Is it a null pointer dereference? An invalid index? A race condition? Let's trace the arguments passed to this function, too. Looking at the call stack, I also need to find out *what* function called `btrfs_remove_chunk` so I can understand the bigger picture of the crash. I'll grab the source code and start poring over it. It seems like the error may have something to do with removing a chunk, and it crashed there... I'll need to know *why* the code was attempting to remove a chunk in the first place, and what data structures it was operating on at the time of the crash. Time to get to work.
|
| 19/2 |
2026/08/08 03:31 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3510 LineCount:30]
Results: map[Contents:3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
]
|
| 20/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 23897
output: 38
thoughts: 49
Thoughts: Okay, here's my interpretation of those thoughts, framed as my own:
**Investigating a btrfs Chunk Creation Failure**
Alright, the `btrfs_create_chunk` function just puked out an error code. Chances are it's `-ENOSPC` (no space) or maybe `-EINVAL` (invalid argument), though could be something else of course. Before I start digging into the intricacies of why *that* particular function is failing, I want to refresh my memory on the interactions of removing chunks. I need to understand what `btrfs_remove_chunk` is doing at the code path before line 3511. It's highly probable the previous chunk is the problem. Maybe it is failing to remove some data. So I need to go back and examine the remove operations to find the actual root cause of this error.
|
| 21/2 |
2026/08/08 03:31 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3480 LineCount:30]
Results: map[Contents:3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
]
|
| 22/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 24740
output: 40
thoughts: 86
Thoughts: **Investigating Persistent -ENOSPC Errors in Btrfs Chunk Creation**
Okay, so `remove_chunk_item` just handed me back `-ENOSPC`. That's not ideal, and it's throwing a wrench into the whole process. Now, the code is trying to salvage things by calling `btrfs_create_chunk`, hoping to get a new chunk allocated. But... surprise, surprise, `btrfs_create_chunk` *also* returned an error, and I'm betting it's `-ENOSPC` yet again. This pattern is a little too repetitive for my liking.
The question now is, why is `btrfs_create_chunk` failing with `-ENOSPC`? This has to be addressed. I need to get a better understanding of what's going on. Let's dig back into the `btrfs_create_chunk` function and see what it's doing under the hood. There's probably something going on that's escaping me, or I'm missing some crucial context from the earlier failure. Time to trace the execution and see where the actual resource constraint is arising, or perhaps a resource conflict.
|
| 23/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_create_chunk]
Results: map[SourceCode:6121: struct btrfs_block_group *btrfs_create_chunk(struct btrfs_trans_handle *trans,
6122: struct btrfs_space_info *space_info,
6123: u64 type)
6124: {
6125: struct btrfs_fs_info *info = trans->fs_info;
6126: struct btrfs_fs_devices *fs_devices = info->fs_devices;
6127: struct btrfs_device_info AUTO_KFREE(devices_info);
6128: struct alloc_chunk_ctl ctl;
6129: int ret;
6130:
6131: lockdep_assert_held(&info->chunk_mutex);
6132:
6133: if (unlikely(!alloc_profile_is_valid(type, 0))) {
6134: DEBUG_WARN("invalid alloc profile for type %llu", type);
6135: return ERR_PTR(-EINVAL);
6136: }
6137:
6138: if (list_empty(&fs_devices->alloc_list)) {
6139: if (btrfs_test_opt(info, ENOSPC_DEBUG))
6140: btrfs_debug(info, "%s: no writable device", __func__);
6141: return ERR_PTR(-ENOSPC);
6142: }
6143:
6144: if (unlikely(!(type & BTRFS_BLOCK_GROUP_TYPE_MASK))) {
6145: btrfs_err(info, "invalid chunk type 0x%llx requested", type);
6146: DEBUG_WARN();
6147: return ERR_PTR(-EINVAL);
6148: }
6149:
6150: ctl.start = find_next_chunk(info);
6151: ctl.type = type;
6152: ctl.space_info = space_info;
6153: init_alloc_chunk_ctl(fs_devices, &ctl);
6154:
6155: devices_info = kzalloc_objs(*devices_info, fs_devices->rw_devices, GFP_NOFS);
6156: if (!devices_info)
6157: return ERR_PTR(-ENOMEM);
6158:
6159: ret = gather_device_info(fs_devices, &ctl, devices_info);
6160: if (ret < 0)
6161: return ERR_PTR(ret);
6162:
6163: ret = decide_stripe_size(fs_devices, &ctl, devices_info);
6164: if (ret < 0)
6165: return ERR_PTR(ret);
6166:
6167: return create_chunk(trans, &ctl, devices_info);
6168: }
SourceFile:fs/btrfs/volumes.c]
|
| 24/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 25937
output: 38
thoughts: 42
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Debugging a Potential Storage Allocation Issue**
Right, so I'm troubleshooting a storage allocation problem. I need to figure out *why* I'm seeing this `-ENOSPC` error. My initial hypothesis is that it could originate from several key areas within the storage subsystem: the `gather_device_info` function, the `decide_stripe_size` function, or the `create_chunk` function. Each of these functions is crucial in the storage allocation process.
To start, I'll drill down into `gather_device_info`. This is where I'll begin my investigation. It's the logical place to start looking for potential causes of `-ENOSPC` because it's the foundation of my available storage pool's information and could be returning insufficient or inaccurate data.
|
| 25/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:gather_device_info]
Results: map[SourceCode:5771: static int gather_device_info(struct btrfs_fs_devices *fs_devices,
5772: struct alloc_chunk_ctl *ctl,
5773: struct btrfs_device_info *devices_info)
5774: {
5775: struct btrfs_fs_info *info = fs_devices->fs_info;
5776: struct btrfs_device *device;
5777: u64 total_avail;
5778: u64 dev_extent_want = ctl->max_stripe_size * ctl->dev_stripes;
5779: int ret;
5780: int ndevs = 0;
5781: u64 max_avail;
5782: u64 dev_offset;
5783:
5784: /*
5785: * in the first pass through the devices list, we gather information
5786: * about the available holes on each device.
5787: */
5788: list_for_each_entry(device, &fs_devices->alloc_list, dev_alloc_list) {
5789: if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state)) {
5790: WARN(1, KERN_ERR
5791: "BTRFS: read-only device in alloc_list\n");
5792: continue;
5793: }
5794:
5795: if (!test_bit(BTRFS_DEV_STATE_IN_FS_METADATA,
5796: &device->dev_state) ||
5797: test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state))
5798: continue;
5799:
5800: if (device->total_bytes > device->bytes_used)
5801: total_avail = device->total_bytes - device->bytes_used;
5802: else
5803: total_avail = 0;
5804:
5805: /* If there is no space on this device, skip it. */
5806: if (total_avail < ctl->dev_extent_min)
5807: continue;
5808:
5809: ret = find_free_dev_extent(device, dev_extent_want, &dev_offset,
5810: &max_avail);
5811: if (ret && ret != -ENOSPC)
5812: return ret;
5813:
5814: if (ret == 0)
5815: max_avail = dev_extent_want;
5816:
5817: if (max_avail < ctl->dev_extent_min) {
5818: if (btrfs_test_opt(info, ENOSPC_DEBUG))
5819: btrfs_debug(info,
5820: "%s: devid %llu has no free space, have=%llu want=%llu",
5821: __func__, device->devid, max_avail,
5822: ctl->dev_extent_min);
5823: continue;
5824: }
5825:
5826: if (ndevs == fs_devices->rw_devices) {
5827: WARN(1, "%s: found more than %llu devices\n",
5828: __func__, fs_devices->rw_devices);
5829: break;
5830: }
5831: devices_info[ndevs].dev_offset = dev_offset;
5832: devices_info[ndevs].max_avail = max_avail;
5833: devices_info[ndevs].total_avail = total_avail;
5834: devices_info[ndevs].dev = device;
5835: ++ndevs;
5836: }
5837: ctl->ndevs = ndevs;
5838:
5839: /*
5840: * now sort the devices by hole size / available space
5841: */
5842: sort(devices_info, ndevs, sizeof(struct btrfs_device_info),
5843: btrfs_cmp_device_info, NULL);
5844:
5845: return 0;
5846: }
SourceFile:fs/btrfs/volumes.c]
|
| 26/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 27456
output: 39
thoughts: 46
Thoughts: Here's my analysis of the situation:
**Decoding the `gather_device_info` Return Value**
Okay, so `gather_device_info` is back. I need to pay attention to its return. It seems a zero return is a good thing – success, likely. But any other return *besides* `-ENOSPC` from `find_free_dev_extent` in that context is an error. So, basically, anything other than a space issue signals a problem with the device info gathering process itself. Now I need to understand what constitutes a successful outcome so I know where to look if something goes wrong. And with that in mind, I need to look at what `decide_stripe_size` is doing.
|
| 27/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:decide_stripe_size]
Results: map[SourceCode:5931: static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
5932: struct alloc_chunk_ctl *ctl,
5933: struct btrfs_device_info *devices_info)
5934: {
5935: struct btrfs_fs_info *info = fs_devices->fs_info;
5936:
5937: /*
5938: * Round down to number of usable stripes, devs_increment can be any
5939: * number so we can't use round_down() that requires power of 2, while
5940: * rounddown is safe.
5941: */
5942: ctl->ndevs = rounddown(ctl->ndevs, ctl->devs_increment);
5943:
5944: if (ctl->ndevs < ctl->devs_min) {
5945: if (btrfs_test_opt(info, ENOSPC_DEBUG)) {
5946: btrfs_debug(info,
5947: "%s: not enough devices with free space: have=%d minimum required=%d",
5948: __func__, ctl->ndevs, ctl->devs_min);
5949: }
5950: return -ENOSPC;
5951: }
5952:
5953: ctl->ndevs = min(ctl->ndevs, ctl->devs_max);
5954:
5955: switch (fs_devices->chunk_alloc_policy) {
5956: default:
5957: btrfs_warn_unknown_chunk_allocation(fs_devices->chunk_alloc_policy);
5958: fallthrough;
5959: case BTRFS_CHUNK_ALLOC_REGULAR:
5960: return decide_stripe_size_regular(ctl, devices_info);
5961: case BTRFS_CHUNK_ALLOC_ZONED:
5962: return decide_stripe_size_zoned(ctl, devices_info);
5963: }
5964: }
SourceFile:fs/btrfs/volumes.c]
|
| 28/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 28305
output: 43
thoughts: 265
Thoughts: **Debugging a Btrfs Chunk Relocation Crash**
Okay, so `decide_stripe_size` is returning `-ENOSPC`, which makes sense; the check confirms insufficient devices with available space. But, hold on... the crash is happening within `btrfs_remove_chunk`? That's unexpected. Why are we even *calling* `btrfs_remove_chunk` in this context?
The stack trace gives us a starting point: it's a cascade from `btrfs_relocate_chunk_finish`, down through `btrfs_relocate_chunk`, `__btrfs_balance`, `btrfs_balance`, and finally culminating in `btrfs_ioctl_balance`. This strongly suggests the crash is somehow related to the balance operation, specifically chunk relocation and its cleanup, but what exactly triggered it? Let's zoom in on `btrfs_relocate_chunk_finish` and figure out what it's supposed to do and what could be failing.
|
| 29/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_relocate_chunk_finish]
Results: map[SourceCode:3581: static int btrfs_relocate_chunk_finish(struct btrfs_fs_info *fs_info,
3582: struct btrfs_block_group *bg)
3583: {
3584: struct btrfs_root *root = fs_info->chunk_root;
3585: struct btrfs_trans_handle *trans;
3586: u64 length;
3587: int ret;
3588:
3589: btrfs_discard_cancel_work(&fs_info->discard_ctl, bg);
3590: length = bg->length;
3591: btrfs_put_block_group(bg);
3592:
3593: /*
3594: * On a zoned file system, discard the whole block group, this will
3595: * trigger a REQ_OP_ZONE_RESET operation on the device zone. If
3596: * resetting the zone fails, don't treat it as a fatal problem from the
3597: * filesystem's point of view.
3598: */
3599: if (btrfs_is_zoned(fs_info)) {
3600: ret = btrfs_discard_extent(fs_info, bg->start, length, NULL, true);
3601: if (ret)
3602: btrfs_info(fs_info, "failed to reset zone %llu after relocation",
3603: bg->start);
3604: }
3605:
3606: trans = btrfs_start_trans_remove_block_group(root->fs_info, bg->start);
3607: if (IS_ERR(trans)) {
3608: ret = PTR_ERR(trans);
3609: btrfs_handle_fs_error(root->fs_info, ret, NULL);
3610: return ret;
3611: }
3612:
3613: /* Step two, delete the device extents and the chunk tree entries. */
3614: ret = btrfs_remove_chunk(trans, bg->start);
3615: btrfs_end_transaction(trans);
3616:
3617: return ret;
3618: }
SourceFile:fs/btrfs/volumes.c]
|
| 30/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 29484
output: 38
thoughts: 62
Thoughts: **Analyzing the Chunk Removal Process**
Okay, so I'm diving into the `btrfs_remove_chunk` function. My understanding is this function's responsible for, well, *removing* a chunk, specifically after it's been relocated. Makes sense. But wait a second... before the actual `remove_chunk_item` call, I see a call to `check_system_chunk`. Hmm, that's interesting. I need to understand why that check is necessary at this point in the process. Time to dig into the `check_system_chunk` function and see what it's up to. This seems like a critical step I need to fully grasp.
|
| 31/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 32/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 30050
output: 40
thoughts: 36
Thoughts: Here's my take on the thought process:
**Diving into the Btrfs Chunk Allocation Logic**
Okay, so I'm trying to trace through this chunk allocation behavior and the first thing I need to do is confirm that the `check_system_chunk` function, which seems central to this, actually lives in the expected place. Right, so `fs/btrfs/block-group.c` is where I'd expect to find it – yep, that's it. Good, that confirms my initial assumptions about its location. Now, let's drill down into the details of the function, `reserve_chunk_space`. That sounds like the key function. Let's see what that function is doing.
|
| 33/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 34/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 31545
output: 41
thoughts: 877
Thoughts: **Analyzing a Btrfs Transaction Abort During Chunk Relocation**
Okay, so I'm digging into this Btrfs issue. It seems like `reserve_chunk_space` is potentially problematic. I see it calls `btrfs_create_chunk`, but crucially, it's a `void` function, so errors from the chunk creation aren't explicitly returned. This means that if `btrfs_create_chunk` fails, `check_system_chunk` ignores the potential failure in `reserve_chunk_space`, which is not optimal.
If `btrfs_create_chunk` fails inside `reserve_chunk_space`, the space isn't reserved. Later, `btrfs_remove_chunk` gets called, which in turn calls `remove_chunk_item`. This `remove_chunk_item` call is where things can go sideways, potentially failing with `-ENOSPC` because there's no space in the system block group to update the chunk tree. And this is where it gets interesting...
`btrfs_remove_chunk` *attempts* to allocate a system chunk *again* if it hits `-ENOSPC`, calling `btrfs_create_chunk` a second time. And here's the kicker: it fails again, and then aborts the transaction!
So, the question is *why* is `btrfs_create_chunk` failing? The syzkaller reproducer gives us some clues. It's calling `BTRFS_IOC_BALANCE_V2`. I note the ioctl order: the reproducer opens a btrfs image, calls `BTRFS_IOC_BALANCE_V2`, and then opens `.` and calls `EXT4_IOC_GROUP_ADD` (probably irrelevant). Also, the image is compressed and it's calling balance with the flags `BTRFS_BALANCE_DATA | BTRFS_BALANCE_METADATA`. Looking at the mount options, it's using compression, and has some interesting options like `enospc_debug`.
Now, I understand this: the filesystem is running out of space. When `btrfs_remove_chunk` tries to update the chunk tree and create a system chunk to mark the old chunk as removed, and there's no space, `btrfs_create_chunk` returns `-ENOSPC`.
My concern is this: is it *correct* to abort the transaction in `btrfs_remove_chunk` if we can't allocate a system chunk? After all, it's removing a chunk, and we might need to allocate a system chunk to update the chunk tree. But if there's no space, `-ENOSPC` is the result.
I see `btrfs_remove_chunk` is actually called from `btrfs_relocate_chunk_finish`, which is called after `btrfs_relocate_chunk`. Therefore, `btrfs_remove_chunk` happens when the chunk has already been relocated. That means a chunk is being balanced, and moved from the devices.
So, if we are balancing and we run out of space on the devices while removing a chunk, it aborts the transaction.
Is this expected? Usually, running out of space should return `-ENOSPC` to the user, not abort the transaction and crash. Maybe because `btrfs_remove_chunk` is called after relocation? I need to look closer at `btrfs_relocate_chunk`.
|
| 35/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_relocate_chunk]
Results: map[SourceCode:3620: int btrfs_relocate_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset, bool verbose)
3621: {
3622: struct btrfs_block_group *block_group;
3623: int ret;
3624:
3625: if (btrfs_fs_incompat(fs_info, EXTENT_TREE_V2)) {
3626: btrfs_err(fs_info,
3627: "relocate: not supported on extent tree v2 yet");
3628: return -EINVAL;
3629: }
3630:
3631: /*
3632: * Prevent races with automatic removal of unused block groups.
3633: * After we relocate and before we remove the chunk with offset
3634: * chunk_offset, automatic removal of the block group can kick in,
3635: * resulting in a failure when calling btrfs_remove_chunk() below.
3636: *
3637: * Make sure to acquire this mutex before doing a tree search (dev
3638: * or chunk trees) to find chunks. Otherwise the cleaner kthread might
3639: * call btrfs_remove_chunk() (through btrfs_delete_unused_bgs()) after
3640: * we release the path used to search the chunk/dev tree and before
3641: * the current task acquires this mutex and calls us.
3642: */
3643: lockdep_assert_held(&fs_info->reclaim_bgs_lock);
3644:
3645: /* step one, relocate all the extents inside this chunk */
3646: btrfs_scrub_pause(fs_info);
3647: ret = btrfs_relocate_block_group(fs_info, chunk_offset, verbose);
3648: btrfs_scrub_continue(fs_info);
3649: if (ret) {
3650: /*
3651: * If we had a transaction abort, stop all running scrubs.
3652: * See transaction.c:cleanup_transaction() why we do it here.
3653: */
3654: if (unlikely(BTRFS_FS_ERROR(fs_info)))
3655: btrfs_scrub_cancel(fs_info);
3656: return ret;
3657: }
3658:
3659: block_group = btrfs_lookup_block_group(fs_info, chunk_offset);
3660: if (!block_group)
3661: return -ENOENT;
3662:
3663: if (should_relocate_using_remap_tree(block_group)) {
3664: /* If we're relocating using the remap tree we're now done. */
3665: btrfs_put_block_group(block_group);
3666: ret = 0;
3667: } else {
3668: ret = btrfs_relocate_chunk_finish(fs_info, block_group);
3669: }
3670:
3671: return ret;
3672: }
SourceFile:fs/btrfs/volumes.c]
|
| 36/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 34123
output: 40
thoughts: 170
Thoughts: **My Reasoning on Btrfs Chunk Relocation**
Okay, so if `btrfs_relocate_block_group` works, the follow-up is `btrfs_relocate_chunk_finish`. Makes sense. Within `btrfs_relocate_chunk_finish`, a transaction kicks off with `btrfs_start_trans_remove_block_group`. The goal, as the name suggests, is to remove the block group at `bg->start`. Then, the function calls `btrfs_remove_chunk(trans, bg->start)`.
Here's where it gets interesting. If `btrfs_remove_chunk` fails, the transaction is *aborted* - it doesn't just return an error. Hold on…why? That seems a bit aggressive. Why the abortion instead of error propagation?
The immediate thought is: has the function already made some modifications to the chunk tree before it even checks the return codes? That would explain the need to back out using the transaction mechanism. It's almost certainly related to the chunk tree and the changes made there. Time to dive back into `btrfs_remove_chunk` and re-evaluate its logic and how it handles potential errors. Let's see what's actually going on.
|
| 37/2 |
2026/08/08 03:31 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_chunk]
Results: map[SourceCode:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
3553: }
3554: }
3555:
3556: btrfs_update_per_profile_avail(fs_info);
3557: mutex_unlock(&fs_info->chunk_mutex);
3558: trans->removing_chunk = false;
3559:
3560: /*
3561: * We are done with chunk btree updates and deletions, so release the
3562: * system space we previously reserved (with check_system_chunk()).
3563: */
3564: btrfs_trans_release_chunk_metadata(trans);
3565:
3566: /* On error, btrfs_remove_block_group() aborts the transaction. */
3567: ret = btrfs_remove_block_group(trans, map);
3568: if (unlikely(ret))
3569: ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
3570:
3571: out:
3572: if (trans->removing_chunk) {
3573: mutex_unlock(&fs_info->chunk_mutex);
3574: trans->removing_chunk = false;
3575: }
3576: /* once for us */
3577: btrfs_free_chunk_map(map);
3578: return ret;
3579: }
SourceFile:fs/btrfs/volumes.c]
|
| 38/2 |
2026/08/08 03:31 |
llm |
debugger |
0m
Model:
Tokens:
input: 36705
output: 43
thoughts: 613
Thoughts: **Analyzing the btrfs Chunk Removal Process**
Okay, so I'm looking at `btrfs_remove_chunk`. My initial thought is that it calls `btrfs_remove_dev_extents(trans, map)` first. Ah, right, that function removes the device extents from the device tree. Makes sense, that's the first step. If *that* succeeds, then the process moves on to removing the chunk item from the chunk tree.
But here's where it gets interesting. If removing the chunk item fails because we need to allocate a system chunk, and there's no space available, the transaction is aborted. My immediate reaction is, "Why the abort?" Because `btrfs_remove_dev_extents` has already messed with the device tree! If we just returned `-ENOSPC` in the error case, we'd end up with a transaction that *commits* with the device extents gone, but the chunk item *still present*. That's clearly an inconsistent state, and obviously something we need to avoid at all costs in a transaction-based system like btrfs. Therefore, we *must* abort the transaction.
But wait a minute, the next question is, why did we even *start* removing the chunk in the first place if we didn't have space to allocate a system chunk? Then I remember: `check_system_chunk` ignores the error from `reserve_chunk_space`! That's a potential area for improvement, it's just a diagnostic. If `check_system_chunk` returned an error, we could fail *before* we modify the device tree.
Right, then I'm checking the flow. Let's see...
```c
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
So first `btrfs_remove_dev_extents` modifies the device tree. Then, we check for system chunk space, and *then* if we don't have enough space, `remove_chunk_item` fails with `-ENOSPC`. We go to allocate the system chunk, it fails with `-ENOSPC`, and the transaction is aborted.
The next question: Why is `btrfs_remove_dev_extents` called *before* `check_system_chunk`? Is it related to the `chunk_mutex`? Nope, `check_system_chunk` *requires* the `chunk_mutex` to be held, so it must be called second. But `btrfs_remove_dev_extents` doesn't hold the `chunk_mutex`.
Can `btrfs_remove_dev_extents` itself return an error? Yes, it can. If `btrfs_remove_dev_extents` returns an error, we return that error and the transaction is NOT aborted. This means that *only* the failure of the chunk item removal/allocation, *after* dev extents have been removed, triggers the abort. But... and I need to be sure about this... if `btrfs_remove_dev_extents` *modifies* the device tree and then returns an error, the transaction doesn't get aborted. I need to be sure that `btrfs_remove_dev_extents` is atomic from that perspective, else we have a problem.
|
| 39/2 |
2026/08/08 03:32 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 40/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 39017
output: 40
thoughts: 187
Thoughts: **Analyzing the Btrfs Device Removal Process**
Okay, so `btrfs_remove_dev_extents` is designed to abort the entire transaction if `btrfs_free_dev_extent` fails. That makes sense, ensuring consistency. But the critical point here is that if we can't remove the chunk item – say, that `btrfs_free_dev_extent` returns a failure, specifically a `-ENOSPC` error – the whole thing blows up. My concern is the root cause of this `-ENOSPC` within the chunk item removal process.
The crux of the issue appears to stem from the potential need to COW (Copy-On-Write) a node within the chunk tree during chunk item removal. That operation itself could require allocating a brand-new metadata block. And, *that* allocation is where things go sideways: it hits the chunk block reserve. If that reserve is insufficient, the system attempts to allocate a *new* system chunk to fulfill the request. If there's no available, unallocated space on the underlying devices to form a system chunk, then the dreaded `-ENOSPC` gets returned.
But...wait a second. Shouldn't `check_system_chunk` have *already* reserved the space necessary for this? That's the expected behavior, isn't it? Something seems off, there's a disconnect. I need to revisit the `check_system_chunk` logic and analyze it with a fresh eye to pinpoint where things are going wrong. I need to be sure that it is reserving enough space in the right cases.
|
| 41/2 |
2026/08/08 03:32 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 42/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 39903
output: 43
thoughts: 110
Thoughts: **Analyzing Potential Error Handling Issues**
Okay, so I'm looking at this code and something seems off. If `reserve_chunk_space` can't get a system chunk, it just silently... doesn't reserve the space. That's not ideal. Then, `remove_chunk_item` comes along, and if *it* needs space to do *its* thing, it *might* fail with `-ENOSPC`. Wait a minute... shouldn't `reserve_chunk_space` be the gatekeeper here? If it fails to reserve the space initially, shouldn't it return an error code itself? That way, `check_system_chunk` could gracefully propagate the error up the call stack. It feels like we're relying on a later function to detect the underlying problem, and that's not robust error handling. I need to dig into the history of both `check_system_chunk` and `reserve_chunk_space` to understand the rationale behind this design. Maybe there's a reason for this, or maybe it's just a subtle oversight that's ripe for improvement. Let's see what the past has to say.
|
| 43/2 |
2026/08/08 03:32 |
tool |
git-log |
0m
Args: map[Count:10 SourcePath:fs/btrfs/block-group.c SymbolName:check_system_chunk]
Results: map[Output:2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
]
|
| 44/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 40341
output: 25
thoughts: 18
Thoughts: Okay, let's dive into this. I'm taking a look at commit `2bb2e00ed978`. My initial thought process involves a few key aspects, which I'll break down:
**Analyzing Commit `2bb2e00ed978`**
First, I need to understand the context. What project is this? What branch? What's the general state of the codebase? Knowing the surrounding environment helps to quickly assess the potential impact of the changes. I'll automatically start by mentally pulling up the surrounding code, the most recent prior commits, and relevant issues, if any are referenced.
Next, I'll be assessing the "size" of the commit. Is this a single, focused change, or a massive restructuring? A large commit raises immediate flags; it'll demand more careful scrutiny. A series of smaller, atomic commits are usually preferable. I will look for clues in the commit message to guide my analysis.
Now, into the changes themselves. I'll be using my expert knowledge to immediately identify areas that are high-risk or potentially problematic. I'll look for:
* **Security vulnerabilities:** Are there any obvious security holes introduced? Are there any new dependencies, and are they properly vetted? I'm thinking of buffer overflows, injection attacks, cross-site scripting (XSS), and privilege escalation.
* **Performance implications:** Does the change introduce any performance bottlenecks? Are there inefficient loops, unnecessary allocations, or poorly-chosen algorithms? I'll be comparing this code to the previous implementation, evaluating for efficiency.
* **Architectural changes:** Are there significant alterations to the design? Are there new abstractions, or is the code becoming less maintainable? Is the commit adhering to the overall architecture of the system?
* **Potential bugs:** Are there any obvious logic errors, off-by-one errors, or incorrect handling of edge cases? Do the code changes seem to be correct, or are there any inconsistencies that may cause issues?
* **Test coverage:** Does the commit include sufficient tests to verify the changes? Are the tests well-written, covering both positive and negative scenarios? Are there any new tests created to cover changes? If not, why?
The ultimate goal is to verify that this commit is sound and safe. It must not introduce bugs, security vulnerabilities, performance regressions, or architectural issues. I'll make sure the team that generated this code has not cut corners.
Essentially, I'm mentally running the code, tracing the logic, and simulating how this will behave in the real world. I need to be confident that this commit can be integrated without causing problems down the line. I'll start by making the relevant assumptions based on my knowledge of the codebase, then move to verification.
|
| 45/2 |
2026/08/08 03:32 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed Oct 13 10:12:49 2021 +0100
btrfs: fix deadlock between chunk allocation and chunk btree modifications
When a task is doing some modification to the chunk btree and it is not in
the context of a chunk allocation or a chunk removal, it can deadlock with
another task that is currently allocating a new data or metadata chunk.
These contexts are the following:
* When relocating a system chunk, when we need to COW the extent buffers
that belong to the chunk btree;
* When adding a new device (ioctl), where we need to add a new device item
to the chunk btree;
* When removing a device (ioctl), where we need to remove a device item
from the chunk btree;
* When resizing a device (ioctl), where we need to update a device item in
the chunk btree and may need to relocate a system chunk that lies beyond
the new device size when shrinking a device.
The problem happens due to a sequence of steps like the following:
1) Task A starts a data or metadata chunk allocation and it locks the
chunk mutex;
2) Task B is relocating a system chunk, and when it needs to COW an extent
buffer of the chunk btree, it has locked both that extent buffer as
well as its parent extent buffer;
3) Since there is not enough available system space, either because none
of the existing system block groups have enough free space or because
the only one with enough free space is in RO mode due to the relocation,
task B triggers a new system chunk allocation. It blocks when trying to
acquire the chunk mutex, currently held by task A;
4) Task A enters btrfs_chunk_alloc_add_chunk_item(), in order to insert
the new chunk item into the chunk btree and update the existing device
items there. But in order to do that, it has to lock the extent buffer
that task B locked at step 2, or its parent extent buffer, but task B
is waiting on the chunk mutex, which is currently locked by task A,
therefore resulting in a deadlock.
One example report when the deadlock happens with system chunk relocation:
INFO: task kworker/u9:5:546 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u9:5 state:D stack:25936 pid: 546 ppid: 2 flags:0x00004000
Workqueue: events_unbound btrfs_async_reclaim_metadata_space
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
rwsem_down_read_slowpath+0x4ee/0x9d0 kernel/locking/rwsem.c:993
__down_read_common kernel/locking/rwsem.c:1214 [inline]
__down_read kernel/locking/rwsem.c:1223 [inline]
down_read_nested+0xe6/0x440 kernel/locking/rwsem.c:1590
__btrfs_tree_read_lock+0x31/0x350 fs/btrfs/locking.c:47
btrfs_tree_read_lock fs/btrfs/locking.c:54 [inline]
btrfs_read_lock_root_node+0x8a/0x320 fs/btrfs/locking.c:191
btrfs_search_slot_get_root fs/btrfs/ctree.c:1623 [inline]
btrfs_search_slot+0x13b4/0x2140 fs/btrfs/ctree.c:1728
btrfs_update_device+0x11f/0x500 fs/btrfs/volumes.c:2794
btrfs_chunk_alloc_add_chunk_item+0x34d/0xea0 fs/btrfs/volumes.c:5504
do_chunk_alloc fs/btrfs/block-group.c:3408 [inline]
btrfs_chunk_alloc+0x84d/0xf50 fs/btrfs/block-group.c:3653
flush_space+0x54e/0xd80 fs/btrfs/space-info.c:670
btrfs_async_reclaim_metadata_space+0x396/0xa90 fs/btrfs/space-info.c:953
process_one_work+0x9df/0x16d0 kernel/workqueue.c:2297
worker_thread+0x90/0xed0 kernel/workqueue.c:2444
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
INFO: task syz-executor:9107 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor state:D stack:23200 pid: 9107 ppid: 7792 flags:0x00004004
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
__mutex_lock_common kernel/locking/mutex.c:669 [inline]
__mutex_lock+0xc96/0x1680 kernel/locking/mutex.c:729
btrfs_chunk_alloc+0x31a/0xf50 fs/btrfs/block-group.c:3631
find_free_extent_update_loop fs/btrfs/extent-tree.c:3986 [inline]
find_free_extent+0x25cb/0x3a30 fs/btrfs/extent-tree.c:4335
btrfs_reserve_extent+0x1f1/0x500 fs/btrfs/extent-tree.c:4415
btrfs_alloc_tree_block+0x203/0x1120 fs/btrfs/extent-tree.c:4813
__btrfs_cow_block+0x412/0x1620 fs/btrfs/ctree.c:415
btrfs_cow_block+0x2f6/0x8c0 fs/btrfs/ctree.c:570
btrfs_search_slot+0x1094/0x2140 fs/btrfs/ctree.c:1768
relocate_tree_block fs/btrfs/relocation.c:2694 [inline]
relocate_tree_blocks+0xf73/0x1770 fs/btrfs/relocation.c:2757
relocate_block_group+0x47e/0xc70 fs/btrfs/relocation.c:3673
btrfs_relocate_block_group+0x48a/0xc60 fs/btrfs/relocation.c:4070
btrfs_relocate_chunk+0x96/0x280 fs/btrfs/volumes.c:3181
__btrfs_balance fs/btrfs/volumes.c:3911 [inline]
btrfs_balance+0x1f03/0x3cd0 fs/btrfs/volumes.c:4301
btrfs_ioctl_balance+0x61e/0x800 fs/btrfs/ioctl.c:4137
btrfs_ioctl+0x39ea/0x7b70 fs/btrfs/ioctl.c:4949
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:874 [inline]
__se_sys_ioctl fs/ioctl.c:860 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:860
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
So fix this by making sure that whenever we try to modify the chunk btree
and we are neither in a chunk allocation context nor in a chunk remove
context, we reserve system space before modifying the chunk btree.
Reported-by: Hao Sun <sunhao.th@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/CACkBjsax51i4mu6C0C3vJqQN3NR_iVuucoeG3U1HXjrgzn5FFQ@mail.gmail.com/
Fixes: 79bd37120b1495 ("btrfs: rework chunk allocation to avoid exhaustion of the system chunk array")
CC: stable@vger.kernel.org # 5.14+
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index de9aeb3733cf..f971d043469c 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3425,25 +3425,6 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
goto out;
}
- /*
- * If this is a system chunk allocation then stop right here and do not
- * add the chunk item to the chunk btree. This is to prevent a deadlock
- * because this system chunk allocation can be triggered while COWing
- * some extent buffer of the chunk btree and while holding a lock on a
- * parent extent buffer, in which case attempting to insert the chunk
- * item (or update the device item) would result in a deadlock on that
- * parent extent buffer. In this case defer the chunk btree updates to
- * the second phase of chunk allocation and keep our reservation until
- * the second phase completes.
- *
- * This is a rare case and can only be triggered by the very few cases
- * we have where we need to touch the chunk btree outside chunk allocation
- * and chunk removal. These cases are basically adding a device, removing
- * a device or resizing a device.
- */
- if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
- return 0;
-
ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
/*
* Normally we are not expected to fail with -ENOSPC here, since we have
@@ -3576,14 +3557,14 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
* This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
* the system chunk array due to concurrent allocations") provides more details.
*
- * For allocation of system chunks, we defer the updates and insertions into the
- * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
- * if the chunk allocation is triggered while COWing an extent buffer of the
- * chunk btree, we are holding a lock on the parent of that extent buffer and
- * doing the chunk btree updates and insertions can require locking that parent.
- * This is for the very few and rare cases where we update the chunk btree that
- * are not chunk allocation or chunk removal: adding a device, removing a device
- * or resizing a device.
+ * Allocation of system chunks does not happen through this function. A task that
+ * needs to update the chunk btree (the only btree that uses system chunks), must
+ * preallocate chunk space by calling either check_system_chunk() or
+ * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
+ * metadata chunk or when removing a chunk, while the later is used before doing
+ * a modification to the chunk btree - use cases for the later are adding,
+ * removing and resizing a device as well as relocation of a system chunk.
+ * See the comment below for more details.
*
* The reservation of system space, done through check_system_chunk(), as well
* as all the updates and insertions into the chunk btree must be done while
@@ -3620,11 +3601,27 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
if (trans->allocating_chunk)
return -ENOSPC;
/*
- * If we are removing a chunk, don't re-enter or we would deadlock.
- * System space reservation and system chunk allocation is done by the
- * chunk remove operation (btrfs_remove_chunk()).
+ * Allocation of system chunks can not happen through this path, as we
+ * could end up in a deadlock if we are allocating a data or metadata
+ * chunk and there is another task modifying the chunk btree.
+ *
+ * This is because while we are holding the chunk mutex, we will attempt
+ * to add the new chunk item to the chunk btree or update an existing
+ * device item in the chunk btree, while the other task that is modifying
+ * the chunk btree is attempting to COW an extent buffer while holding a
+ * lock on it and on its parent - if the COW operation triggers a system
+ * chunk allocation, then we can deadlock because we are holding the
+ * chunk mutex and we may need to access that extent buffer or its parent
+ * in order to add the chunk item or update a device item.
+ *
+ * Tasks that want to modify the chunk tree should reserve system space
+ * before updating the chunk btree, by calling either
+ * btrfs_reserve_chunk_metadata() or check_system_chunk().
+ * It's possible that after a task reserves the space, it still ends up
+ * here - this happens in the cases described above at do_chunk_alloc().
+ * The task will have to either retry or fail.
*/
- if (trans->removing_chunk)
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
@@ -3723,17 +3720,14 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-/*
- * Reserve space in the system space for allocating or removing a chunk
- */
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+static void reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
- u64 thresh;
int ret = 0;
- u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -3746,19 +3740,13 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
- num_devs = get_profile_num_devs(fs_info, type);
-
- /* num_devs device items to update and 1 chunk item to add or remove */
- thresh = btrfs_calc_metadata_size(fs_info, num_devs) +
- btrfs_calc_insert_metadata_size(fs_info, 1);
-
- if (left < thresh && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
- left, thresh, type);
+ left, bytes, type);
btrfs_dump_space_info(fs_info, info, 0, 0);
}
- if (left < thresh) {
+ if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
@@ -3767,21 +3755,20 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
- *
- * Also, if our caller is allocating a system chunk, do not
- * attempt to insert the chunk item in the chunk btree, as we
- * could deadlock on an extent buffer since our caller may be
- * COWing an extent buffer from the chunk btree.
*/
bg = btrfs_create_chunk(trans, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
- } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ } else {
/*
* If we fail to add the chunk item here, we end up
* trying again at phase 2 of chunk allocation, at
* btrfs_create_pending_block_groups(). So ignore
- * any error here.
+ * any error here. An ENOSPC here could happen, due to
+ * the cases described at do_chunk_alloc() - the system
+ * block group we just created was just turned into RO
+ * mode by a scrub for example, or a running discard
+ * temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
@@ -3790,12 +3777,61 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (!ret) {
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
- thresh, BTRFS_RESERVE_NO_FLUSH);
+ bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
- trans->chunk_bytes_reserved += thresh;
+ trans->chunk_bytes_reserved += bytes;
}
}
+/*
+ * Reserve space in the system space for allocating or removing a chunk.
+ * The caller must be holding fs_info->chunk_mutex.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ const u64 num_devs = get_profile_num_devs(fs_info, type);
+ u64 bytes;
+
+ /* num_devs device items to update and 1 chunk item to add or remove. */
+ bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
+ btrfs_calc_insert_metadata_size(fs_info, 1);
+
+ reserve_chunk_space(trans, bytes, type);
+}
+
+/*
+ * Reserve space in the system space, if needed, for doing a modification to the
+ * chunk btree.
+ *
+ * @trans: A transaction handle.
+ * @is_item_insertion: Indicate if the modification is for inserting a new item
+ * in the chunk btree or if it's for the deletion or update
+ * of an existing item.
+ *
+ * This is used in a context where we need to update the chunk btree outside
+ * block group allocation and removal, to avoid a deadlock with a concurrent
+ * task that is allocating a metadata or data block group and therefore needs to
+ * update the chunk btree while holding the chunk mutex. After the update to the
+ * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
+ *
+ */
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ u64 bytes;
+
+ if (is_item_insertion)
+ bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
+ else
+ bytes = btrfs_calc_metadata_size(fs_info, 1);
+
+ mutex_lock(&fs_info->chunk_mutex);
+ reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
+ mutex_unlock(&fs_info->chunk_mutex);
+}
+
void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
{
struct btrfs_block_group *block_group;
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 07f977d3816c..5878b7ce3b78 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -293,6 +293,8 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index fed823596248..33a0ee7ac590 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -2692,8 +2692,12 @@ static int relocate_tree_block(struct btrfs_trans_handle *trans,
list_add_tail(&node->list, &rc->backref_cache.changed);
} else {
path->lowest_level = node->level;
+ if (root == root->fs_info->chunk_root)
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, key, path, 0, 1);
btrfs_release_path(path);
+ if (root == root->fs_info->chunk_root)
+ btrfs_trans_release_chunk_metadata(trans);
if (ret > 0)
ret = 0;
}
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index debba6f04858..9eab8a741166 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1847,8 +1847,10 @@ static int btrfs_add_dev_item(struct btrfs_trans_handle *trans,
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, true);
ret = btrfs_insert_empty_item(trans, trans->fs_info->chunk_root, path,
&key, sizeof(*dev_item));
+ btrfs_trans_release_chunk_metadata(trans);
if (ret)
goto out;
@@ -1921,7 +1923,9 @@ static int btrfs_rm_dev_item(struct btrfs_device *device)
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret) {
if (ret > 0)
ret = -ENOENT;
@@ -2513,7 +2517,9 @@ static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
key.type = BTRFS_DEV_ITEM_KEY;
while (1) {
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0)
goto error;
@@ -2862,6 +2868,7 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
struct btrfs_super_block *super_copy = fs_info->super_copy;
u64 old_total;
u64 diff;
+ int ret;
if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state))
return -EACCES;
@@ -2890,7 +2897,11 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
&trans->transaction->dev_update_list);
mutex_unlock(&fs_info->chunk_mutex);
- return btrfs_update_device(trans, device);
+ btrfs_reserve_chunk_metadata(trans, false);
+ ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
}
static int btrfs_free_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
@@ -4925,8 +4936,10 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
round_down(old_total - diff, fs_info->sectorsize));
mutex_unlock(&fs_info->chunk_mutex);
+ btrfs_reserve_chunk_metadata(trans, false);
/* Now btrfs_update_device() will change the on-disk size. */
ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0) {
btrfs_abort_transaction(trans, ret);
btrfs_end_transaction(trans);
]
|
| 46/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 48489
output: 43
thoughts: 1457
Thoughts: **Analyzing Btrfs Chunk Removal with Resource Constraints**
Okay, so I'm digging into this `check_system_chunk` function and its interactions during chunk removal in Btrfs. It seems a bit convoluted, and I'm trying to figure out the logic flow and pinpoint potential issues.
Initially, I see the comment stating that errors from `btrfs_create_chunk` are ignored in `check_system_chunk` because a retry might happen later. My immediate thought is, if there's no space for the initial system chunk allocation, a subsequent allocation will also likely fail, and this doesn't seem to be handled correctly, which is confirmed later in the code. `reserve_chunk_space` can't reserve space if it can't create the chunk, which leads to `btrfs_remove_chunk` and then `remove_chunk_item` failing with `-ENOSPC`. The code then tries to allocate the system chunk again, fails again (with `-ENOSPC`), and aborts the transaction.
My immediate question: why does the system just abort the transaction instead of returning an error? It's clear that it has to abort because `btrfs_remove_dev_extents` was already called, meaning the device tree has been modified. Returning `-ENOSPC` would leave the file system in a corrupted state, with some changes applied and others not. So, aborting is the safe choice.
However, this makes me question the process: why are we *removing* the chunk if we don't have the space in the first place? I see that it's called after `btrfs_relocate_block_group` has successfully moved the chunk's extents, meaning it's empty. Now we want to remove it, but this removal involves updating the chunk tree, and this can require system chunk allocation. The question then becomes, if allocation fails, what happens? We are seemingly stuck with an empty chunk, but we've already removed the extents, so what about just leaving the empty chunk?
The problem now lies in the fact that `btrfs_remove_chunk` already called `btrfs_remove_dev_extents`. Ideally, we would need to check for available space *before* this call to `btrfs_remove_dev_extents`, since it modifies the device tree. But, `check_system_chunk` is actually called *after* `btrfs_remove_dev_extents`.
Why is `check_system_chunk` called after that? It's because `btrfs_remove_dev_extents` needs `device_list_mutex`, and `check_system_chunk` requires `chunk_mutex`. Now, I ask myself if we can reserve the space before modifying anything.
I then consider the possibility of checking system chunk space *before* calling `btrfs_remove_dev_extents`. This would mean holding `chunk_mutex` during `btrfs_remove_dev_extents`. The comments explicitly say *not* to do this: holding `chunk_mutex` while in `btrfs_remove_dev_extents` can lead to a deadlock! This is because, while removing the device extents, we might need to allocate a new metadata chunk, which might need to lock `chunk_mutex` again. So that is not an option.
Now I am back to the logic flow: We *must* call `btrfs_remove_dev_extents` without holding `chunk_mutex`. But if `btrfs_remove_dev_extents` needs to allocate a metadata chunk, it may need to allocate a system chunk as well. The question now is: What if `btrfs_remove_dev_extents` does *not* need a metadata chunk, but `remove_chunk_item` does? In this scenario, `remove_chunk_item` would need system space. And here is where we get back to the current scenario: If `check_system_chunk` fails to allocate it, it proceeds to ignore the error. Then, `remove_chunk_item` fails with `-ENOSPC`. Then, `btrfs_remove_chunk` allocates the chunk again, fails again and aborts.
And here's the crux: If `check_system_chunk` fails to allocate a system chunk, it means there is no space on the devices. But if there is no space on the devices, we cannot allocate. However, we're *removing* a chunk. Removing the chunk should *free* space. The question becomes: Does `btrfs_remove_dev_extents` (that's the function that frees the device extents) make the space available immediately for allocation of a new system chunk? This
|
| 47/2 |
2026/08/08 03:32 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 48/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 51883
output: 40
thoughts: 198
Thoughts: **Debugging a Btrfs Chunk Creation Failure**
Okay, so `btrfs_remove_dev_extents` *should* be freeing up space on the device. Great! That `device->bytes_used` variable is definitely getting decremented by the correct `dev_extent_len` amount. Fantastic! Which means, logically, that space *should* now be available for a new chunk. But then… why the hell am I still seeing `-ENOSPC` errors during `btrfs_create_chunk`? This makes no sense!
Right, time to retrace our steps through `btrfs_create_chunk`. It calls `gather_device_info`, which, as I recall, calculates `device->total_bytes - device->bytes_used` to determine available space. But *I* just decreased `bytes_used`! So, `total_avail` should definitely be large enough, theoretically, right?
Alright, let's keep going. `gather_device_info` calls `find_free_dev_extent`, which is supposed to locate the free space. It takes the device, the desired extent size (`dev_extent_want`), and some output pointers for offset and maximum availability.
Now, does `find_free_dev_extent` actually *find* the space that I just freed? I need to dig into `find_free_dev_extent` and see what's going on in *there*. Let's go!
|
| 49/2 |
2026/08/08 03:32 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:find_free_dev_extent]
Results: map[SourceCode:1787: /*
1788: * Find free space in the specified device.
1789: *
1790: * @device: the device which we search the free space in
1791: * @num_bytes: the size of the free space that we need
1792: * @search_start: the position from which to begin the search
1793: * @start: store the start of the free space.
1794: * @len: the size of the free space. that we find, or the size
1795: * of the max free space if we don't find suitable free space
1796: *
1797: * This does a pretty simple search, the expectation is that it is called very
1798: * infrequently and that a given device has a small number of extents.
1799: *
1800: * @start is used to store the start of the free space if we find. But if we
1801: * don't find suitable free space, it will be used to store the start position
1802: * of the max free space.
1803: *
1804: * @len is used to store the size of the free space that we find.
1805: * But if we don't find suitable free space, it is used to store the size of
1806: * the max free space.
1807: *
1808: * NOTE: This function will search *commit* root of device tree, and does extra
1809: * check to ensure dev extents are not double allocated.
1810: * This makes the function safe to allocate dev extents but may not report
1811: * correct usable device space, as device extent freed in current transaction
1812: * is not reported as available.
1813: */
1814: static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
1815: u64 *start, u64 *len)
1816: {
1817: struct btrfs_fs_info *fs_info = device->fs_info;
1818: struct btrfs_root *root = fs_info->dev_root;
1819: struct btrfs_key key;
1820: struct btrfs_dev_extent *dev_extent;
1821: BTRFS_PATH_AUTO_FREE(path);
1822: u64 search_start;
1823: u64 hole_size;
1824: u64 max_hole_start;
1825: u64 max_hole_size = 0;
1826: u64 extent_end;
1827: u64 search_end = device->total_bytes;
1828: int ret;
1829: int slot;
1830: struct extent_buffer *l;
1831:
1832: search_start = dev_extent_search_start(device);
1833: max_hole_start = search_start;
1834:
1835: WARN_ON(device->zone_info &&
1836: !IS_ALIGNED(num_bytes, device->zone_info->zone_size));
1837:
1838: path = btrfs_alloc_path();
1839: if (!path) {
1840: ret = -ENOMEM;
1841: goto out;
1842: }
1843:
1844: if (search_start >= search_end ||
1845: test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
1846: ret = -ENOSPC;
1847: goto out;
1848: }
1849:
1850: path->reada = READA_FORWARD;
1851: path->search_commit_root = true;
1852: path->skip_locking = true;
1853:
1854: key.objectid = device->devid;
1855: key.type = BTRFS_DEV_EXTENT_KEY;
1856: key.offset = search_start;
1857:
1858: ret = btrfs_search_backwards(root, &key, path);
1859: if (ret < 0)
1860: goto out;
1861:
1862: while (search_start < search_end) {
1863: l = path->nodes[0];
1864: slot = path->slots[0];
1865: if (slot >= btrfs_header_nritems(l)) {
1866: ret = btrfs_next_leaf(root, path);
1867: if (ret == 0)
1868: continue;
1869: if (ret < 0)
1870: goto out;
1871:
1872: break;
1873: }
1874: btrfs_item_key_to_cpu(l, &key, slot);
1875:
1876: if (key.objectid < device->devid)
1877: goto next;
1878:
1879: if (key.objectid > device->devid)
1880: break;
1881:
1882: if (key.type != BTRFS_DEV_EXTENT_KEY)
1883: goto next;
1884:
1885: if (key.offset > search_end)
1886: break;
1887:
1888: if (key.offset > search_start) {
1889: hole_size = key.offset - search_start;
1890: dev_extent_hole_check(device, &search_start, &hole_size,
1891: num_bytes);
1892:
1893: if (hole_size > max_hole_size) {
1894: max_hole_start = search_start;
1895: max_hole_size = hole_size;
1896: }
1897:
1898: /*
1899: * If this free space is greater than which we need,
1900: * it must be the max free space that we have found
1901: * until now, so max_hole_start must point to the start
1902: * of this free space and the length of this free space
1903: * is stored in max_hole_size. Thus, we return
1904: * max_hole_start and max_hole_size and go back to the
1905: * caller.
1906: */
1907: if (hole_size >= num_bytes) {
1908: ret = 0;
1909: goto out;
1910: }
1911: }
1912:
1913: dev_extent = btrfs_item_ptr(l, slot, struct btrfs_dev_extent);
1914: extent_end = key.offset + btrfs_dev_extent_length(l,
1915: dev_extent);
1916: if (extent_end > search_start)
1917: search_start = extent_end;
1918: next:
1919: path->slots[0]++;
1920: cond_resched();
1921: }
1922:
1923: /*
1924: * At this point, search_start should be the end of
1925: * allocated dev extents, and when shrinking the device,
1926: * search_end may be smaller than search_start.
1927: */
1928: if (search_end > search_start) {
1929: hole_size = search_end - search_start;
1930: dev_extent_hole_check(device, &search_start, &hole_size, num_bytes);
1931:
1932: if (hole_size > max_hole_size) {
1933: max_hole_start = search_start;
1934: max_hole_size = hole_size;
1935: }
1936: }
1937:
1938: /* See above. */
1939: if (max_hole_size < num_bytes)
1940: ret = -ENOSPC;
1941: else
1942: ret = 0;
1943:
1944: ASSERT(max_hole_start + max_hole_size <= search_end,
1945: "max_hole_start=%llu max_hole_size=%llu search_end=%llu",
1946: max_hole_start, max_hole_size, search_end);
1947: out:
1948: *start = max_hole_start;
1949: if (len)
1950: *len = max_hole_size;
1951: return ret;
1952: }
SourceFile:fs/btrfs/volumes.c]
|
| 50/2 |
2026/08/08 03:32 |
llm |
debugger |
0m
Model:
Tokens:
input: 54990
output: 40
thoughts: 1702
Thoughts: **Root Cause of ENOSPC During Chunk Removal**
Okay, so here's the situation. It all boils down to how `find_free_dev_extent` behaves. It searches the *commit* root of the device tree, and this is crucial. The comment's right: it's designed this way to avoid double-allocation, but the downside is that it doesn't see space freed *within the current transaction*. Right, so if we just freed device extents in, say, `btrfs_remove_dev_extents`, that space is invisible to `find_free_dev_extent` immediately.
Here’s the scenario:
1. Filesystem is full.
2. Relocating a chunk, `btrfs_relocate_chunk` is in play.
3. `btrfs_remove_chunk` gets called from relocation, which then leads to `btrfs_remove_dev_extents`. `btrfs_remove_dev_extents` frees device extents, making the chunk's space available.
4. Then, within `btrfs_remove_chunk`, we try to ensure a system chunk is available by calling `check_system_chunk`, which, in turn, attempts a `btrfs_create_chunk` and subsequently `find_free_dev_extent`.
5. Because `find_free_dev_extent` is looking at the *commit* root, it *doesn't* see the freshly freed space, returning `-ENOSPC` because, from its perspective, the device is still full.
6. `check_system_chunk` just ignores the return from `find_free_dev_extent`, which is why we’re stuck. No space is reserved.
7. Now, the `remove_chunk_item` call is next, which needs to COW a node. This metadata allocation also fails due to the lack of free space `-ENOSPC`, because the space we freed is not visible.
8. The return from `remove_chunk_item` is `-ENOSPC` and is caught by `btrfs_remove_chunk`, which then *tries again* to allocate a system chunk! And it fails again for the same reason. This just leads to an abort.
It's a classic `ENOSPC` deadlock: We *need* space, and the space we're making available by removing the chunk can't be found by the allocation routines in the current transaction.
So, how to fix it? The key is to address the space availability *before* modifying the device tree. But, `btrfs_remove_chunk` is called from `btrfs_relocate_chunk_finish`, which starts a *new* transaction!
We can't reserve system space *before* the transaction starts. We need to do it within the transaction. We need to check for enough system space *before* `btrfs_remove_dev_extents` is called. The issue is that we can't call `check_system_chunk` before `btrfs_remove_dev_extents` because if `check_system_chunk` fails *before* it returns `-ENOSPC` the transaction would be aborted and `btrfs_remove_dev_extents` would have already freed the extent, making an incomplete cleanup.
The critical piece here is `check_system_chunk`. Right now, it doesn't return an error; it just fails silently! So we need to modify `check_system_chunk` to return an error code if the space allocation fails.
The reason why `find_free_dev_extent` uses the commit root is to avoid allocating space that might get freed later in an aborted transaction. So we *can't* rely on using the space freed in the *current* transaction. Therefore, when the device is full, if we cannot allocate a new system chunk, we cannot remove the chunk.
The solution is to add a check for the space *before* we call `btrfs_remove_dev_extents`. Then, the modified code will look like:
```c
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
...
```
Let's modify `check_system_chunk` to return `int` and see what happens.
|
| 51/2 |
2026/08/08 03:33 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 52/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 58045
output: 40
thoughts: 423
Thoughts: **Chunk Removal Strategy: A Btrfs Conundrum**
Okay, so I'm looking at this chunk removal process and seeing a potential for unnecessary failures and transaction aborts. The core issue is the interaction between `check_system_chunk`, `reserve_chunk_space`, and `remove_chunk_item`. Initially, I thought if `check_system_chunk` fails, it's an easy abort. But then, `reserve_chunk_space` seems to *ignore* the `btrfs_create_chunk` failure, stating that the system space might not even be needed. So, if we fail early just because of a system space issue, we might be too hasty.
My immediate thought is, let's just proceed. If we *do* need that system space later, we'll hit `-ENOSPC` in `remove_chunk_item`, right? And *then* we can just abort the transaction. But wait... aborting the transaction makes the filesystem read-only, which is the worst possible outcome. It's far better to fail the chunk removal and leave an empty block group. That's harmless.
But then, if we get `-ENOSPC` in `remove_chunk_item`, why can't we *catch* that error, and simply NOT abort? Why are we forced into this read-only state? Ah, right. It's because `btrfs_remove_dev_extents` has *already* been called by that point. Those device extents are GONE. If we try to commit the transaction, we'll have a chunk item still present in the metadata but the device extents it's supposed to point to are already removed. That is outright *corruption*. So, once `btrfs_remove_dev_extents` is called, we *must* succeed in removing the chunk item, or we *have* to abort. This is non-negotiable.
So now, I'm thinking... why does `btrfs_remove_dev_extents` *have* to be called before `remove_chunk_item`? There has to be a reason, let me go back and look at the control flow in `btrfs_remove_chunk` to see why the extents have to be removed first.
|
| 53/2 |
2026/08/08 03:33 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_chunk]
Results: map[SourceCode:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
3553: }
3554: }
3555:
3556: btrfs_update_per_profile_avail(fs_info);
3557: mutex_unlock(&fs_info->chunk_mutex);
3558: trans->removing_chunk = false;
3559:
3560: /*
3561: * We are done with chunk btree updates and deletions, so release the
3562: * system space we previously reserved (with check_system_chunk()).
3563: */
3564: btrfs_trans_release_chunk_metadata(trans);
3565:
3566: /* On error, btrfs_remove_block_group() aborts the transaction. */
3567: ret = btrfs_remove_block_group(trans, map);
3568: if (unlikely(ret))
3569: ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
3570:
3571: out:
3572: if (trans->removing_chunk) {
3573: mutex_unlock(&fs_info->chunk_mutex);
3574: trans->removing_chunk = false;
3575: }
3576: /* once for us */
3577: btrfs_free_chunk_map(map);
3578: return ret;
3579: }
SourceFile:fs/btrfs/volumes.c]
|
| 54/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 61097
output: 42
thoughts: 1091
Thoughts: **Analyzing System Chunk Allocation During Device Removal**
Okay, so I'm diving into the complexities of `btrfs_remove_dev_extents`. My main concern is ensuring consistency. If this function successfully modifies the device tree, then removing the chunk item *must* also succeed. Otherwise, we're in a bad state. I need a way to fail gracefully, but also ensure that I have the space needed for a successful transaction.
My initial thought was to call `check_system_chunk` *before* calling `btrfs_remove_dev_extents`. This seemed like a reasonable pre-flight check for available system chunk space. I could've locked the chunk mutex, checked the system chunk and exited if it failed.
But then I realized that `check_system_chunk` doesn't actually return an error if it fails to *allocate* system space; it only returns an error if it fails to *find* a chunk. So, I thought if I changed `check_system_chunk` to return an error code from the allocation itself, then I could bail out early.
That led me to another potential issue: if I reserved space before `btrfs_remove_dev_extents`, would that reservation be held while `btrfs_remove_dev_extents` runs? Yes, `trans->chunk_bytes_reserved` is just a counter, so that's fine.
But here's a wrinkle: `btrfs_remove_dev_extents` might *itself* need to allocate a metadata block. And, this could trigger a system chunk allocation, which might use up the space I had "reserved".
Wait, hang on. `btrfs_remove_dev_extents` doesn't use `trans->chunk_bytes_reserved`. It uses the normal block reserve. Okay, so if the normal block reserve is empty, it *might* trigger a chunk allocation. But that would allocate a *metadata* chunk, which *does* require system space. And *that* will call `check_system_chunk`! Excellent, and importantly, `btrfs_remove_dev_extents` doesn't hold `chunk_mutex`, so it's safe for it to trigger a chunk allocation.
So, reserving system space before `btrfs_remove_dev_extents` is okay, because it's going to reserve space again if it needs to, and that should be fine. It is an extra step, but at least it's not harmful.
But then I had to go back and understand *why* `check_system_chunk` was ignoring errors from `btrfs_create_chunk` in the first place. The comment in `reserve_chunk_space` explains it: the space might not be needed, since nodes might already be COW'd. Okay, so we *might* not actually need the space. If I change it to return the error, I might fail operations that would have succeeded. I can't know for sure if the space is needed *until* I modify the tree.
Wait, but... If I'm in `btrfs_remove_chunk` and `remove_chunk_item` fails with `-ENOSPC`, then *I know* I need the space! That's when I call `btrfs_create_chunk`, but it fails because `find_free_dev_extent` searches the commit root, so it doesn't see the space just freed by `btrfs_remove_dev_extents`.
Here is the problem: I just freed the space, and then I try to create the system chunk. So, I have the system chunk space, but the allocator can't see it because it isn't committed.
If `find_free_dev_extent` could see the space I just freed, the chunk creation would succeed. This means that if it allocates the space and commits, everything is fine. And, if the transaction aborts, the free is rolled back, and the allocation is rolled back. Yes, it's safe to reuse that freed space, but I'm not sure how to make that happen.
|
| 55/2 |
2026/08/08 03:33 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_free_dev_extent]
Results: map[SourceCode:1954: static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
1955: struct btrfs_device *device,
1956: u64 start, u64 *dev_extent_len)
1957: {
1958: struct btrfs_fs_info *fs_info = device->fs_info;
1959: struct btrfs_root *root = fs_info->dev_root;
1960: int ret;
1961: BTRFS_PATH_AUTO_FREE(path);
1962: struct btrfs_key key;
1963: struct btrfs_key found_key;
1964: struct extent_buffer *leaf = NULL;
1965: struct btrfs_dev_extent *extent = NULL;
1966:
1967: path = btrfs_alloc_path();
1968: if (!path)
1969: return -ENOMEM;
1970:
1971: key.objectid = device->devid;
1972: key.type = BTRFS_DEV_EXTENT_KEY;
1973: key.offset = start;
1974: again:
1975: ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
1976: if (ret > 0) {
1977: ret = btrfs_previous_item(root, path, key.objectid,
1978: BTRFS_DEV_EXTENT_KEY);
1979: if (ret)
1980: return ret;
1981: leaf = path->nodes[0];
1982: btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
1983: extent = btrfs_item_ptr(leaf, path->slots[0],
1984: struct btrfs_dev_extent);
1985: BUG_ON(found_key.offset > start || found_key.offset +
1986: btrfs_dev_extent_length(leaf, extent) < start);
1987: key = found_key;
1988: btrfs_release_path(path);
1989: goto again;
1990: } else if (ret == 0) {
1991: leaf = path->nodes[0];
1992: extent = btrfs_item_ptr(leaf, path->slots[0],
1993: struct btrfs_dev_extent);
1994: } else {
1995: return ret;
1996: }
1997:
1998: *dev_extent_len = btrfs_dev_extent_length(leaf, extent);
1999:
2000: ret = btrfs_del_item(trans, root, path);
2001: if (ret == 0)
2002: set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
2003: return ret;
2004: }
SourceFile:fs/btrfs/volumes.c]
|
| 56/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 64022
output: 57
thoughts: 115
Thoughts: **Investigating Btrfs Free Extent Management**
Okay, so `btrfs_free_dev_extent` is responsible for removing an extent item from the device tree. Got it. And it flags the transaction with `BTRFS_TRANS_HAVE_FREE_BGS`. Now, the crucial question is why `find_free_dev_extent` *ever* searches the commit root, because if it's looking there, it won't see the deleted extent. That seems inherently problematic, right? A search of the commit root shouldn't be impacted by operations performed in the current transaction. This is making me uneasy...
I need to understand *why* `find_free_dev_extent` is even looking at the commit root. It's time to dig into the git history of that function. I need to see when `search_commit_root` was introduced and what the rationale was behind that change. There has to be a reason, and understanding that reason is critical to grasping the context and the potential ramifications of this seemingly disconnected behavior. Let's see what the commit messages have to say.
|
| 57/2 |
2026/08/08 03:33 |
tool |
git-log |
0m
Args: map[CodeRegexp:search_commit_root = true Count:10 SourcePath:fs/btrfs/volumes.c SymbolName:find_free_dev_extent]
Results: map[Output:b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
d7fe41044b3a btrfs: use bool type for btrfs_path members used as booleans
7fc35cc559cb btrfs: more trivial BTRFS_PATH_AUTO_FREE conversions
94cb8d714434 btrfs: use verbose ASSERT() in volumes.c
dba6ae0b439f btrfs: unify ordering of btrfs_key initializations
20218dfbaa31 btrfs: make sure to initialize start and len in find_free_dev_extent
ed8947bc73aa btrfs: merge find_free_dev_extent() and find_free_dev_extent_start()
3c538de0f2a7 btrfs: limit device extents to the device size
0ff40a910f56 btrfs: introduce btrfs_search_backwards function
1cd6121f2a38 btrfs: zoned: implement zoned chunk allocator
]
|
| 58/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 64714
output: 26
thoughts: 17
Thoughts: Okay, here's my attempt at summarizing the thought process, as requested, assuming I'm an expert in a field where I might encounter something like this (e.g., software engineering, data analysis, cryptography).
**Analyzing This Identifier**
Okay, so I'm looking at this identifier, `b14c5e04bd0f`. My initial instinct is to categorize it. It's clearly alphanumeric, and the length strongly suggests it's a hexadecimal representation. Could be a hash value, a unique ID, a GUID, or part of a more complex key. The lack of context is a bit frustrating – is this a timestamp, a transaction ID, a file checksum? Knowing the source would be crucial for a definitive answer.
Given its format, I'm leaning toward a hash or unique identifier. The structure of hex characters is pretty uniform, so I need to decide what it's representing. If I know the origin system, I could check if it is formatted as an MD5, SHA-1, SHA-256 or another hash. Is this data from a known system? Any metadata about where this came from is crucial. The lack of more info means I'm guessing and trying to narrow the scope. Next I'll start checking.
|
| 59/2 |
2026/08/08 03:33 |
tool |
git-show |
0m
Args:
Results: map[Output:commit b14c5e04bd0f722ed631845599d52d03fcae1bc1
Author: Boris Burkov <boris@bur.io>
Date: Thu Jan 29 16:11:21 2026 -0800
btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
I have been observing a number of systems aborting at
insert_dev_extents() in btrfs_create_pending_block_groups(). The
following is a sample stack trace of such an abort coming from forced
chunk allocation (typically behind CONFIG_BTRFS_EXPERIMENTAL) but this
can theoretically happen to any DUP chunk allocation.
[81.801] ------------[ cut here ]------------
[81.801] BTRFS: Transaction aborted (error -17)
[81.801] WARNING: fs/btrfs/block-group.c:2876 at btrfs_create_pending_block_groups+0x721/0x770 [btrfs], CPU#1: bash/319
[81.802] Modules linked in: virtio_net btrfs xor zstd_compress raid6_pq null_blk
[81.803] CPU: 1 UID: 0 PID: 319 Comm: bash Kdump: loaded Not tainted 6.19.0-rc6+ #319 NONE
[81.803] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.17.0-2-2 04/01/2014
[81.804] RIP: 0010:btrfs_create_pending_block_groups+0x723/0x770 [btrfs]
[81.806] RSP: 0018:ffffa36241a6bce8 EFLAGS: 00010282
[81.806] RAX: 000000000000000d RBX: ffff8e699921e400 RCX: 0000000000000000
[81.807] RDX: 0000000002040001 RSI: 00000000ffffffef RDI: ffffffffc0608bf0
[81.807] RBP: 00000000ffffffef R08: ffff8e69830f6000 R09: 0000000000000007
[81.808] R10: ffff8e699921e5e8 R11: 0000000000000000 R12: ffff8e6999228000
[81.808] R13: ffff8e6984d82000 R14: ffff8e69966a69c0 R15: ffff8e69aa47b000
[81.809] FS: 00007fec6bdd9740(0000) GS:ffff8e6b1b379000(0000) knlGS:0000000000000000
[81.809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[81.810] CR2: 00005604833670f0 CR3: 0000000116679000 CR4: 00000000000006f0
[81.810] Call Trace:
[81.810] <TASK>
[81.810] __btrfs_end_transaction+0x3e/0x2b0 [btrfs]
[81.811] btrfs_force_chunk_alloc_store+0xcd/0x140 [btrfs]
[81.811] kernfs_fop_write_iter+0x15f/0x240
[81.812] vfs_write+0x264/0x500
[81.812] ksys_write+0x6c/0xe0
[81.812] do_syscall_64+0x66/0x770
[81.812] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[81.813] RIP: 0033:0x7fec6be66197
[81.814] RSP: 002b:00007fffb159dd30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[81.815] RAX: ffffffffffffffda RBX: 00007fec6bdd9740 RCX: 00007fec6be66197
[81.815] RDX: 0000000000000002 RSI: 0000560483374f80 RDI: 0000000000000001
[81.816] RBP: 0000560483374f80 R08: 0000000000000000 R09: 0000000000000000
[81.816] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002
[81.817] R13: 00007fec6bfb85c0 R14: 00007fec6bfb5ee0 R15: 00005604833729c0
[81.817] </TASK>
[81.817] irq event stamp: 20039
[81.818] hardirqs last enabled at (20047): [<ffffffff99a68302>] __up_console_sem+0x52/0x60
[81.818] hardirqs last disabled at (20056): [<ffffffff99a682e7>] __up_console_sem+0x37/0x60
[81.819] softirqs last enabled at (19470): [<ffffffff999d2b46>] __irq_exit_rcu+0x96/0xc0
[81.819] softirqs last disabled at (19463): [<ffffffff999d2b46>] __irq_exit_rcu+0x96/0xc0
[81.820] ---[ end trace 0000000000000000 ]---
[81.820] BTRFS: error (device dm-7 state A) in btrfs_create_pending_block_groups:2876: errno=-17 Object already exists
Inspecting these aborts with drgn, I observed a pattern of overlapping
chunk_maps. Note how stripe 1 of the first chunk overlaps in physical
address with stripe 0 of the second chunk.
Physical Start Physical End Length Logical Type Stripe
----------------------------------------------------------------------------------------------------
0x0000000102500000 0x0000000142500000 1.0G 0x0000000641d00000 META|DUP 0/2
0x0000000142500000 0x0000000182500000 1.0G 0x0000000641d00000 META|DUP 1/2
0x0000000142500000 0x0000000182500000 1.0G 0x0000000601d00000 META|DUP 0/2
0x0000000182500000 0x00000001c2500000 1.0G 0x0000000601d00000 META|DUP 1/2
Now how could this possibly happen? All chunk allocation is protected by
the chunk_mutex so racing allocations should see a consistent view of
the CHUNK_ALLOCATED bit in the chunk allocation extent-io-tree
(device->alloc_state as set by chunk_map_device_set_bits()) The tree
itself is protected by a spin lock, and clearing/setting the bits is
always protected by fs_info->mapping_tree_lock, so no race is apparent.
It turns out that there is a subtle bug in the logic regarding chunk
allocations that have happened in the current transaction, known as
"pending extents". The chunk allocation as defined in
find_free_dev_extent() is a loop which searches the commit root of the
dev_root and looks for gaps between DEV_EXTENT items. For those gaps, it
then checks alloc_state bitmap for any pending extents and adjusts the
hole that it finds accordingly. However, the logic in that adjustment
assumes that the first pending extent is the only one in that range.
e.g., given a layout with two non-consecutive pending extents in a hole
passed to dev_extent_hole_check() via *hole_start and *hole_size:
|----pending A----| real hole |----pending B----|
| candidate hole |
*hole_start *hole_start + *hole_size
the code incorrectly returns a "hole" from the end of pending extent A
until the passed in hole end, failing to account for pending B.
However, it is not entirely obvious that it is actually possible to
produce such a layout. I was able to reproduce it, but with some
contortions: I continued to use the force chunk allocation sysfs file
and I introduced a long delay (10 seconds) into the start of the cleaner
thread. I also prevented the unused bgs cleaning logic from ever
deleting metadata bgs. These help make it easier to deterministically
produce the condition but shouldn't really matter if you imagine the
conditions happening by race/luck. Allocations/frees can happen
concurrently with the cleaner thread preparing to process an unused
extent and both create some used chunks with an unused chunk
interleaved, all during one transaction. Then btrfs_delete_unused_bgs()
sees the unused one and clears it, leaving a range with several pending
chunk allocations and a gap in the middle.
The basic idea is that the unused_bgs cleanup work happens on a worker
so if we allocate 3 block groups in one transaction, then the cleaner
work kicked off by the previous transaction comes through and deletes
the middle one of the 3, then the commit root shows no dev extents and
we have the bad pattern in the extent-io-tree. One final consideration
is that the code happens to loop to the next hole if there are no more
extents at all, so we need one more dev extent way past the area we are
working in. Something like the following demonstrates the technique:
# push the BG frontier out to 20G
fallocate -l 20G $mnt/foo
# allocate one more that will prevent the "no more dev extents" luck
fallocate -l 1G $mnt/sticky
# sync
sync
# clear out the allocation area
rm $mnt/foo
sync
_cleaner
# let everything quiesce
sleep 20
sync
# dev tree should have one bg 20G out and the rest at the beginning..
# sort of like an empty FS but with a random sticky chunk.
# kick off the cleaner in the background, remember it will sleep 10s
# before doing interesting work
_cleaner &
sleep 3
# create 3 trivial block groups, all empty, all immediately marked as unused.
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
echo 1 > "$(_btrfs_sysfs_space_info $dev data)/force_chunk_alloc"
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
# let the cleaner thread definitely finish, it will remove the data bg
sleep 10
# this allocation sees the non-consecutive pending metadata chunks with
# data chunk gap of 1G and allocates a 2G extent in that hole. ENOSPC!
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
As for the fix, it is not that obvious. I could not see a trivial way to
do it even by adding backup loops into find_free_dev_extent(), so I
opted to change the semantics of dev_extent_hole_check() to not stop
looping until it finds a sufficiently big hole. For clarity, this also
required changing the helper function contains_pending_extent() into two
new helpers which find the first pending extent and the first suitable
hole in a range.
I attempted to clean up the documentation and range calculations to be
as consistent and clear as possible for the future.
I also looked at the zoned case and concluded that the loop there is
different and not to be unified with this one. As far as I can tell, the
zoned check will only further constrain the hole so looping back to find
more holes is acceptable. Though given that zoned really only appends, I
find it highly unlikely that it is susceptible to this bug.
Fixes: 1b9845081633 ("Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole")
Reported-by: Dimitrios Apostolou <jimis@gmx.net>
Closes: https://lore.kernel.org/linux-btrfs/q7760374-q1p4-029o-5149-26p28421s468@tzk.arg/
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index d33780082b8d..329a922893b4 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1509,30 +1509,158 @@ struct btrfs_device *btrfs_scan_one_device(const char *path,
}
/*
- * Try to find a chunk that intersects [start, start + len] range and when one
- * such is found, record the end of it in *start
+ * Find the first pending extent intersecting a range.
+ *
+ * @device: the device to search
+ * @start: start of the range to check
+ * @len: length of the range to check
+ * @pending_start: output pointer for the start of the found pending extent
+ * @pending_end: output pointer for the end of the found pending extent (inclusive)
+ *
+ * Search for a pending chunk allocation that intersects the half-open range
+ * [start, start + len).
+ *
+ * Return: true if a pending extent was found, false otherwise.
+ * If the return value is true, store the first pending extent in
+ * [*pending_start, *pending_end]. Otherwise, the two output variables
+ * may still be modified, to something outside the range and should not
+ * be used.
*/
-static bool contains_pending_extent(struct btrfs_device *device, u64 *start,
- u64 len)
+static bool first_pending_extent(struct btrfs_device *device, u64 start, u64 len,
+ u64 *pending_start, u64 *pending_end)
{
- u64 physical_start, physical_end;
-
lockdep_assert_held(&device->fs_info->chunk_mutex);
- if (btrfs_find_first_extent_bit(&device->alloc_state, *start,
- &physical_start, &physical_end,
+ if (btrfs_find_first_extent_bit(&device->alloc_state, start,
+ pending_start, pending_end,
CHUNK_ALLOCATED, NULL)) {
- if (in_range(physical_start, *start, len) ||
- in_range(*start, physical_start,
- physical_end + 1 - physical_start)) {
- *start = physical_end + 1;
+ if (in_range(*pending_start, start, len) ||
+ in_range(start, *pending_start, *pending_end + 1 - *pending_start)) {
return true;
}
}
return false;
}
+/*
+ * Find the first real hole accounting for pending extents.
+ *
+ * @device: the device containing the candidate hole
+ * @start: input/output pointer for the hole start position
+ * @len: input/output pointer for the hole length
+ * @min_hole_size: the size of hole we are looking for
+ *
+ * Given a potential hole specified by [*start, *start + *len), check for pending
+ * chunk allocations within that range. If pending extents are found, the hole is
+ * adjusted to represent the first true free space that is large enough when
+ * accounting for pending chunks.
+ *
+ * Note that this function must handle various cases involving non consecutive
+ * pending extents.
+ *
+ * Returns: true if a suitable hole was found and false otherwise.
+ * If the return value is true, then *start and *len are set to represent the hole.
+ * If the return value is false, then *start is set to the largest hole we
+ * found and *len is set to its length.
+ * If there are no holes at all, then *start is set to the end of the range and
+ * *len is set to 0.
+ */
+static bool find_hole_in_pending_extents(struct btrfs_device *device, u64 *start,
+ u64 *len, u64 min_hole_size)
+{
+ u64 pending_start, pending_end;
+ u64 end;
+ u64 max_hole_start = 0;
+ u64 max_hole_len = 0;
+
+ lockdep_assert_held(&device->fs_info->chunk_mutex);
+
+ if (*len == 0)
+ return false;
+
+ end = *start + *len - 1;
+
+ /*
+ * Loop until we either see a large enough hole or check every pending
+ * extent overlapping the candidate hole.
+ * At every hole that we observe, record it if it is the new max.
+ * At the end of the iteration, set the output variables to the max hole.
+ */
+ while (true) {
+ if (first_pending_extent(device, *start, *len, &pending_start, &pending_end)) {
+ /*
+ * Case 1: the pending extent overlaps the start of
+ * candidate hole. That means the true hole is after the
+ * pending extent, but we need to find the next pending
+ * extent to properly size the hole. In the next loop,
+ * we will reduce to case 2 or 3.
+ * e.g.,
+ *
+ * |----pending A----| real hole |----pending B----|
+ * | candidate hole |
+ * *start end
+ */
+ if (pending_start <= *start) {
+ *start = pending_end + 1;
+ goto next;
+ }
+ /*
+ * Case 2: The pending extent starts after *start (and overlaps
+ * [*start, end), so the first hole just goes up to the start
+ * of the pending extent.
+ * e.g.,
+ *
+ * | real hole |----pending A----|
+ * | candidate hole |
+ * *start end
+ */
+ *len = pending_start - *start;
+ if (*len > max_hole_len) {
+ max_hole_start = *start;
+ max_hole_len = *len;
+ }
+ if (*len >= min_hole_size)
+ break;
+ /*
+ * If the hole wasn't big enough, then we advance past
+ * the pending extent and keep looking.
+ */
+ *start = pending_end + 1;
+ goto next;
+ } else {
+ /*
+ * Case 3: There is no pending extent overlapping the
+ * range [*start, *start + *len - 1], so the only remaining
+ * hole is the remaining range.
+ * e.g.,
+ *
+ * | candidate hole |
+ * | real hole |
+ * *start end
+ */
+
+ if (*len > max_hole_len) {
+ max_hole_start = *start;
+ max_hole_len = *len;
+ }
+ break;
+ }
+next:
+ if (*start > end)
+ break;
+ *len = end - *start + 1;
+ }
+ if (max_hole_len) {
+ *start = max_hole_start;
+ *len = max_hole_len;
+ } else {
+ *start = end + 1;
+ *len = 0;
+ }
+ return max_hole_len >= min_hole_size;
+}
+
static u64 dev_extent_search_start(struct btrfs_device *device)
{
switch (device->fs_devices->chunk_alloc_policy) {
@@ -1597,59 +1725,57 @@ static bool dev_extent_hole_check_zoned(struct btrfs_device *device,
}
/*
- * Check if specified hole is suitable for allocation.
+ * Validate and adjust a hole for chunk allocation
+ *
+ * @device: the device containing the candidate hole
+ * @hole_start: input/output pointer for the hole start position
+ * @hole_size: input/output pointer for the hole size
+ * @num_bytes: minimum allocation size required
*
- * @device: the device which we have the hole
- * @hole_start: starting position of the hole
- * @hole_size: the size of the hole
- * @num_bytes: the size of the free space that we need
+ * Check if the specified hole is suitable for allocation and adjust it if
+ * necessary. The hole may be modified to skip over pending chunk allocations
+ * and to satisfy stricter zoned requirements on zoned filesystems.
*
- * This function may modify @hole_start and @hole_size to reflect the suitable
- * position for allocation. Returns 1 if hole position is updated, 0 otherwise.
+ * For regular (non-zoned) allocation, if the hole after adjustment is smaller
+ * than @num_bytes, the search continues past additional pending extents until
+ * either a sufficiently large hole is found or no more pending extents exist.
+ *
+ * Return: true if a suitable hole was found and false otherwise.
+ * If the return value is true, then *hole_start and *hole_size are set to
+ * represent the hole we found.
+ * If the return value is false, then *hole_start is set to the largest
+ * hole we found and *hole_size is set to its length.
+ * If there are no holes at all, then *hole_start is set to the end of the range
+ * and *hole_size is set to 0.
*/
static bool dev_extent_hole_check(struct btrfs_device *device, u64 *hole_start,
u64 *hole_size, u64 num_bytes)
{
- bool changed = false;
- u64 hole_end = *hole_start + *hole_size;
+ bool found = false;
+ const u64 hole_end = *hole_start + *hole_size - 1;
- for (;;) {
- /*
- * Check before we set max_hole_start, otherwise we could end up
- * sending back this offset anyway.
- */
- if (contains_pending_extent(device, hole_start, *hole_size)) {
- if (hole_end >= *hole_start)
- *hole_size = hole_end - *hole_start;
- else
- *hole_size = 0;
- changed = true;
- }
+ ASSERT(*hole_size > 0);
- switch (device->fs_devices->chunk_alloc_policy) {
- default:
- btrfs_warn_unknown_chunk_allocation(device->fs_devices->chunk_alloc_policy);
- fallthrough;
- case BTRFS_CHUNK_ALLOC_REGULAR:
- /* No extra check */
- break;
- case BTRFS_CHUNK_ALLOC_ZONED:
- if (dev_extent_hole_check_zoned(device, hole_start,
- hole_size, num_bytes)) {
- changed = true;
- /*
- * The changed hole can contain pending extent.
- * Loop again to check that.
- */
- continue;
- }
- break;
- }
+again:
+ *hole_size = hole_end - *hole_start + 1;
+ found = find_hole_in_pending_extents(device, hole_start, hole_size, num_bytes);
+ if (!found)
+ return found;
+ ASSERT(*hole_size >= num_bytes);
+ switch (device->fs_devices->chunk_alloc_policy) {
+ default:
+ btrfs_warn_unknown_chunk_allocation(device->fs_devices->chunk_alloc_policy);
+ fallthrough;
+ case BTRFS_CHUNK_ALLOC_REGULAR:
+ return found;
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ if (dev_extent_hole_check_zoned(device, hole_start, hole_size, num_bytes))
+ goto again;
break;
}
- return changed;
+ return found;
}
/*
@@ -1708,7 +1834,7 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
ret = -ENOMEM;
goto out;
}
-again:
+
if (search_start >= search_end ||
test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
ret = -ENOSPC;
@@ -1795,11 +1921,7 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
*/
if (search_end > search_start) {
hole_size = search_end - search_start;
- if (dev_extent_hole_check(device, &search_start, &hole_size,
- num_bytes)) {
- btrfs_release_path(path);
- goto again;
- }
+ dev_extent_hole_check(device, &search_start, &hole_size, num_bytes);
if (hole_size > max_hole_size) {
max_hole_start = search_start;
@@ -5022,6 +5144,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
u64 diff;
u64 start;
u64 free_diff = 0;
+ u64 pending_start, pending_end;
new_size = round_down(new_size, fs_info->sectorsize);
start = new_size;
@@ -5067,7 +5190,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
* in-memory chunks are synced to disk so that the loop below sees them
* and relocates them accordingly.
*/
- if (contains_pending_extent(device, &start, diff)) {
+ if (first_pending_extent(device, start, diff, &pending_start, &pending_end)) {
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_commit_transaction(trans);
if (ret)
]
|
| 60/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 72657
output: 26
thoughts: 54
Thoughts: Okay, let's see, my initial thought is:
**Examining the Scope of `find_free_dev_extent`**
Right, so `find_free_dev_extent` is searching the commit root because it was introduced in the commit `1cd6121f2a38` or earlier. That's a key detail to understand its behavior and potential implications. Now, I need to understand the specifics of that commit. Let me pull up `1cd6121f2a38` and take a look.
|
| 61/2 |
2026/08/08 03:33 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 1cd6121f2a382a840f01f506694b54bf403fddc9
Author: Naohiro Aota <naohiro.aota@wdc.com>
Date: Thu Feb 4 19:21:48 2021 +0900
btrfs: zoned: implement zoned chunk allocator
Implement a zoned chunk and device extent allocator. One device zone
becomes a device extent so that a zone reset affects only this device
extent and does not change the state of blocks in the neighbor device
extents.
To implement the allocator, we need to extend the following functions for
a zoned filesystem.
- init_alloc_chunk_ctl
- dev_extent_search_start
- dev_extent_hole_check
- decide_stripe_size
init_alloc_chunk_ctl_zoned() is mostly the same as regular one. It always
set the stripe_size to the zone size and aligns the parameters to the zone
size.
dev_extent_search_start() only aligns the start offset to zone boundaries.
We don't care about the first 1MB like in regular filesystem because we
anyway reserve the first two zones for superblock logging.
dev_extent_hole_check_zoned() checks if zones in given hole are either
conventional or empty sequential zones. Also, it skips zones reserved for
superblock logging.
With the change to the hole, the new hole may now contain pending extents.
So, in this case, loop again to check that.
Finally, decide_stripe_size_zoned() should shrink the number of devices
instead of stripe size because we need to honor stripe_size == zone_size.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 07cd4742c123..ae2aeadad5a0 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1414,11 +1414,62 @@ static u64 dev_extent_search_start(struct btrfs_device *device, u64 start)
* make sure to start at an offset of at least 1MB.
*/
return max_t(u64, start, SZ_1M);
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ /*
+ * We don't care about the starting region like regular
+ * allocator, because we anyway use/reserve the first two zones
+ * for superblock logging.
+ */
+ return ALIGN(start, device->zone_info->zone_size);
default:
BUG();
}
}
+static bool dev_extent_hole_check_zoned(struct btrfs_device *device,
+ u64 *hole_start, u64 *hole_size,
+ u64 num_bytes)
+{
+ u64 zone_size = device->zone_info->zone_size;
+ u64 pos;
+ int ret;
+ bool changed = false;
+
+ ASSERT(IS_ALIGNED(*hole_start, zone_size));
+
+ while (*hole_size > 0) {
+ pos = btrfs_find_allocatable_zones(device, *hole_start,
+ *hole_start + *hole_size,
+ num_bytes);
+ if (pos != *hole_start) {
+ *hole_size = *hole_start + *hole_size - pos;
+ *hole_start = pos;
+ changed = true;
+ if (*hole_size < num_bytes)
+ break;
+ }
+
+ ret = btrfs_ensure_empty_zones(device, pos, num_bytes);
+
+ /* Range is ensured to be empty */
+ if (!ret)
+ return changed;
+
+ /* Given hole range was invalid (outside of device) */
+ if (ret == -ERANGE) {
+ *hole_start += *hole_size;
+ *hole_size = 0;
+ return 1;
+ }
+
+ *hole_start += zone_size;
+ *hole_size -= zone_size;
+ changed = true;
+ }
+
+ return changed;
+}
+
/**
* dev_extent_hole_check - check if specified hole is suitable for allocation
* @device: the device which we have the hole
@@ -1426,7 +1477,7 @@ static u64 dev_extent_search_start(struct btrfs_device *device, u64 start)
* @hole_size: the size of the hole
* @num_bytes: the size of the free space that we need
*
- * This function may modify @hole_start and @hole_end to reflect the suitable
+ * This function may modify @hole_start and @hole_size to reflect the suitable
* position for allocation. Returns 1 if hole position is updated, 0 otherwise.
*/
static bool dev_extent_hole_check(struct btrfs_device *device, u64 *hole_start,
@@ -1435,24 +1486,39 @@ static bool dev_extent_hole_check(struct btrfs_device *device, u64 *hole_start,
bool changed = false;
u64 hole_end = *hole_start + *hole_size;
- /*
- * Check before we set max_hole_start, otherwise we could end up
- * sending back this offset anyway.
- */
- if (contains_pending_extent(device, hole_start, *hole_size)) {
- if (hole_end >= *hole_start)
- *hole_size = hole_end - *hole_start;
- else
- *hole_size = 0;
- changed = true;
- }
+ for (;;) {
+ /*
+ * Check before we set max_hole_start, otherwise we could end up
+ * sending back this offset anyway.
+ */
+ if (contains_pending_extent(device, hole_start, *hole_size)) {
+ if (hole_end >= *hole_start)
+ *hole_size = hole_end - *hole_start;
+ else
+ *hole_size = 0;
+ changed = true;
+ }
+
+ switch (device->fs_devices->chunk_alloc_policy) {
+ case BTRFS_CHUNK_ALLOC_REGULAR:
+ /* No extra check */
+ break;
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ if (dev_extent_hole_check_zoned(device, hole_start,
+ hole_size, num_bytes)) {
+ changed = true;
+ /*
+ * The changed hole can contain pending extent.
+ * Loop again to check that.
+ */
+ continue;
+ }
+ break;
+ default:
+ BUG();
+ }
- switch (device->fs_devices->chunk_alloc_policy) {
- case BTRFS_CHUNK_ALLOC_REGULAR:
- /* No extra check */
break;
- default:
- BUG();
}
return changed;
@@ -1505,6 +1571,9 @@ static int find_free_dev_extent_start(struct btrfs_device *device,
search_start = dev_extent_search_start(device, search_start);
+ WARN_ON(device->zone_info &&
+ !IS_ALIGNED(num_bytes, device->zone_info->zone_size));
+
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
@@ -4899,6 +4968,37 @@ static void init_alloc_chunk_ctl_policy_regular(
ctl->dev_extent_min = BTRFS_STRIPE_LEN * ctl->dev_stripes;
}
+static void init_alloc_chunk_ctl_policy_zoned(
+ struct btrfs_fs_devices *fs_devices,
+ struct alloc_chunk_ctl *ctl)
+{
+ u64 zone_size = fs_devices->fs_info->zone_size;
+ u64 limit;
+ int min_num_stripes = ctl->devs_min * ctl->dev_stripes;
+ int min_data_stripes = (min_num_stripes - ctl->nparity) / ctl->ncopies;
+ u64 min_chunk_size = min_data_stripes * zone_size;
+ u64 type = ctl->type;
+
+ ctl->max_stripe_size = zone_size;
+ if (type & BTRFS_BLOCK_GROUP_DATA) {
+ ctl->max_chunk_size = round_down(BTRFS_MAX_DATA_CHUNK_SIZE,
+ zone_size);
+ } else if (type & BTRFS_BLOCK_GROUP_METADATA) {
+ ctl->max_chunk_size = ctl->max_stripe_size;
+ } else if (type & BTRFS_BLOCK_GROUP_SYSTEM) {
+ ctl->max_chunk_size = 2 * ctl->max_stripe_size;
+ ctl->devs_max = min_t(int, ctl->devs_max,
+ BTRFS_MAX_DEVS_SYS_CHUNK);
+ }
+
+ /* We don't want a chunk larger than 10% of writable space */
+ limit = max(round_down(div_factor(fs_devices->total_rw_bytes, 1),
+ zone_size),
+ min_chunk_size);
+ ctl->max_chunk_size = min(limit, ctl->max_chunk_size);
+ ctl->dev_extent_min = zone_size * ctl->dev_stripes;
+}
+
static void init_alloc_chunk_ctl(struct btrfs_fs_devices *fs_devices,
struct alloc_chunk_ctl *ctl)
{
@@ -4919,6 +5019,9 @@ static void init_alloc_chunk_ctl(struct btrfs_fs_devices *fs_devices,
case BTRFS_CHUNK_ALLOC_REGULAR:
init_alloc_chunk_ctl_policy_regular(fs_devices, ctl);
break;
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ init_alloc_chunk_ctl_policy_zoned(fs_devices, ctl);
+ break;
default:
BUG();
}
@@ -5045,6 +5148,38 @@ static int decide_stripe_size_regular(struct alloc_chunk_ctl *ctl,
return 0;
}
+static int decide_stripe_size_zoned(struct alloc_chunk_ctl *ctl,
+ struct btrfs_device_info *devices_info)
+{
+ u64 zone_size = devices_info[0].dev->zone_info->zone_size;
+ /* Number of stripes that count for block group size */
+ int data_stripes;
+
+ /*
+ * It should hold because:
+ * dev_extent_min == dev_extent_want == zone_size * dev_stripes
+ */
+ ASSERT(devices_info[ctl->ndevs - 1].max_avail == ctl->dev_extent_min);
+
+ ctl->stripe_size = zone_size;
+ ctl->num_stripes = ctl->ndevs * ctl->dev_stripes;
+ data_stripes = (ctl->num_stripes - ctl->nparity) / ctl->ncopies;
+
+ /* stripe_size is fixed in zoned filesysmte. Reduce ndevs instead. */
+ if (ctl->stripe_size * data_stripes > ctl->max_chunk_size) {
+ ctl->ndevs = div_u64(div_u64(ctl->max_chunk_size * ctl->ncopies,
+ ctl->stripe_size) + ctl->nparity,
+ ctl->dev_stripes);
+ ctl->num_stripes = ctl->ndevs * ctl->dev_stripes;
+ data_stripes = (ctl->num_stripes - ctl->nparity) / ctl->ncopies;
+ ASSERT(ctl->stripe_size * data_stripes <= ctl->max_chunk_size);
+ }
+
+ ctl->chunk_size = ctl->stripe_size * data_stripes;
+
+ return 0;
+}
+
static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
@@ -5072,6 +5207,8 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
switch (fs_devices->chunk_alloc_policy) {
case BTRFS_CHUNK_ALLOC_REGULAR:
return decide_stripe_size_regular(ctl, devices_info);
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ return decide_stripe_size_zoned(ctl, devices_info);
default:
BUG();
}
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 04e2b26823c2..598ac225176d 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -214,6 +214,7 @@ BTRFS_DEVICE_GETSET_FUNCS(bytes_used);
enum btrfs_chunk_allocation_policy {
BTRFS_CHUNK_ALLOC_REGULAR,
+ BTRFS_CHUNK_ALLOC_ZONED,
};
/*
diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c
index 6699f626a86e..69fd0d078b9b 100644
--- a/fs/btrfs/zoned.c
+++ b/fs/btrfs/zoned.c
@@ -1,11 +1,13 @@
// SPDX-License-Identifier: GPL-2.0
+#include <linux/bitops.h>
#include <linux/slab.h>
#include <linux/blkdev.h>
#include "ctree.h"
#include "volumes.h"
#include "zoned.h"
#include "rcu-string.h"
+#include "disk-io.h"
/* Maximum number of zones to report per blkdev_report_zones() call */
#define BTRFS_REPORT_NR_ZONES 4096
@@ -559,6 +561,7 @@ int btrfs_check_zoned_mode(struct btrfs_fs_info *fs_info)
fs_info->zone_size = zone_size;
fs_info->max_zone_append_size = max_zone_append_size;
+ fs_info->fs_devices->chunk_alloc_policy = BTRFS_CHUNK_ALLOC_ZONED;
/*
* Check mount options here, because we might change fs_info->zoned
@@ -779,3 +782,141 @@ int btrfs_reset_sb_log_zones(struct block_device *bdev, int mirror)
sb_zone << zone_sectors_shift,
zone_sectors * BTRFS_NR_SB_LOG_ZONES, GFP_NOFS);
}
+
+/**
+ * btrfs_find_allocatable_zones - find allocatable zones within a given region
+ *
+ * @device: the device to allocate a region on
+ * @hole_start: the position of the hole to allocate the region
+ * @num_bytes: size of wanted region
+ * @hole_end: the end of the hole
+ * @return: position of allocatable zones
+ *
+ * Allocatable region should not contain any superblock locations.
+ */
+u64 btrfs_find_allocatable_zones(struct btrfs_device *device, u64 hole_start,
+ u64 hole_end, u64 num_bytes)
+{
+ struct btrfs_zoned_device_info *zinfo = device->zone_info;
+ const u8 shift = zinfo->zone_size_shift;
+ u64 nzones = num_bytes >> shift;
+ u64 pos = hole_start;
+ u64 begin, end;
+ bool have_sb;
+ int i;
+
+ ASSERT(IS_ALIGNED(hole_start, zinfo->zone_size));
+ ASSERT(IS_ALIGNED(num_bytes, zinfo->zone_size));
+
+ while (pos < hole_end) {
+ begin = pos >> shift;
+ end = begin + nzones;
+
+ if (end > zinfo->nr_zones)
+ return hole_end;
+
+ /* Check if zones in the region are all empty */
+ if (btrfs_dev_is_sequential(device, pos) &&
+ find_next_zero_bit(zinfo->empty_zones, end, begin) != end) {
+ pos += zinfo->zone_size;
+ continue;
+ }
+
+ have_sb = false;
+ for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
+ u32 sb_zone;
+ u64 sb_pos;
+
+ sb_zone = sb_zone_number(shift, i);
+ if (!(end <= sb_zone ||
+ sb_zone + BTRFS_NR_SB_LOG_ZONES <= begin)) {
+ have_sb = true;
+ pos = ((u64)sb_zone + BTRFS_NR_SB_LOG_ZONES) << shift;
+ break;
+ }
+
+ /* We also need to exclude regular superblock positions */
+ sb_pos = btrfs_sb_offset(i);
+ if (!(pos + num_bytes <= sb_pos ||
+ sb_pos + BTRFS_SUPER_INFO_SIZE <= pos)) {
+ have_sb = true;
+ pos = ALIGN(sb_pos + BTRFS_SUPER_INFO_SIZE,
+ zinfo->zone_size);
+ break;
+ }
+ }
+ if (!have_sb)
+ break;
+ }
+
+ return pos;
+}
+
+int btrfs_reset_device_zone(struct btrfs_device *device, u64 physical,
+ u64 length, u64 *bytes)
+{
+ int ret;
+
+ *bytes = 0;
+ ret = blkdev_zone_mgmt(device->bdev, REQ_OP_ZONE_RESET,
+ physical >> SECTOR_SHIFT, length >> SECTOR_SHIFT,
+ GFP_NOFS);
+ if (ret)
+ return ret;
+
+ *bytes = length;
+ while (length) {
+ btrfs_dev_set_zone_empty(device, physical);
+ physical += device->zone_info->zone_size;
+ length -= device->zone_info->zone_size;
+ }
+
+ return 0;
+}
+
+int btrfs_ensure_empty_zones(struct btrfs_device *device, u64 start, u64 size)
+{
+ struct btrfs_zoned_device_info *zinfo = device->zone_info;
+ const u8 shift = zinfo->zone_size_shift;
+ unsigned long begin = start >> shift;
+ unsigned long end = (start + size) >> shift;
+ u64 pos;
+ int ret;
+
+ ASSERT(IS_ALIGNED(start, zinfo->zone_size));
+ ASSERT(IS_ALIGNED(size, zinfo->zone_size));
+
+ if (end > zinfo->nr_zones)
+ return -ERANGE;
+
+ /* All the zones are conventional */
+ if (find_next_bit(zinfo->seq_zones, begin, end) == end)
+ return 0;
+
+ /* All the zones are sequential and empty */
+ if (find_next_zero_bit(zinfo->seq_zones, begin, end) == end &&
+ find_next_zero_bit(zinfo->empty_zones, begin, end) == end)
+ return 0;
+
+ for (pos = start; pos < start + size; pos += zinfo->zone_size) {
+ u64 reset_bytes;
+
+ if (!btrfs_dev_is_sequential(device, pos) ||
+ btrfs_dev_is_empty_zone(device, pos))
+ continue;
+
+ /* Free regions should be empty */
+ btrfs_warn_in_rcu(
+ device->fs_info,
+ "zoned: resetting device %s (devid %llu) zone %llu for allocation",
+ rcu_str_deref(device->name), device->devid, pos >> shift);
+ WARN_ON_ONCE(1);
+
+ ret = btrfs_reset_device_zone(device, pos, zinfo->zone_size,
+ &reset_bytes);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
diff --git a/fs/btrfs/zoned.h b/fs/btrfs/zoned.h
index 5e78786bb723..6c8f83c48c2e 100644
--- a/fs/btrfs/zoned.h
+++ b/fs/btrfs/zoned.h
@@ -36,6 +36,11 @@ int btrfs_sb_log_location(struct btrfs_device *device, int mirror, int rw,
u64 *bytenr_ret);
void btrfs_advance_sb_log(struct btrfs_device *device, int mirror);
int btrfs_reset_sb_log_zones(struct block_device *bdev, int mirror);
+u64 btrfs_find_allocatable_zones(struct btrfs_device *device, u64 hole_start,
+ u64 hole_end, u64 num_bytes);
+int btrfs_reset_device_zone(struct btrfs_device *device, u64 physical,
+ u64 length, u64 *bytes);
+int btrfs_ensure_empty_zones(struct btrfs_device *device, u64 start, u64 size);
#else /* CONFIG_BLK_DEV_ZONED */
static inline int btrfs_get_dev_zone(struct btrfs_device *device, u64 pos,
struct blk_zone *zone)
@@ -91,6 +96,26 @@ static inline int btrfs_reset_sb_log_zones(struct block_device *bdev, int mirror
return 0;
}
+static inline u64 btrfs_find_allocatable_zones(struct btrfs_device *device,
+ u64 hole_start, u64 hole_end,
+ u64 num_bytes)
+{
+ return hole_start;
+}
+
+static inline int btrfs_reset_device_zone(struct btrfs_device *device,
+ u64 physical, u64 length, u64 *bytes)
+{
+ *bytes = 0;
+ return 0;
+}
+
+static inline int btrfs_ensure_empty_zones(struct btrfs_device *device,
+ u64 start, u64 size)
+{
+ return 0;
+}
+
#endif
static inline bool btrfs_dev_is_sequential(struct btrfs_device *device, u64 pos)
]
|
| 62/2 |
2026/08/08 03:33 |
llm |
debugger |
0m
Model:
Tokens:
input: 79154
output: 43
thoughts: 265
Thoughts: **Analyzing Chunk Allocation in Btrfs**
Okay, first things first, I need to remind myself that `search_commit_root` was introduced in commit `b14c5e04bd0f`. Let's pull up that commit message; it should clarify things. Ah, here it is: "The chunk allocation as defined in find_free_dev_extent() is a loop which searches the commit root of the dev_root and looks for gaps between DEV_EXTENT items. For those gaps, it then checks alloc_state bitmap for any pending extents and adjusts the hole that it finds accordingly."
Right, so `find_free_dev_extent` does indeed look at the commit root, *but* it also takes `device->alloc_state` into account for pending extents. That makes sense. Now, the key question arises: if it's looking at `device->alloc_state` for pending extents, is it also considering *freed* extents? Hmmm...
I think not! Looking closely at how `device->alloc_state` is used, I remember it only tracks `CHUNK_ALLOCATED` bits. So, when a chunk is allocated, `chunk_map_device_set_bits` would be setting the `CHUNK_ALLOCATED` bit in `device->alloc_state`. This means `alloc_state` is only concerned with active allocation and not previous allocations which are freed.
Then, the critical point is that when a chunk is *freed*, does the system clear that `CHUNK_ALLOCATED` bit? I need to check the code. Specifically, let's look at `btrfs_remove_dev_extents`.
|
| 63/2 |
2026/08/08 03:34 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 64/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 80692
output: 18
thoughts: 55
Thoughts: Here's my summary, as the expert:
**Concerning `CHUNK_ALLOCATED` in Btrfs Device Management**
Okay, hold on a second. I just realized something critical with the `btrfs_remove_dev_extents` function. It *doesn't* actually clear the `CHUNK_ALLOCATED` bit from `device->alloc_state`! That's... unexpected, and potentially problematic. This means we might have stale allocation state lingering around after a device removal, or maybe the code relies on something else to clear it. It's a subtle but significant issue.
The question now becomes: when *is* `CHUNK_ALLOCATED` cleared? It has to be cleared *somewhere*, otherwise our allocation tracking will become completely corrupted. I need to understand the lifecycle of this flag. My first instinct is to do a quick search through the codebase for other instances of `CHUNK_ALLOCATED`. I'll need to trace its manipulation to get the full picture. Let's see what the code tells us.
|
| 65/2 |
2026/08/08 03:34 |
tool |
grepper |
0m
Args: map[Expression:CHUNK_ALLOCATED]
Results: map[Output:fs/btrfs/dev-replace.c=786=static int btrfs_set_target_alloc_state(struct btrfs_device *srcdev,
--
fs/btrfs/dev-replace.c-798- &found_start, &found_end,
fs/btrfs/dev-replace.c:799: CHUNK_ALLOCATED, &cached_state)) {
fs/btrfs/dev-replace.c-800- ret = btrfs_set_extent_bit(&tgtdev->alloc_state, found_start,
fs/btrfs/dev-replace.c:801: found_end, CHUNK_ALLOCATED, NULL);
fs/btrfs/dev-replace.c-802- if (ret)
--
fs/btrfs/extent-io-tree.h=18=enum {
--
fs/btrfs/extent-io-tree.h-83- */
fs/btrfs/extent-io-tree.h:84:#define CHUNK_ALLOCATED EXTENT_DIRTY
fs/btrfs/extent-io-tree.h-85-#define CHUNK_TRIMMED EXTENT_DEFRAG
fs/btrfs/extent-io-tree.h:86:#define CHUNK_STATE_MASK (CHUNK_ALLOCATED | \
fs/btrfs/extent-io-tree.h-87- CHUNK_TRIMMED)
--
fs/btrfs/extent-tree.c=6618=static int btrfs_trim_free_extents_throttle(struct btrfs_device *device,
--
fs/btrfs/extent-tree.c-6658- &start, &end,
fs/btrfs/extent-tree.c:6659: CHUNK_TRIMMED | CHUNK_ALLOCATED);
fs/btrfs/extent-tree.c-6660- start = max(start, cur_start);
--
fs/btrfs/relocation.c=4780=int btrfs_last_identity_remap_gone(struct btrfs_chunk_map *chunk_map,
--
fs/btrfs/relocation.c-4826- write_lock(&trans->fs_info->mapping_tree_lock);
fs/btrfs/relocation.c:4827: btrfs_chunk_map_device_clear_bits(chunk_map, CHUNK_ALLOCATED);
fs/btrfs/relocation.c-4828- write_unlock(&trans->fs_info->mapping_tree_lock);
--
fs/btrfs/tests/chunk-allocation-tests.c=242=static int test_find_hole_in_pending(u32 sectorsize, u32 nodesize)
--
fs/btrfs/tests/chunk-allocation-tests.c-277- start, start + len - 1,
fs/btrfs/tests/chunk-allocation-tests.c:278: CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/chunk-allocation-tests.c-279- }
--
fs/btrfs/tests/chunk-allocation-tests.c-303- btrfs_clear_extent_bit(&device->alloc_state, 0, (u64)-1,
fs/btrfs/tests/chunk-allocation-tests.c:304: CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/chunk-allocation-tests.c-305- if (ret)
--
fs/btrfs/tests/chunk-allocation-tests.c=387=static int test_first_pending_extent(u32 sectorsize, u32 nodesize)
--
fs/btrfs/tests/chunk-allocation-tests.c-419- start, start + len - 1,
fs/btrfs/tests/chunk-allocation-tests.c:420: CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/chunk-allocation-tests.c-421- }
--
fs/btrfs/tests/chunk-allocation-tests.c-450- btrfs_clear_extent_bit(&device->alloc_state, 0, (u64)-1,
fs/btrfs/tests/chunk-allocation-tests.c:451: CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/chunk-allocation-tests.c-452- if (ret)
--
fs/btrfs/tests/extent-io-tests.c=558=static int test_find_first_clear_extent_bit(void)
--
fs/btrfs/tests/extent-io-tests.c-580- btrfs_set_extent_bit(&tree, SZ_1M, SZ_4M - 1,
fs/btrfs/tests/extent-io-tests.c:581: CHUNK_TRIMMED | CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/extent-io-tests.c-582-
fs/btrfs/tests/extent-io-tests.c-583- btrfs_find_first_clear_extent_bit(&tree, SZ_512K, &start, &end,
fs/btrfs/tests/extent-io-tests.c:584: CHUNK_TRIMMED | CHUNK_ALLOCATED);
fs/btrfs/tests/extent-io-tests.c-585-
--
fs/btrfs/tests/extent-io-tests.c-593- btrfs_set_extent_bit(&tree, SZ_32M, SZ_64M - 1,
fs/btrfs/tests/extent-io-tests.c:594: CHUNK_TRIMMED | CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/extent-io-tests.c-595-
--
fs/btrfs/tests/extent-io-tests.c-599- btrfs_find_first_clear_extent_bit(&tree, 12 * SZ_1M, &start, &end,
fs/btrfs/tests/extent-io-tests.c:600: CHUNK_TRIMMED | CHUNK_ALLOCATED);
fs/btrfs/tests/extent-io-tests.c-601-
--
fs/btrfs/tests/extent-io-tests.c-612- btrfs_find_first_clear_extent_bit(&tree, SZ_2M, &start, &end,
fs/btrfs/tests/extent-io-tests.c:613: CHUNK_TRIMMED | CHUNK_ALLOCATED);
fs/btrfs/tests/extent-io-tests.c-614-
--
fs/btrfs/tests/extent-io-tests.c-624- */
fs/btrfs/tests/extent-io-tests.c:625: btrfs_set_extent_bit(&tree, SZ_64M, SZ_64M + SZ_8M - 1, CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/extent-io-tests.c-626- btrfs_find_first_clear_extent_bit(&tree, SZ_64M + SZ_1M, &start, &end,
--
fs/btrfs/tests/extent-io-tests.c-663- dump_extent_io_tree(&tree);
fs/btrfs/tests/extent-io-tests.c:664: btrfs_clear_extent_bit(&tree, 0, (u64)-1, CHUNK_TRIMMED | CHUNK_ALLOCATED, NULL);
fs/btrfs/tests/extent-io-tests.c-665-
--
fs/btrfs/volumes.c=1535=bool btrfs_first_pending_extent(struct btrfs_device *device, u64 start, u64 len,
--
fs/btrfs/volumes.c-1541- pending_start, pending_end,
fs/btrfs/volumes.c:1542: CHUNK_ALLOCATED, NULL)) {
fs/btrfs/volumes.c-1543-
--
fs/btrfs/volumes.c=5990=void btrfs_remove_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_map *map)
--
fs/btrfs/volumes.c-5994- RB_CLEAR_NODE(&map->rb_node);
fs/btrfs/volumes.c:5995: btrfs_chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
fs/btrfs/volumes.c-5996- write_unlock(&fs_info->mapping_tree_lock);
--
fs/btrfs/volumes.c=6018=int btrfs_add_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_map *map)
--
fs/btrfs/volumes.c-6029- }
fs/btrfs/volumes.c:6030: chunk_map_device_set_bits(map, CHUNK_ALLOCATED);
fs/btrfs/volumes.c-6031- btrfs_chunk_map_device_clear_bits(map, CHUNK_TRIMMED);
--
fs/btrfs/volumes.c=6377=void btrfs_mapping_tree_free(struct btrfs_fs_info *fs_info)
--
fs/btrfs/volumes.c-6387- RB_CLEAR_NODE(&map->rb_node);
fs/btrfs/volumes.c:6388: btrfs_chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
fs/btrfs/volumes.c-6389- /* Once for the tree ref. */
]
|
| 66/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 83517
output: 55
thoughts: 43
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Debugging Chunk Allocation**
Right, so `CHUNK_ALLOCATED` is finally cleared within `btrfs_remove_chunk_map`. Excellent! That's the target. Now, I need to understand the control flow that leads to its invocation. I need to trace back from that point. Let's dig into the call sites of `btrfs_remove_chunk_map` and see what triggers its execution. This should reveal the specific scenarios where that flag, and the subsequent deallocation, is handled. I need to understand what this means for data integrity and recovery.
|
| 67/2 |
2026/08/08 03:34 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/volumes.c IncludeSnippetLines:0 Name:btrfs_remove_chunk_map SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_block_group SourceFile:fs/btrfs/block-group.c SourceLine:1373] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_unfreeze_block_group SourceFile:fs/btrfs/block-group.c SourceLine:4816] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:create_chunk SourceFile:fs/btrfs/volumes.c SourceLine:6096]] TruncatedOutput:false]
|
| 68/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 83942
output: 38
thoughts: 198
Thoughts: **My Analysis of Chunk Removal in Btrfs**
Okay, so I'm following the logic of the Btrfs chunk removal process. `btrfs_remove_chunk_map` is called by `btrfs_remove_block_group`, and I see that `btrfs_remove_block_group` is, in turn, called at the very end of `btrfs_remove_chunk`. This is the important part: it's *after* the chunk's device extents are removed, and *after* the chunk item is removed. Specifically:
```c
/* On error, btrfs_remove_block_group() aborts the transaction. */
ret = btrfs_remove_block_group(trans, map);
```
So, the `CHUNK_ALLOCATED` flag is cleared *after* those operations. This raises a crucial point in the timing. If `CHUNK_ALLOCATED` remains set *during* these earlier steps, `find_free_dev_extent` could misinterpret the space. My concern is that `first_pending_extent`, in particular, will see the space as pending (allocated) because it checks the `CHUNK_ALLOCATED` flag. I need to make sure this is accurate. I should go back and examine `first_pending_extent` in more detail.
|
| 69/2 |
2026/08/08 03:34 |
tool |
codesearch-definition-source |
0m
Error:
Args: map[ContextFile:fs/btrfs/volumes.c Name:first_pending_extent]
Results: map[SourceCode: SourceFile:]
|
| 70/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 84489
output: 19
thoughts: 39
Thoughts: Here's my summary, as the thought process unfolds:
**Digging into the Btrfs Pending Extent Logic**
Okay, so I'm trying to wrap my head around this Btrfs stuff...and I vaguely remember a term, like, `btrfs_first_pending_extent`...or something in that vein. Gotta be precise here, no room for ambiguity. Let's start with what *should* be the most obvious place: `first_pending_extent` within `volumes.c`. It's likely that's where I need to start to understand the logic.
|
| 71/2 |
2026/08/08 03:34 |
tool |
grepper |
0m
Args: map[Expression:first_pending_extent]
Results: map[Output:fs/btrfs/tests/chunk-allocation-tests.c=242=static int test_find_hole_in_pending(u32 sectorsize, u32 nodesize)
--
fs/btrfs/tests/chunk-allocation-tests.c-315- * Describes the inputs to the system and expected results
fs/btrfs/tests/chunk-allocation-tests.c:316: * when testing btrfs_first_pending_extent().
fs/btrfs/tests/chunk-allocation-tests.c-317- */
--
fs/btrfs/tests/chunk-allocation-tests.c=334=static const struct first_pending_test_case first_pending_tests[] = {
--
fs/btrfs/tests/chunk-allocation-tests.c-386-
fs/btrfs/tests/chunk-allocation-tests.c:387:static int test_first_pending_extent(u32 sectorsize, u32 nodesize)
fs/btrfs/tests/chunk-allocation-tests.c-388-{
--
fs/btrfs/tests/chunk-allocation-tests.c-392-
fs/btrfs/tests/chunk-allocation-tests.c:393: test_msg("running first_pending_extent tests");
fs/btrfs/tests/chunk-allocation-tests.c-394-
--
fs/btrfs/tests/chunk-allocation-tests.c-423- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/tests/chunk-allocation-tests.c:424: found = btrfs_first_pending_extent(device, test_case->hole_start,
fs/btrfs/tests/chunk-allocation-tests.c-425- test_case->hole_len,
--
fs/btrfs/tests/chunk-allocation-tests.c=461=int btrfs_test_chunk_allocation(u32 sectorsize, u32 nodesize)
--
fs/btrfs/tests/chunk-allocation-tests.c-466-
fs/btrfs/tests/chunk-allocation-tests.c:467: ret = test_first_pending_extent(sectorsize, nodesize);
fs/btrfs/tests/chunk-allocation-tests.c-468- if (ret)
--
fs/btrfs/volumes.c=1462=struct btrfs_device *btrfs_scan_one_device(const char *path,
--
fs/btrfs/volumes.c-1534- */
fs/btrfs/volumes.c:1535:bool btrfs_first_pending_extent(struct btrfs_device *device, u64 start, u64 len,
fs/btrfs/volumes.c-1536- u64 *pending_start, u64 *pending_end)
--
fs/btrfs/volumes.c=1575=bool btrfs_find_hole_in_pending_extents(struct btrfs_device *device, u64 *start,
--
fs/btrfs/volumes.c-1596- while (true) {
fs/btrfs/volumes.c:1597: if (btrfs_first_pending_extent(device, *start, *len, &pending_start, &pending_end)) {
fs/btrfs/volumes.c-1598- /*
--
fs/btrfs/volumes.c=5226=int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
--
fs/btrfs/volumes.c-5293- */
fs/btrfs/volumes.c:5294: if (btrfs_first_pending_extent(device, start, diff, &pending_start, &pending_end)) {
fs/btrfs/volumes.c-5295- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/volumes.h=930=void btrfs_chunk_map_device_clear_bits(struct btrfs_chunk_map *map, unsigned int bits);
fs/btrfs/volumes.h-931-
fs/btrfs/volumes.h:932:bool btrfs_first_pending_extent(struct btrfs_device *device, u64 start, u64 len,
fs/btrfs/volumes.h-933- u64 *pending_start, u64 *pending_end);
]
|
| 72/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 85825
output: 53
thoughts: 794
Thoughts: **Btrfs Chunk Removal: A Deep Dive into Allocation and Transactional Quirks**
Okay, so I'm digging into how chunk removal works in btrfs, specifically looking at how we can handle the edge case where the device is full during chunk removal. I see that the `btrfs_first_pending_extent` uses the `CHUNK_ALLOCATED` flag, and this flag gets cleared in `btrfs_remove_chunk_map`, which in turn is called from `btrfs_remove_block_group`, which is *also* called at the very end of `btrfs_remove_chunk`. So, the chunk we're trying to remove is still flagged as `CHUNK_ALLOCATED` in the device's `alloc_state` during the `btrfs_remove_chunk` process.
This means that `find_free_dev_extent` skips over this space! It sees it as a "pending extent" that's still "allocated", even though `btrfs_remove_dev_extents` has already deleted the corresponding device extent from the device tree.
The core issue seems to be the timing of clearing `CHUNK_ALLOCATED`. We allocate the chunk, set the flag, and remove the chunk, deleting the device extent. But `CHUNK_ALLOCATED` stays set until the block group removal. I'm wondering if clearing `CHUNK_ALLOCATED` earlier, maybe in `btrfs_remove_dev_extents`, would help. But wait... even if we clear it there, `find_free_dev_extent` searches the *commit* root. The commit root *still* has the device extent, right? And it would skip the space anyway!
Right, `find_free_dev_extent` skips either space in the commit root *or* space with `CHUNK_ALLOCATED`. Since the chunk we are removing was allocated in a previous transaction, it *is* in the commit root! Even if we clear the flag, the function skips the extent.
So, in the current transaction, the space freed by `btrfs_remove_chunk` is *simply* not available for allocation. This means that if the device is full, we *cannot* allocate a system chunk, even if we are removing a chunk to create space. Because of that, the chunk removal itself will also fail and that's not good.
To handle this gracefully, we need to check if we have enough system space *before* we even modify the device tree in `btrfs_remove_dev_extents`. How do we do that? We could call `check_system_chunk` before `btrfs_remove_dev_extents`. The problem is that `check_system_chunk` doesn't return an error. We need to either modify it to return an error, or create a function like `btrfs_check_system_chunk` that does.
But, `check_system_chunk` calls `reserve_chunk_space`, and `reserve_chunk_space` ignores errors from `btrfs_create_chunk`! If we change `reserve_chunk_space` to return those errors, how does that ripple through? I need to track down where `reserve_chunk_space` is called to figure out the implications.
|
| 73/2 |
2026/08/08 03:34 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:0 Name:reserve_chunk_space SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_reserve_chunk_metadata SourceFile:fs/btrfs/block-group.c SourceLine:4615] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:check_system_chunk SourceFile:fs/btrfs/block-group.c SourceLine:4584]] TruncatedOutput:false]
|
| 74/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 87509
output: 53
thoughts: 60
Thoughts: **Analyzing the Potential Impact of a Code Change**
Okay, so I'm looking at `reserve_chunk_space`. It's used by both `check_system_chunk` and `btrfs_reserve_chunk_metadata`. The idea is to make these functions return an error instead of handling the error internally. Now, if I change those error handling methods, I know that all the places that *call* those functions are going to need to be adjusted to properly handle the new error codes. Before I make any changes, I need to see exactly where `check_system_chunk` is called to understand the scope of this refactor. Looks like I need to trace the call graph. Let's see all the callers of `check_system_chunk`...
|
| 75/2 |
2026/08/08 03:34 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:0 Name:check_system_chunk SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_inc_block_group_ro SourceFile:fs/btrfs/block-group.c SourceLine:3234] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3494] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:do_chunk_alloc SourceFile:fs/btrfs/block-group.c SourceLine:4141]] TruncatedOutput:false]
|
| 76/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 87972
output: 57
thoughts: 60
Thoughts: Okay, here's my interpretation of this information, from the perspective of someone deeply familiar with Btrfs internals:
**Chunk Allocation and Metadata Reservation: My Key Considerations**
Alright, I'm looking at the core chunk allocation and metadata reservation logic in the Btrfs filesystem. It seems I've got a couple of key areas to focus on here, based on the identified callers of these specific functions.
First, `check_system_chunk` is being invoked from a few critical parts of the code. I see that `btrfs_inc_block_group_ro`, `btrfs_remove_chunk` and `do_chunk_alloc` are calling it. This suggests `check_system_chunk` plays a crucial role in validating or verifying the integrity of system chunks during read-only block group increments, chunk removal operations and most likely whenever chunks are allocated. I need to be doubly sure that `check_system_chunk` is robust and handles all the edge cases correctly since these functions are pretty fundamental to how Btrfs works. A bug here could lead to some nasty data corruption or unexpected behavior. This also means I need to revisit the associated error handling and recovery paths.
Then, there's `btrfs_reserve_chunk_metadata`. The listed callers are not listed in the document. This means, based on my understanding of Btrfs, that this is a key part of the metadata management. I'd be looking closely at how this function ensures sufficient space for metadata updates related to chunk allocation and modification. I'd pay close attention to the potential for race conditions or deadlocks within this function. I'll need to scrutinize how it handles concurrency and resource contention to prevent data inconsistencies when multiple processes are working with the filesystem simultaneously. The efficiency of this function is going to be important too - this can seriously affect the overall performance of the Btrfs filesystem.
|
| 77/2 |
2026/08/08 03:34 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:0 Name:btrfs_reserve_chunk_metadata SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_add_dev_item SourceFile:fs/btrfs/volumes.c SourceLine:2085] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_finish_sprout SourceFile:fs/btrfs/volumes.c SourceLine:2801] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_grow_device SourceFile:fs/btrfs/volumes.c SourceLine:3182] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_rm_dev_item SourceFile:fs/btrfs/volumes.c SourceLine:2151] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_shrink_device SourceFile:fs/btrfs/volumes.c SourceLine:5407] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:relocate_cowonly_block SourceFile:fs/btrfs/relocation.c SourceLine:2662] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:remove_chunk_stripes SourceFile:fs/btrfs/relocation.c SourceLine:4752]] TruncatedOutput:false]
|
| 78/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 88890
output: 46
thoughts: 1139
Thoughts: **Refining Chunk Removal: A Thought Experiment**
Okay, so I'm looking at this chunk removal process in Btrfs, and the current logic in `reserve_chunk_space` is giving me pause. The idea was to have it return an error to be handled directly by `btrfs_remove_chunk`. But, hold on... the comments in `reserve_chunk_space` explicitly state that it might fail harmlessly because we might not actually need the system chunk! This means that some chunk removals could succeed even if the space reservation fails because COW operations might not be needed. Failing early would prevent *successful* chunk removals, which is definitely not what we want.
The crucial question is, how do we know if we *will* need to COW nodes? We don't. That's the crux. If we can't reserve system chunk space, and we're unlucky enough to actually *need* to COW, we end up with an `-ENOSPC` in `remove_chunk_item`. Then, since we call `btrfs_remove_dev_extents` first, the transaction gets aborted! We're essentially failing when we could have potentially succeeded!
The key here is the order of operations. Why is `btrfs_remove_dev_extents` called *before* `remove_chunk_item`? If we flipped the order, then a failure in `remove_chunk_item` would become a simple return of `-ENOSPC` without the transaction getting aborted.
Let's break down the implications. If `remove_chunk_item` fails, it *shouldn't* have modified the chunk tree. If it tries to modify the chunk tree with `btrfs_del_item`, and it fails with `-ENOSPC`, it means that it couldn't allocate a block to COW a node. So, the chunk tree is, crucially, unchanged. The transaction would not be aborted.
The next question: Does the dependency between `remove_chunk_item` and `btrfs_remove_dev_extents` matter? In allocation, the device extent gets added *before* the chunk item. So let's review the code for allocating a chunk.
Here's a thought... Let me review the code for allocating a chunk. Aha! `btrfs_chunk_alloc_add_chunk_item` *also* adds the device extents. So it seems like there is no dependency. We can call them in either order. That solves it!
Therefore, we can reverse the order. We move `remove_chunk_item` before `btrfs_remove_dev_extents`. Then, when `remove_chunk_item` fails with `-ENOSPC`, the transaction isn't aborted, and we can just return `-ENOSPC`. This avoids aborting the transaction unnecessarily. I think this is the better approach!
|
| 79/2 |
2026/08/08 03:34 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_chunk_alloc_add_chunk_item]
Results: map[SourceCode:6170: /*
6171: * This function, btrfs_chunk_alloc_add_chunk_item(), typically belongs to the
6172: * phase 1 of chunk allocation. It belongs to phase 2 only when allocating system
6173: * chunks.
6174: *
6175: * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
6176: * phases.
6177: */
6178: int btrfs_chunk_alloc_add_chunk_item(struct btrfs_trans_handle *trans,
6179: struct btrfs_block_group *bg)
6180: {
6181: struct btrfs_fs_info *fs_info = trans->fs_info;
6182: struct btrfs_root *chunk_root = fs_info->chunk_root;
6183: struct btrfs_key key;
6184: struct btrfs_chunk *chunk;
6185: struct btrfs_stripe *stripe;
6186: struct btrfs_chunk_map *map;
6187: size_t item_size;
6188: int i;
6189: int ret;
6190:
6191: /*
6192: * We take the chunk_mutex for 2 reasons:
6193: *
6194: * 1) Updates and insertions in the chunk btree must be done while holding
6195: * the chunk_mutex, as well as updating the system chunk array in the
6196: * superblock. See the comment on top of btrfs_chunk_alloc() for the
6197: * details;
6198: *
6199: * 2) To prevent races with the final phase of a device replace operation
6200: * that replaces the device object associated with the map's stripes,
6201: * because the device object's id can change at any time during that
6202: * final phase of the device replace operation
6203: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
6204: * replaced device and then see it with an ID of BTRFS_DEV_REPLACE_DEVID,
6205: * which would cause a failure when updating the device item, which does
6206: * not exists, or persisting a stripe of the chunk item with such ID.
6207: * Here we can't use the device_list_mutex because our caller already
6208: * has locked the chunk_mutex, and the final phase of device replace
6209: * acquires both mutexes - first the device_list_mutex and then the
6210: * chunk_mutex. Using any of those two mutexes protects us from a
6211: * concurrent device replace.
6212: */
6213: lockdep_assert_held(&fs_info->chunk_mutex);
6214:
6215: map = btrfs_get_chunk_map(fs_info, bg->start, bg->length);
6216: if (IS_ERR(map)) {
6217: ret = PTR_ERR(map);
6218: btrfs_abort_transaction(trans, ret);
6219: return ret;
6220: }
6221:
6222: item_size = btrfs_chunk_item_size(map->num_stripes);
6223:
6224: chunk = kzalloc(item_size, GFP_NOFS);
6225: if (unlikely(!chunk)) {
6226: ret = -ENOMEM;
6227: btrfs_abort_transaction(trans, ret);
6228: goto out;
6229: }
6230:
6231: for (i = 0; i < map->num_stripes; i++) {
6232: struct btrfs_device *device = map->stripes[i].dev;
6233:
6234: ret = btrfs_update_device(trans, device);
6235: if (ret)
6236: goto out;
6237: }
6238:
6239: stripe = &chunk->stripe;
6240: for (i = 0; i < map->num_stripes; i++) {
6241: struct btrfs_device *device = map->stripes[i].dev;
6242: const u64 dev_offset = map->stripes[i].physical;
6243:
6244: btrfs_set_stack_stripe_devid(stripe, device->devid);
6245: btrfs_set_stack_stripe_offset(stripe, dev_offset);
6246: memcpy(stripe->dev_uuid, device->uuid, BTRFS_UUID_SIZE);
6247: stripe++;
6248: }
6249:
6250: btrfs_set_stack_chunk_length(chunk, bg->length);
6251: btrfs_set_stack_chunk_owner(chunk, BTRFS_EXTENT_TREE_OBJECTID);
6252: btrfs_set_stack_chunk_stripe_len(chunk, BTRFS_STRIPE_LEN);
6253: btrfs_set_stack_chunk_type(chunk, map->type);
6254: btrfs_set_stack_chunk_num_stripes(chunk, map->num_stripes);
6255: btrfs_set_stack_chunk_io_align(chunk, BTRFS_STRIPE_LEN);
6256: btrfs_set_stack_chunk_io_width(chunk, BTRFS_STRIPE_LEN);
6257: btrfs_set_stack_chunk_sector_size(chunk, fs_info->sectorsize);
6258: btrfs_set_stack_chunk_sub_stripes(chunk, map->sub_stripes);
6259:
6260: key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
6261: key.type = BTRFS_CHUNK_ITEM_KEY;
6262: key.offset = bg->start;
6263:
6264: ret = btrfs_insert_item(trans, chunk_root, &key, chunk, item_size);
6265: if (ret)
6266: goto out;
6267:
6268: set_bit(BLOCK_GROUP_FLAG_CHUNK_ITEM_INSERTED, &bg->runtime_flags);
6269:
6270: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
6271: ret = btrfs_add_system_chunk(fs_info, &key, chunk, item_size);
6272: if (ret)
6273: goto out;
6274: }
6275:
6276: out:
6277: kfree(chunk);
6278: btrfs_free_chunk_map(map);
6279: return ret;
6280: }
SourceFile:fs/btrfs/volumes.c]
|
| 80/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 92677
output: 24
thoughts: 65
Thoughts: **My Btrfs Chunk Allocation Conundrum**
Okay, hold on a second. I was just reviewing the `btrfs_chunk_alloc_add_chunk_item` function, and something's not quite clicking. It *doesn't* actually seem to be adding the device extents to the device tree. That's... unexpected. My understanding was that this was the point of the whole chunk allocation process. Where the heck are these extents being populated, then?
Alright, time to shift gears. I need to figure out where the `btrfs_add_dev_extent` function is called, or at least search for instances of `BTRFS_DEV_EXTENT_KEY` in the codebase. That key has to be used somewhere to insert those new extents, right? Time to start digging through the source... this is going to take a bit.
|
| 81/2 |
2026/08/08 03:34 |
tool |
grepper |
0m
Args: map[Expression:BTRFS_DEV_EXTENT_KEY]
Results: map[Output:fs/btrfs/block-group.c=2832=static int insert_dev_extent(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-2850- key.objectid = device->devid;
fs/btrfs/block-group.c:2851: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/block-group.c-2852- key.offset = start;
--
fs/btrfs/dev-replace.c=440=static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
--
fs/btrfs/dev-replace.c-497- key.objectid = src_dev->devid;
fs/btrfs/dev-replace.c:498: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/dev-replace.c-499- key.offset = 0;
--
fs/btrfs/dev-replace.c-506-
fs/btrfs/dev-replace.c:507: if (found_key.type != BTRFS_DEV_EXTENT_KEY)
fs/btrfs/dev-replace.c-508- break;
--
fs/btrfs/print-tree.c=375=static void key_type_string(const struct btrfs_key *key, char *buf, int buf_size)
--
fs/btrfs/print-tree.c-406- [BTRFS_DEV_ITEM_KEY] = "DEV_ITEM",
fs/btrfs/print-tree.c:407: [BTRFS_DEV_EXTENT_KEY] = "DEV_EXTENT",
fs/btrfs/print-tree.c-408- [BTRFS_TEMPORARY_ITEM_KEY] = "TEMPORARY_ITEM",
--
fs/btrfs/print-tree.c=432=void btrfs_print_leaf(const struct extent_buffer *l)
--
fs/btrfs/print-tree.c-533- break;
fs/btrfs/print-tree.c:534: case BTRFS_DEV_EXTENT_KEY:
fs/btrfs/print-tree.c-535- dev_extent = btrfs_item_ptr(l, i,
--
fs/btrfs/scrub.c=2646=int scrub_enumerate_chunks(struct scrub_ctx *sctx,
--
fs/btrfs/scrub.c-2671- key.objectid = scrub_dev->devid;
fs/btrfs/scrub.c:2672: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/scrub.c-2673- key.offset = 0ull;
--
fs/btrfs/scrub.c-2703-
fs/btrfs/scrub.c:2704: if (found_key.type != BTRFS_DEV_EXTENT_KEY)
fs/btrfs/scrub.c-2705- break;
--
fs/btrfs/tree-checker.c=2319=static enum btrfs_tree_block_status check_leaf_item(struct extent_buffer *leaf,
--
fs/btrfs/tree-checker.c-2354- break;
fs/btrfs/tree-checker.c:2355: case BTRFS_DEV_EXTENT_KEY:
fs/btrfs/tree-checker.c-2356- ret = check_dev_extent_item(leaf, key, slot, prev_key);
--
fs/btrfs/volumes.c=1814=static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
--
fs/btrfs/volumes.c-1854- key.objectid = device->devid;
fs/btrfs/volumes.c:1855: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/volumes.c-1856- key.offset = search_start;
--
fs/btrfs/volumes.c-1881-
fs/btrfs/volumes.c:1882: if (key.type != BTRFS_DEV_EXTENT_KEY)
fs/btrfs/volumes.c-1883- goto next;
--
fs/btrfs/volumes.c=1954=static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-1971- key.objectid = device->devid;
fs/btrfs/volumes.c:1972: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/volumes.c-1973- key.offset = start;
--
fs/btrfs/volumes.c-1977- ret = btrfs_previous_item(root, path, key.objectid,
fs/btrfs/volumes.c:1978: BTRFS_DEV_EXTENT_KEY);
fs/btrfs/volumes.c-1979- if (ret)
--
fs/btrfs/volumes.c=5226=int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
--
fs/btrfs/volumes.c-5305- key.objectid = device->devid;
fs/btrfs/volumes.c:5306: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/volumes.c-5307- key.offset = (u64)-1;
--
fs/btrfs/volumes.c=8666=int btrfs_verify_dev_extents(struct btrfs_fs_info *fs_info)
--
fs/btrfs/volumes.c-8688- key.objectid = 1;
fs/btrfs/volumes.c:8689: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/volumes.c-8690- key.offset = 0;
--
fs/btrfs/volumes.c-8718- btrfs_item_key_to_cpu(leaf, &key, slot);
fs/btrfs/volumes.c:8719: if (key.type != BTRFS_DEV_EXTENT_KEY)
fs/btrfs/volumes.c-8720- break;
--
fs/btrfs/zoned.c=296=static int calculate_emulated_zone_size(struct btrfs_fs_info *fs_info)
--
fs/btrfs/zoned.c-305- key.objectid = 1;
fs/btrfs/zoned.c:306: key.type = BTRFS_DEV_EXTENT_KEY;
fs/btrfs/zoned.c-307- key.offset = 0;
--
include/uapi/linux/btrfs_tree.h-281-
include/uapi/linux/btrfs_tree.h:282:#define BTRFS_DEV_EXTENT_KEY 204
include/uapi/linux/btrfs_tree.h-283-#define BTRFS_DEV_ITEM_KEY 216
]
|
| 82/2 |
2026/08/08 03:34 |
llm |
debugger |
0m
Model:
Tokens:
input: 94901
output: 53
thoughts: 40
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Diving into btrfs's Block Group Allocation**
Right, `insert_dev_extent` in `fs/btrfs/block-group.c`... that's where things get interesting in terms of block group management, isn't it? My immediate thought is, let's trace this and see what's happening. I need to understand the call sites. I need to figure out the **context** in which `insert_dev_extent` is called. It’s critical to grasp how extent insertion is triggered to fully understand its impact on overall allocation and free space management within a block group. This isn't just about the function itself; it's about the bigger picture of how btrfs interacts with physical storage. I need to see if it's being called during allocation or perhaps during defragmentation or something else entirely. So, I need to know the calling functions. Time to crack open the call graph or just go grep'ing the source for its occurrences, with the goal of answering the 'when' and 'why' of this insertion.
|
| 83/2 |
2026/08/08 03:35 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:0 Name:insert_dev_extent SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:insert_dev_extents SourceFile:fs/btrfs/block-group.c SourceLine:2902]] TruncatedOutput:false]
|
| 84/2 |
2026/08/08 03:35 |
llm |
debugger |
0m
Model:
Tokens:
input: 95316
output: 48
thoughts: 1500
Thoughts: **Analyzing Chunk Removal and Transactional Behavior in Btrfs**
Okay, so `insert_dev_extents` is called from `btrfs_create_pending_block_groups`. This means the device extents are *not* added during `btrfs_chunk_alloc_add_chunk_item`. They're added later, during `btrfs_create_pending_block_groups`, which happens during transaction commit. This means `btrfs_chunk_alloc` is just reserving space in memory, setting `CHUNK_ALLOCATED` on the device's `alloc_state`.
Now, when we remove a chunk, we *immediately* delete the device extents from the device tree via `btrfs_remove_dev_extents`. Why do we do this immediately instead of during transaction commit like the chunk allocation? Initially, I thought it was for immediate reuse of the space. But `find_free_dev_extent` searches the commit root and skips `CHUNK_ALLOCATED`! So that doesn't work. The extents are still visible in the commit root if we delete them immediately. They're still skipped, so the space isn't available!
The current approach presents a challenge: if `remove_chunk_item` fails with `-ENOSPC`, we abort the transaction. If we swap the order and call `remove_chunk_item` *before* `btrfs_remove_dev_extents`, like so:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
if (ret == -ENOSPC) {
...
}
if (ret) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
...
```
If `remove_chunk_item` fails with `-ENOSPC`, we can just return `-ENOSPC` without modifying the device tree.
The `remove_chunk_item` modifies the chunk tree. If it fails with `-ENOSPC`, it means it couldn't allocate a metadata block to COW a node. But `btrfs_del_item` returns `-ENOSPC` *before* it modifies the tree if it can't allocate a block. So, this order change allows us to fail gracefully without corrupting the chunk tree!
Now, is there a reason `btrfs_remove_dev_extents` MUST be called first? Looking at the comment in `btrfs_remove_dev_extents`, it says we delete the extent items first. Why? Maybe to handle a failure? If `btrfs_remove_dev_extents` fails, we abort the transaction. But so does failure in the current setup. So, if we call it first and it fails, it also aborts. If `remove_chunk_item` fails, we don't abort, and can return `-ENOSPC`. If `remove_chunk_item` *succeeds* and `btrfs_remove_dev_extents` fails, we do abort. Is aborting after btrfs_remove_dev_extents better?
I considered that `btrfs_remove_dev_extents` might be less likely to fail with `-ENOSPC` because the device tree might be smaller. But, both use the same block reserve.
Wait, `check_system_chunk` reserves space for BOTH!
```c
/* num_devs device items to update and 1 chunk item to add or remove. */
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
```
`check_system_chunk` reserves space for updating *device* items in the chunk tree and the chunk item itself. It *doesn't* reserve space for `btrfs_remove_dev_extents`. `btrfs_remove_dev_extents` modifies the *device* tree, not the chunk tree. It uses the normal transaction's block reserve.
Wait, `btrfs_remove_chunk` is called from `btrfs_relocate_chunk_finish`, which starts a transaction with `
|
| 85/2 |
2026/08/08 03:35 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:btrfs_start_trans_remove_block_group]
Results: map[SourceCode:1383: struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
1384: struct btrfs_fs_info *fs_info, const u64 chunk_offset)
1385: {
1386: struct btrfs_root *root = btrfs_block_group_root(fs_info);
1387: struct btrfs_chunk_map *map;
1388: unsigned int num_items;
1389:
1390: if (unlikely(!root)) {
1391: btrfs_err(fs_info, "missing block group root");
1392: return ERR_PTR(-EUCLEAN);
1393: }
1394:
1395: map = btrfs_find_chunk_map(fs_info, chunk_offset, 1);
1396: ASSERT(map != NULL);
1397: ASSERT(map->start == chunk_offset);
1398:
1399: /*
1400: * We need to reserve 3 + N units from the metadata space info in order
1401: * to remove a block group (done at btrfs_remove_chunk() and at
1402: * btrfs_remove_block_group()), which are used for:
1403: *
1404: * 1 unit for adding the free space inode's orphan (located in the tree
1405: * of tree roots).
1406: * 1 unit for deleting the block group item (located in the extent
1407: * tree).
1408: * 1 unit for deleting the free space item (located in tree of tree
1409: * roots).
1410: * N units for deleting N device extent items corresponding to each
1411: * stripe (located in the device tree).
1412: *
1413: * In order to remove a block group we also need to reserve units in the
1414: * system space info in order to update the chunk tree (update one or
1415: * more device items and remove one chunk item), but this is done at
1416: * btrfs_remove_chunk() through a call to check_system_chunk().
1417: */
1418: num_items = 3 + map->num_stripes;
1419: btrfs_free_chunk_map(map);
1420:
1421: return btrfs_start_transaction_fallback_global_rsv(root, num_items);
1422: }
SourceFile:fs/btrfs/block-group.c]
|
| 86/2 |
2026/08/08 03:35 |
llm |
debugger |
0m
Model:
Tokens:
input: 98612
output: 43
thoughts: 990
Thoughts: **Refactoring Btrfs Chunk Removal for Robustness**
Okay, so I'm looking at the `btrfs_remove_chunk` function, specifically how it handles removing device extents and chunk items. The current order is a bit fragile, and I think I've got a better way.
Right now, the code first reserves space for `btrfs_remove_dev_extents`. This prevents `-ENOSPC` errors during the device extent removal, which is good. However, `remove_chunk_item` relies on the `chunk_block_rsv`, which is reserved by `check_system_chunk`, but `check_system_chunk` ignores errors if it can't allocate the system chunk, meaning `remove_chunk_item` *can* fail with `-ENOSPC`. If this happens, the transaction aborts, which is not ideal since the block group space has already been allocated.
The idea is to swap the order. Call `remove_chunk_item` first. This needs the `chunk_mutex`, and then call `btrfs_remove_dev_extents`. Now, if `remove_chunk_item` fails with `-ENOSPC`, we simply return the error, and the transaction is *not* aborted prematurely. We're also making sure that we don't hold the `chunk_mutex` while calling `btrfs_remove_dev_extents`.
I'm thinking, if `remove_chunk_item` *succeeds*, but `btrfs_remove_dev_extents` *fails* (e.g., `-ENOMEM`), it's okay to abort the transaction. `btrfs_remove_dev_extents` handles aborting the transaction if it fails.
But, is it safe to modify the chunk tree before device extents are removed? It seems fine. If the chunk item is removed, the chunk is gone from the chunk tree. If a crash happens in between (which is impossible as they are in the same transaction), it doesn't matter. Concurrent operations are safe because `remove_chunk_item` holds `chunk_mutex` and `btrfs_remove_dev_extents` holds `device_list_mutex`, so the mutual exclusion prevents issues.
So, here's what the code would look like:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
if (ret == -ENOSPC) {
/* try to allocate system chunk and retry */
...
}
if (ret) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
goto out;
}
trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto out_unlock;
}
}
btrfs_update_per_profile_avail(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out; /* wait, if it fails, we must abort because chunk item is removed! */
```
And as it turns out, `btrfs_remove_dev_extents` already aborts the transaction on failure. So, swapping the order is perfectly fine for error handling! I'll have to check the history, but I think this is a cleaner approach to ensure the transaction is handled safely in the face of resource allocation issues.
|
| 87/2 |
2026/08/08 03:35 |
tool |
git-log |
0m
Args: map[Count:10 SourcePath:fs/btrfs/volumes.c SymbolName:btrfs_remove_chunk]
Results: map[Output:c84053d9f7f7 btrfs: update per-profile available estimation
6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
979e1dc3d69e btrfs: handle deletions from remapped block group
a929904cf73b btrfs: add unlikely annotations to branches leading to transaction abort
4d5a047e0736 btrfs: add space_info parameter for block group creation
7dc66abb5a47 btrfs: use a dedicated data structure for chunk maps
f6f39f7a0add btrfs: rename btrfs_alloc_chunk to btrfs_create_chunk
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
64bc6c2a3453 btrfs: Remove superfluous check form btrfs_remove_chunk
f4208794d02e btrfs: Remove fs_info form btrfs_free_chunk
]
|
| 88/2 |
2026/08/08 03:35 |
llm |
debugger |
0m
Model:
Tokens:
input: 100815
output: 60
thoughts: 28
Thoughts: Okay, here's my summary, taking the perspective of someone deeply familiar with storage systems and btrfs:
**Checking the btrfs Codeflow**
Alright, let's dive into this. My immediate thought is to check a specific detail within the btrfs code. I need to quickly determine the exact ordering of operations. The key question is whether `btrfs_remove_dev_extents` *always* gets called before `remove_chunk_item`. This is crucial because the sequence of these function calls could indicate underlying logic or dependency concerns when dealing with device removal and chunk management. If there's a specific order, it likely means one function relies on the other to be properly setup, or properly cleaned. I need to ensure that the code is behaving in the way that I expect it to. The specific functions are the key here, and the context is very important.
|
| 89/2 |
2026/08/08 03:35 |
tool |
git-log |
0m
Args: map[CodeRegexp:btrfs_remove_dev_extents Count:10 SourcePath:fs/btrfs/volumes.c SymbolName:btrfs_remove_chunk]
Results: map[Output:c84053d9f7f7 btrfs: update per-profile available estimation
6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
979e1dc3d69e btrfs: handle deletions from remapped block group
a929904cf73b btrfs: add unlikely annotations to branches leading to transaction abort
4d5a047e0736 btrfs: add space_info parameter for block group creation
7dc66abb5a47 btrfs: use a dedicated data structure for chunk maps
f6f39f7a0add btrfs: rename btrfs_alloc_chunk to btrfs_create_chunk
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
64bc6c2a3453 btrfs: Remove superfluous check form btrfs_remove_chunk
f4208794d02e btrfs: Remove fs_info form btrfs_free_chunk
]
|
| 90/2 |
2026/08/08 03:35 |
llm |
debugger |
0m
Model:
Tokens:
input: 101367
output: 26
thoughts: 30
Thoughts: Okay, here's my take on those thoughts, framed as a first-person expert summary:
**Diving into `btrfs_remove_chunk` Implementation**
Alright, time to get my hands dirty. Let's dig into the implementation of `btrfs_remove_chunk` – specifically, I'm referencing commit `79bd37120b14`. This function is critical; it's the heart of how Btrfs handles chunk removal, which, depending on the kernel version and the specifics of the filesystem metadata and chunk allocation, can get pretty complex. I need to refresh my mental model of how data distribution is currently set up here. I want to check how the logic deals with all the edge cases: what if the chunk is partially full? What if it's the last chunk on a particular device? Are we dealing with RAID levels, and if so, how does this interact with rebalancing? I need to look closely at the error handling to ensure data integrity is maintained, no matter what. The allocation bits have to be updated correctly, obviously. Time to examine the code line by line and think through the implications of each section.
|
| 91/2 |
2026/08/08 03:35 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1281 lines.
[commit 79bd37120b149532af5b21953643ed74af69654f
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:06 2021 +0100
btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
Commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations") fixed a problem that resulted in
exhausting the system chunk array in the superblock when there are many
tasks allocating chunks in parallel. Basically too many tasks enter the
first phase of chunk allocation without previous tasks having finished
their second phase of allocation, resulting in too many system chunks
being allocated. That was originally observed when running the fallocate
tests of stress-ng on a PowerPC machine, using a node size of 64K.
However that commit also introduced a deadlock where a task in phase 1 of
the chunk allocation waited for another task that had allocated a system
chunk to finish its phase 2, but that other task was waiting on an extent
buffer lock held by the first task, therefore resulting in both tasks not
making any progress. That change was later reverted by a patch with the
subject "btrfs: fix deadlock with concurrent chunk allocations involving
system chunks", since there is no simple and short solution to address it
and the deadlock is relatively easy to trigger on zoned filesystems, while
the system chunk array exhaustion is not so common.
This change reworks the chunk allocation to avoid the system chunk array
exhaustion. It accomplishes that by making the first phase of chunk
allocation do the updates of the device items in the chunk btree and the
insertion of the new chunk item in the chunk btree. This is done while
under the protection of the chunk mutex (fs_info->chunk_mutex), in the
same critical section that checks for available system space, allocates
a new system chunk if needed and reserves system chunk space. This way
we do not have chunk space reserved until the second phase completes.
The same logic is applied to chunk removal as well, since it keeps
reserved system space long after it is done updating the chunk btree.
For direct allocation of system chunks, the previous behaviour remains,
because otherwise we would deadlock on extent buffers of the chunk btree.
Changes to the chunk btree are by large done by chunk allocation and chunk
removal, which first reserve chunk system space and then later do changes
to the chunk btree. The other remaining cases are uncommon and correspond
to adding a device, removing a device and resizing a device. All these
other cases do not pre-reserve system space, they modify the chunk btree
right away, so they don't hold reserved space for a long period like chunk
allocation and chunk removal do.
The diff of this change is huge, but more than half of it is just addition
of comments describing both how things work regarding chunk allocation and
removal, including both the new behavior and the parts of the old behavior
that did not change.
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a26209f98279..c557327b4545 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2207,6 +2207,13 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
return ret;
}
+/*
+ * This function, insert_block_group_item(), belongs to the phase 2 of chunk
+ * allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
static int insert_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_block_group *block_group)
{
@@ -2229,15 +2236,19 @@ static int insert_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_insert_item(trans, root, &key, &bgi, sizeof(bgi));
}
+/*
+ * This function, btrfs_create_pending_block_groups(), belongs to the phase 2 of
+ * chunk allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *block_group;
int ret = 0;
- if (!trans->can_flush_pending_bgs)
- return;
-
while (!list_empty(&trans->new_bgs)) {
int index;
@@ -2252,6 +2263,13 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
ret = insert_block_group_item(trans, block_group);
if (ret)
btrfs_abort_transaction(trans, ret);
+ if (!block_group->chunk_item_inserted) {
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, block_group);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret)
+ btrfs_abort_transaction(trans, ret);
+ }
ret = btrfs_finish_chunk_alloc(trans, block_group->start,
block_group->length);
if (ret)
@@ -2275,8 +2293,9 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
btrfs_trans_release_chunk_metadata(trans);
}
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size)
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *cache;
@@ -2286,7 +2305,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
cache = btrfs_create_block_group_cache(fs_info, chunk_offset);
if (!cache)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
cache->length = size;
set_free_space_tree_thresholds(cache);
@@ -2300,7 +2319,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
ret = btrfs_load_block_group_zone_info(cache, true);
if (ret) {
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
ret = exclude_super_stripes(cache);
@@ -2308,7 +2327,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
/* We may have excluded something, so call this just in case */
btrfs_free_excluded_extents(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
add_new_free_space(cache, chunk_offset, chunk_offset + size);
@@ -2335,7 +2354,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
if (ret) {
btrfs_remove_free_space_cache(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
/*
@@ -2354,7 +2373,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
btrfs_update_delayed_refs_rsv(trans);
set_avail_alloc_bits(fs_info, type);
- return 0;
+ return cache;
}
/*
@@ -3232,11 +3251,203 @@ int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type)
return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
+static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ /*
+ * Check if we have enough space in the system space info because we
+ * will need to update device items in the chunk btree and insert a new
+ * chunk item in the chunk btree as well. This will allocate a new
+ * system block group if needed.
+ */
+ check_system_chunk(trans, flags);
+
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ goto out;
+ }
+
+ /*
+ * If this is a system chunk allocation then stop right here and do not
+ * add the chunk item to the chunk btree. This is to prevent a deadlock
+ * because this system chunk allocation can be triggered while COWing
+ * some extent buffer of the chunk btree and while holding a lock on a
+ * parent extent buffer, in which case attempting to insert the chunk
+ * item (or update the device item) would result in a deadlock on that
+ * parent extent buffer. In this case defer the chunk btree updates to
+ * the second phase of chunk allocation and keep our reservation until
+ * the second phase completes.
+ *
+ * This is a rare case and can only be triggered by the very few cases
+ * we have where we need to touch the chunk btree outside chunk allocation
+ * and chunk removal. These cases are basically adding a device, removing
+ * a device or resizing a device.
+ */
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ return 0;
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ /*
+ * Normally we are not expected to fail with -ENOSPC here, since we have
+ * previously reserved space in the system space_info and allocated one
+ * new system chunk if necessary. However there are two exceptions:
+ *
+ * 1) We may have enough free space in the system space_info but all the
+ * existing system block groups have a profile which can not be used
+ * for extent allocation.
+ *
+ * This happens when mounting in degraded mode. For example we have a
+ * RAID1 filesystem with 2 devices, lose one device and mount the fs
+ * using the other device in degraded mode. If we then allocate a chunk,
+ * we may have enough free space in the existing system space_info, but
+ * none of the block groups can be used for extent allocation since they
+ * have a RAID1 profile, and because we are in degraded mode with a
+ * single device, we are forced to allocate a new system chunk with a
+ * SINGLE profile. Making check_system_chunk() iterate over all system
+ * block groups and check if they have a usable profile and enough space
+ * can be slow on very large filesystems, so we tolerate the -ENOSPC and
+ * try again after forcing allocation of a new system chunk. Like this
+ * we avoid paying the cost of that search in normal circumstances, when
+ * we were not mounted in degraded mode;
+ *
+ * 2) We had enough free space info the system space_info, and one suitable
+ * block group to allocate from when we called check_system_chunk()
+ * above. However right after we called it, the only system block group
+ * with enough free space got turned into RO mode by a running scrub,
+ * and in this case we have to allocate a new one and retry. We only
+ * need do this allocate and retry once, since we have a transaction
+ * handle and scrub uses the commit root to search for block groups.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+out:
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
+}
+
/*
- * If force is CHUNK_ALLOC_FORCE:
+ * Chunk allocation is done in 2 phases:
+ *
+ * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
+ * the chunk, the chunk mapping, create its block group and add the items
+ * that belong in the chunk btree to it - more specifically, we need to
+ * update device items in the chunk btree and add a new chunk item to it.
+ *
+ * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
+ * group item to the extent btree and the device extent items to the devices
+ * btree.
+ *
+ * This is done to prevent deadlocks. For example when COWing a node from the
+ * extent btree we are holding a write lock on the node's parent and if we
+ * trigger chunk allocation and attempted to insert the new block group item
+ * in the extent btree right way, we could deadlock because the path for the
+ * insertion can include that parent node. At first glance it seems impossible
+ * to trigger chunk allocation after starting a transaction since tasks should
+ * reserve enough transaction units (metadata space), however while that is true
+ * most of the time, chunk allocation may still be triggered for several reasons:
+ *
+ * 1) When reserving metadata, we check if there is enough free space in the
+ * metadata space_info and therefore don't trigger allocation of a new chunk.
+ * However later when the task actually tries to COW an extent buffer from
+ * the extent btree or from the device btree for example, it is forced to
+ * allocate a new block group (chunk) because the only one that had enough
+ * free space was just turned to RO mode by a running scrub for example (or
+ * device replace, block group reclaim thread, etc), so we can not use it
+ * for allocating an extent and end up being forced to allocate a new one;
+ *
+ * 2) Because we only check that the metadata space_info has enough free bytes,
+ * we end up not allocating a new metadata chunk in that case. However if
+ * the filesystem was mounted in degraded mode, none of the existing block
+ * groups might be suitable for extent allocation due to their incompatible
+ * profile (for e.g. mounting a 2 devices filesystem, where all block groups
+ * use a RAID1 profile, in degraded mode using a single device). In this case
+ * when the task attempts to COW some extent buffer of the extent btree for
+ * example, it will trigger allocation of a new metadata block group with a
+ * suitable profile (SINGLE profile in the example of the degraded mount of
+ * the RAID1 filesystem);
+ *
+ * 3) The task has reserved enough transaction units / metadata space, but when
+ * it attempts to COW an extent buffer from the extent or device btree for
+ * example, it does not find any free extent in any metadata block group,
+ * therefore forced to try to allocate a new metadata block group.
+ * This is because some other task allocated all available extents in the
+ * meanwhile - this typically happens with tasks that don't reserve space
+ * properly, either intentionally or as a bug. One example where this is
+ * done intentionally is fsync, as it does not reserve any transaction units
+ * and ends up allocating a variable number of metadata extents for log
+ * tree extent buffers.
+ *
+ * We also need this 2 phases setup when adding a device to a filesystem with
+ * a seed device - we must create new metadata and system chunks without adding
+ * any of the block group items to the chunk, extent and device btrees. If we
+ * did not do it this way, we would get ENOSPC when attempting to update those
+ * btrees, since all the chunks from the seed device are read-only.
+ *
+ * Phase 1 does the updates and insertions to the chunk btree because if we had
+ * it done in phase 2 and have a thundering herd of tasks allocating chunks in
+ * parallel, we risk having too many system chunks allocated by many tasks if
+ * many tasks reach phase 1 without the previous ones completing phase 2. In the
+ * extreme case this leads to exhaustion of the system chunk array in the
+ * superblock. This is easier to trigger if using a btree node/leaf size of 64K
+ * and with RAID filesystems (so we have more device items in the chunk btree).
+ * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
+ * the system chunk array due to concurrent allocations") provides more details.
+ *
+ * For allocation of system chunks, we defer the updates and insertions into the
+ * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
+ * if the chunk allocation is triggered while COWing an extent buffer of the
+ * chunk btree, we are holding a lock on the parent of that extent buffer and
+ * doing the chunk btree updates and insertions can require locking that parent.
+ * This is for the very few and rare cases where we update the chunk btree that
+ * are not chunk allocation or chunk removal: adding a device, removing a device
+ * or resizing a device.
+ *
+ * The reservation of system space, done through check_system_chunk(), as well
+ * as all the updates and insertions into the chunk btree must be done while
+ * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
+ * an extent buffer from the chunks btree we never trigger allocation of a new
+ * system chunk, which would result in a deadlock (trying to lock twice an
+ * extent buffer of the chunk btree, first time before triggering the chunk
+ * allocation and the second time during chunk allocation while attempting to
+ * update the chunks btree). The system chunk array is also updated while holding
+ * that mutex. The same logic applies to removing chunks - we must reserve system
+ * space, update the chunk btree and the system chunk array in the superblock
+ * while holding fs_info->chunk_mutex.
+ *
+ * This function, btrfs_chunk_alloc(), belongs to phase 1.
+ *
+ * If @force is CHUNK_ALLOC_FORCE:
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
+ * If @force is NOT CHUNK_ALLOC_FORCE:
* - return 0 if it doesn't need to allocate a new chunk,
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
@@ -3253,6 +3464,13 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
@@ -3316,13 +3534,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
force_metadata_allocation(fs_info);
}
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
+ ret = do_chunk_alloc(trans, flags);
trans->allocating_chunk = false;
spin_lock(&space_info->lock);
@@ -3341,22 +3553,6 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
space_info->chunk_alloc = 0;
spin_unlock(&space_info->lock);
mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
return ret;
}
@@ -3409,14 +3605,31 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *bg;
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
+ *
+ * Also, if our caller is allocating a system chunk, do not
+ * attempt to insert the chunk item in the chunk btree, as we
+ * could deadlock on an extent buffer since our caller may be
+ * COWing an extent buffer from the chunk btree.
*/
- ret = btrfs_alloc_chunk(trans, flags);
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ /*
+ * If we fail to add the chunk item here, we end up
+ * trying again at phase 2 of chunk allocation, at
+ * btrfs_create_pending_block_groups(). So ignore
+ * any error here.
+ */
+ btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ }
}
if (!ret) {
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 7b927425dc71..c72a71efcb18 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -97,6 +97,7 @@ struct btrfs_block_group {
unsigned int removed:1;
unsigned int to_copy:1;
unsigned int relocating_repair:1;
+ unsigned int chunk_item_inserted:1;
int disk_cache_state;
@@ -268,8 +269,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work);
void btrfs_reclaim_bgs(struct btrfs_fs_info *fs_info);
void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg);
int btrfs_read_block_groups(struct btrfs_fs_info *info);
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size);
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
bool do_chunk_alloc);
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 4bc3ca2cbd7d..c5c08c87e130 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -364,49 +364,6 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
return 0;
}
-static struct extent_buffer *alloc_tree_block_no_bg_flush(
- struct btrfs_trans_handle *trans,
- struct btrfs_root *root,
- u64 parent_start,
- const struct btrfs_disk_key *disk_key,
- int level,
- u64 hint,
- u64 empty_size,
- enum btrfs_lock_nesting nest)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct extent_buffer *ret;
-
- /*
- * If we are COWing a node/leaf from the extent, chunk, device or free
- * space trees, make sure that we do not finish block group creation of
- * pending block groups. We do this to avoid a deadlock.
- * COWing can result in allocation of a new chunk, and flushing pending
- * block groups (btrfs_create_pending_block_groups()) can be triggered
- * when finishing allocation of a new chunk. Creation of a pending block
- * group modifies the extent, chunk, device and free space trees,
- * therefore we could deadlock with ourselves since we are holding a
- * lock on an extent buffer that btrfs_create_pending_block_groups() may
- * try to COW later.
- * For similar reasons, we also need to delay flushing pending block
- * groups when splitting a leaf or node, from one of those trees, since
- * we are holding a write lock on it and its parent or when inserting a
- * new root node for one of those trees.
- */
- if (root == fs_info->extent_root ||
- root == fs_info->chunk_root ||
- root == fs_info->dev_root ||
- root == fs_info->free_space_root)
- trans->can_flush_pending_bgs = false;
-
- ret = btrfs_alloc_tree_block(trans, root, parent_start,
- root->root_key.objectid, disk_key, level,
- hint, empty_size, nest);
- trans->can_flush_pending_bgs = true;
-
- return ret;
-}
-
/*
* does the dirty work in cow of a single block. The parent block (if
* supplied) is updated to point to the new cow copy. The new buffer is marked
@@ -455,8 +412,9 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
if ((root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) && parent)
parent_start = parent->start;
- cow = alloc_tree_block_no_bg_flush(trans, root, parent_start, &disk_key,
- level, search_start, empty_size, nest);
+ cow = btrfs_alloc_tree_block(trans, root, parent_start,
+ root->root_key.objectid, &disk_key, level,
+ search_start, empty_size, nest);
if (IS_ERR(cow))
return PTR_ERR(cow);
@@ -2458,9 +2416,9 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
else
btrfs_node_key(lower, &lower_key, 0);
- c = alloc_tree_block_no_bg_flush(trans, root, 0, &lower_key, level,
- root->node->start, 0,
- BTRFS_NESTING_NEW_ROOT);
+ c = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &lower_key, level, root->node->start, 0,
+ BTRFS_NESTING_NEW_ROOT);
if (IS_ERR(c))
return PTR_ERR(c);
@@ -2589,8 +2547,9 @@ static noinline int split_node(struct btrfs_trans_handle *trans,
mid = (c_nritems + 1) / 2;
btrfs_node_key(c, &disk_key, mid);
- split = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, level,
- c->start, 0, BTRFS_NESTING_SPLIT);
+ split = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, level, c->start, 0,
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(split))
return PTR_ERR(split);
@@ -3381,10 +3340,10 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans,
* BTRFS_NESTING_SPLIT_THE_SPLITTENING if we need to, but for now just
* use BTRFS_NESTING_NEW_ROOT.
*/
- right = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, 0,
- l->start, 0, num_doubles ?
- BTRFS_NESTING_NEW_ROOT :
- BTRFS_NESTING_SPLIT);
+ right = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, 0, l->start, 0,
+ num_doubles ? BTRFS_NESTING_NEW_ROOT :
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(right))
return PTR_ERR(right);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 443c348bc6f3..14b9fdc8aaa9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -254,8 +254,11 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
}
/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
+ * To be called after doing the chunk btree updates right after allocating a new
+ * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
+ * chunk after all chunk btree updates and after finishing the second phase of
+ * chunk allocation (btrfs_create_pending_block_groups()) in case some block
+ * group had its chunk item insertion delayed to the second phase.
*/
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
@@ -264,8 +267,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
if (!trans->chunk_bytes_reserved)
return;
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
trans->chunk_bytes_reserved = 0;
@@ -696,7 +697,6 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,
h->fs_info = root->fs_info;
h->type = type;
- h->can_flush_pending_bgs = true;
INIT_LIST_HEAD(&h->new_bgs);
smp_mb();
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index a18d67796b54..ba45065f9451 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -132,7 +132,7 @@ struct btrfs_trans_handle {
short aborted;
bool adding_csums;
bool allocating_chunk;
- bool can_flush_pending_bgs;
+ bool removing_chunk;
bool reloc_reserved;
bool in_fsync;
struct btrfs_root *root;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 782e16795bc4..c6c14315b1c9 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1745,19 +1745,14 @@ static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_handle_fs_error(fs_info, ret, "Slot search failed");
goto out;
}
*dev_extent_len = btrfs_dev_extent_length(leaf, extent);
ret = btrfs_del_item(trans, root, path);
- if (ret) {
- btrfs_handle_fs_error(fs_info, ret,
- "Failed to remove dev extent item");
- } else {
+ if (ret == 0)
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
- }
out:
btrfs_free_path(path);
return ret;
@@ -2942,7 +2937,7 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
u32 cur;
struct btrfs_key key;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
array_size = btrfs_super_sys_array_size(super_copy);
ptr = super_copy->sys_chunk_array;
@@ -2972,7 +2967,6 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
cur += len;
}
}
- mutex_unlock(&fs_info->chunk_mutex);
return ret;
}
@@ -3012,6 +3006,29 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,
return em;
}
+static int remove_chunk_item(struct btrfs_trans_handle *trans,
+ struct map_lookup *map, u64 chunk_offset)
+{
+ int i;
+
+ /*
+ * Removing chunk items and updating the device items in the chunks btree
+ * requires holding the chunk_mutex.
+ * See the comment at btrfs_chunk_alloc() for the details.
+ */
+ lockdep_assert_held(&trans->fs_info->chunk_mutex);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ int ret;
+
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ if (ret)
+ return ret;
+ }
+
+ return btrfs_free_chunk(trans, chunk_offset);
+}
+
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3032,14 +3049,16 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(em);
}
map = em->map_lookup;
- mutex_lock(&fs_info->chunk_mutex);
- check_system_chunk(trans, map->type);
- mutex_unlock(&fs_info->chunk_mutex);
/*
- * Take the device list mutex to prevent races with the final phase of
- * a device replace operation that replaces the device object associated
- * with map stripes (dev-replace.c:btrfs_dev_replace_finishing()).
+ * First delete the device extent items from the devices btree.
+ * We take the device_list_mutex to avoid racing with the finishing phase
+ * of a device replace operation. See the comment below before acquiring
+ * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
+ * because that can result in a deadlock when deleting the device extent
+ * items from the devices btree - COWing an extent buffer from the btree
+ * may result in allocating a new metadata chunk, which would attempt to
+ * lock again fs_info->chunk_mutex.
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
@@ -3061,18 +3080,73 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
btrfs_clear_space_info_full(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
}
+ }
+ mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_update_device(trans, device);
+ /*
+ * We acquire fs_info->chunk_mutex for 2 reasons:
+ *
+ * 1) Just like with the first phase of the chunk allocation, we must
+ * reserve system space, do all chunk btree updates and deletions, and
+ * update the system chunk array in the superblock while holding this
+ * mutex. This is for similar reasons as explained on the comment at
+ * the top of btrfs_chunk_alloc();
+ *
+ * 2) Prevent races with the final phase of a device replace operation
+ * that replaces the device object associated with the map's stripes,
+ * because the device object's id can change at any time during that
+ * final phase of the device replace operation
+ * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
+ * replaced device and then see it with an ID of
+ * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
+ * the device item, which does not exists on the chunk btree.
+ * The finishing phase of device replace acquires both the
+ * device_list_mutex and the chunk_mutex, in that order, so we are
+ * safe by just acquiring the chunk_mutex.
+ */
+ trans->removing_chunk = true;
+ mutex_lock(&fs_info->chunk_mutex);
+
+ check_system_chunk(trans, map->type);
+
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ /*
+ * Normally we should not get -ENOSPC since we reserved space before
+ * through the call to check_system_chunk().
+ *
+ * Despite our system space_info having enough free space, we may not
+ * be able to allocate extents from its block groups, because all have
+ * an incompatible profile, which will force us to allocate a new system
+ * block group with the right profile, or right after we called
+ * check_system_space() above, a scrub turned the only system block group
+ * with enough free space into RO mode.
+ * This is explained with more detail at do_chunk_alloc().
+ *
+ * So if we get -ENOSPC, allocate a new system chunk and retry once.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
goto out;
}
- }
- mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_free_chunk(trans, chunk_offset);
- if (ret) {
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -3087,6 +3161,15 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
}
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+
+ /*
+ * We are done with chunk btree updates and deletions, so release the
+ * system space we previously reserved (with check_system_chunk()).
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+
ret = btrfs_remove_block_group(trans, chunk_offset, em);
if (ret) {
btrfs_abort_transaction(trans, ret);
@@ -3094,6 +3177,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
out:
+ if (trans->removing_chunk) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ }
/* once for us */
free_extent_map(em);
return ret;
@@ -4860,13 +4947,12 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
u32 array_size;
u8 *ptr;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
+
array_size = btrfs_super_sys_array_size(super_copy);
if (array_size + item_size + sizeof(disk_key)
- > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
- mutex_unlock(&fs_info->chunk_mutex);
+ > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
return -EFBIG;
- }
ptr = super_copy->sys_chunk_array + array_size;
btrfs_cpu_key_to_disk(&disk_key, key);
@@ -4875,7 +4961,6 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
memcpy(ptr, chunk, item_size);
item_size += sizeof(disk_key);
btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
- mutex_unlock(&fs_info->chunk_mutex);
return 0;
}
@@ -5225,13 +5310,14 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
}
}
-static int create_chunk(struct btrfs_trans_handle *trans,
+static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
{
struct btrfs_fs_info *info = trans->fs_info;
struct map_lookup *map = NULL;
struct extent_map_tree *em_tree;
+ struct btrfs_block_group *block_group;
struct extent_map *em;
u64 start = ctl->start;
u64 type = ctl->type;
@@ -5241,7 +5327,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
map = kmalloc(map_lookup_size(ctl->num_stripes), GFP_NOFS);
if (!map)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
map->num_stripes = ctl->num_stripes;
for (i = 0; i < ctl->ndevs; ++i) {
@@ -5263,7 +5349,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
em = alloc_extent_map();
if (!em) {
kfree(map);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
set_bit(EXTENT_FLAG_FS_MAPPING, &em->flags);
em->map_lookup = map;
@@ -5279,12 +5365,12 @@ static int create_chunk(struct btrfs_trans_handle *trans,
if (ret) {
write_unlock(&em_tree->lock);
free_extent_map(em);
- return ret;
+ return ERR_PTR(ret);
}
write_unlock(&em_tree->lock);
- ret = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
- if (ret)
+ block_group = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
+ if (IS_ERR(block_group))
goto error_del_extent;
for (i = 0; i < map->num_stripes; i++) {
@@ -5304,7 +5390,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
check_raid56_incompat_flag(info, type);
check_raid1c34_incompat_flag(info, type);
- return 0;
+ return block_group;
error_del_extent:
write_lock(&em_tree->lock);
@@ -5316,34 +5402,36 @@ static int create_chunk(struct btrfs_trans_handle *trans,
/* One for the tree reference */
free_extent_map(em);
- return ret;
+ return block_group;
}
-int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
+struct btrfs_block_group *btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ u64 type)
{
struct btrfs_fs_info *info = trans->fs_info;
struct btrfs_fs_devices *fs_devices = info->fs_devices;
struct btrfs_device_info *devices_info = NULL;
struct alloc_chunk_ctl ctl;
+ struct btrfs_block_group *block_group;
int ret;
lockdep_assert_held(&info->chunk_mutex);
if (!alloc_profile_is_valid(type, 0)) {
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (list_empty(&fs_devices->alloc_list)) {
if (btrfs_test_opt(info, ENOSPC_DEBUG))
btrfs_debug(info, "%s: no writable device", __func__);
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
}
if (!(type & BTRFS_BLOCK_GROUP_TYPE_MASK)) {
btrfs_err(info, "invalid chunk type 0x%llx requested", type);
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
ctl.start = find_next_chunk(info);
@@ -5353,46 +5441,43 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
devices_info = kcalloc(fs_devices->rw_devices, sizeof(*devices_info),
GFP_NOFS);
if (!devices_info)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = gather_device_info(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
ret = decide_stripe_size(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
- ret = create_chunk(trans, &ctl, devices_info);
+ block_group = create_chunk(trans, &ctl, devices_info);
out:
kfree(devices_info);
- return ret;
+ return block_group;
}
/*
- * Chunk allocation falls into two parts. The first part does work
]
]
|
| 92/2 |
2026/08/08 03:35 |
llm |
debugger |
0m
Model:
Tokens:
input: 115707
output: 40
thoughts: 1753
Thoughts: **Analyzing the btrfs Chunk Removal Process**
Okay, so I'm looking at commit `79bd37120b149532af5b21953643ed74af69654f` where they modified the `btrfs_remove_chunk` function. Before this change, the chunk removal process looked like this:
1. Lock `fs_info->chunk_mutex`.
2. `check_system_chunk`.
3. Unlock `fs_info->chunk_mutex`.
4. Lock `fs_devices->device_list_mutex`.
5. Iterate through stripes, calling `btrfs_free_dev_extent` and `btrfs_update_device`.
6. Unlock `fs_devices->device_list_mutex`.
7. `btrfs_free_chunk`.
The key realization here is that `btrfs_free_chunk` was called *after* `btrfs_free_dev_extent`. `btrfs_free_chunk` is the one that removes the chunk item from the chunk tree, and `btrfs_free_dev_extent` removes entries from the device tree. This order makes sense because the chunk tree is modified, so should be done after the device extents have been dealt with.
Now, with the new code:
1. Lock `fs_devices->device_list_mutex`.
2. Iterate through stripes, calling `btrfs_free_dev_extent`.
3. Unlock `fs_devices->device_list_mutex`.
4. Lock `fs_info->chunk_mutex`.
5. `check_system_chunk`.
6. `remove_chunk_item`.
The question is, why? Why make this change?
My initial thought is that we could have a more graceful failure by moving the `remove_chunk_item` call and handling any `-ENOSPC` errors. We could potentially return `-ENOSPC` and NOT abort the transaction! However, there is the potential that it fails to allocate system chunks and retries.
However, after a closer look, a new question arises. The `btrfs_free_dev_extent` call also *updates* the device item using `device->bytes_used`. It adds the device to `trans->transaction->dev_update_list` which will update it in the device tree during commit.
Now, I see that `remove_chunk_item` calls `btrfs_update_device`, which updates the device item *in the chunk tree* with the current `device->bytes_used`. If we swap the order, then we'll be updating the chunk tree entry with the *old* `bytes_used` value, and then the device will be updated during the commit.
The device item will eventually be updated during transaction commit because of that `trans->transaction->dev_update_list`. However, during that commit phase, we're using the normal block reserve, not the chunk block reserve. Updating the device during transaction commit could fail if there's no system space available, which would abort the transaction.
My conclusion is that `remove_chunk_item` *must* be done before the `btrfs_free_dev_extent` calls to update the device item in the chunk tree. Doing the device update *during commit* would be risky because it would use the standard block reserve, and the update could fail if system space is exhausted. But, if the device updates are performed on the transaction commit, they will use the new `bytes_used` value anyway, meaning it may still work!
|
| 93/2 |
2026/08/08 03:36 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_update_device]
Results: map[SourceCode:3105: int btrfs_update_device(struct btrfs_trans_handle *trans, struct btrfs_device *device)
3106: {
3107: int ret;
3108: BTRFS_PATH_AUTO_FREE(path);
3109: struct btrfs_root *root = device->fs_info->chunk_root;
3110: struct btrfs_dev_item *dev_item;
3111: struct extent_buffer *leaf;
3112: struct btrfs_key key;
3113:
3114: path = btrfs_alloc_path();
3115: if (!path)
3116: return -ENOMEM;
3117:
3118: key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
3119: key.type = BTRFS_DEV_ITEM_KEY;
3120: key.offset = device->devid;
3121:
3122: ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
3123: if (ret < 0)
3124: return ret;
3125:
3126: if (ret > 0)
3127: return -ENOENT;
3128:
3129: leaf = path->nodes[0];
3130: dev_item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_dev_item);
3131:
3132: btrfs_set_device_id(leaf, dev_item, device->devid);
3133: btrfs_set_device_type(leaf, dev_item, device->type);
3134: btrfs_set_device_io_align(leaf, dev_item, device->io_align);
3135: btrfs_set_device_io_width(leaf, dev_item, device->io_width);
3136: btrfs_set_device_sector_size(leaf, dev_item, device->sector_size);
3137: btrfs_set_device_total_bytes(leaf, dev_item,
3138: btrfs_device_get_disk_total_bytes(device));
3139: btrfs_set_device_bytes_used(leaf, dev_item,
3140: btrfs_device_get_bytes_used(device));
3141: return ret;
3142: }
SourceFile:fs/btrfs/volumes.c]
|
| 94/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 119020
output: 19
thoughts: 121
Thoughts: Okay, let me think this through.
**Analyzing Chunk Tree Modification During Transaction Commit**
Alright, so `btrfs_update_device` is the function that directly touches the chunk tree, right? I need to be careful with that. And if it's being called during a transaction commit, then any potential conflicts could be problematic. I need to understand the timing here. The next thing that leaps to mind is `btrfs_create_pending_block_groups`. Ah, yes, *that* also modifies the chunk tree, but only if `chunk_item_inserted` is false. That's another potential point of contention. Wait... am I right? It seems `btrfs_update_device` for the devices in `dev_update_list` is definitely being called inside `btrfs_commit_transaction`. That feels like a race condition waiting to happen! Now, the critical piece is where is `dev_update_list` being *processed* and what's its context. Gotta trace that and confirm.
|
| 95/2 |
2026/08/08 03:36 |
tool |
grepper |
0m
Args: map[Expression:dev_update_list]
Results: map[Output:fs/btrfs/dev-replace.c=440=static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
--
fs/btrfs/dev-replace.c-462- while (fs_info->running_transaction &&
fs/btrfs/dev-replace.c:463: !list_empty(&fs_info->running_transaction->dev_update_list)) {
fs/btrfs/dev-replace.c-464- spin_unlock(&fs_info->trans_lock);
--
fs/btrfs/disk-io.c=4890=void btrfs_cleanup_one_transaction(struct btrfs_transaction *cur_trans)
--
fs/btrfs/disk-io.c-4898-
fs/btrfs/disk-io.c:4899: list_for_each_entry_safe(dev, tmp, &cur_trans->dev_update_list,
fs/btrfs/disk-io.c-4900- post_commit_list) {
--
fs/btrfs/transaction.c=142=void btrfs_put_transaction(struct btrfs_transaction *transaction)
--
fs/btrfs/transaction.c-174- }
fs/btrfs/transaction.c:175: WARN_ON(!list_empty(&transaction->dev_update_list));
fs/btrfs/transaction.c-176- kfree(transaction);
--
fs/btrfs/transaction.c=270=static noinline int join_transaction(struct btrfs_fs_info *fs_info,
--
fs/btrfs/transaction.c-378- INIT_LIST_HEAD(&cur_trans->pending_snapshots);
fs/btrfs/transaction.c:379: INIT_LIST_HEAD(&cur_trans->dev_update_list);
fs/btrfs/transaction.c-380- INIT_LIST_HEAD(&cur_trans->switch_commits);
--
fs/btrfs/transaction.h=51=struct btrfs_transaction {
--
fs/btrfs/transaction.h-76- struct list_head pending_snapshots;
fs/btrfs/transaction.h:77: struct list_head dev_update_list;
fs/btrfs/transaction.h-78- struct list_head switch_commits;
--
fs/btrfs/volumes.c=3144=int btrfs_grow_device(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-3177- list_add_tail(&device->post_commit_list,
fs/btrfs/volumes.c:3178: &trans->transaction->dev_update_list);
fs/btrfs/volumes.c-3179- btrfs_update_per_profile_avail(fs_info);
--
fs/btrfs/volumes.c=3404=int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
--
fs/btrfs/volumes.c-3441- list_add_tail(&device->post_commit_list,
fs/btrfs/volumes.c:3442: &trans->transaction->dev_update_list);
fs/btrfs/volumes.c-3443- }
--
fs/btrfs/volumes.c=5226=int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
--
fs/btrfs/volumes.c-5398- list_add_tail(&device->post_commit_list,
fs/btrfs/volumes.c:5399: &trans->transaction->dev_update_list);
fs/btrfs/volumes.c-5400-
--
fs/btrfs/volumes.c=6052=static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-6106- list_add_tail(&dev->post_commit_list,
fs/btrfs/volumes.c:6107: &trans->transaction->dev_update_list);
fs/btrfs/volumes.c-6108- }
--
fs/btrfs/volumes.c=8504=void btrfs_commit_device_sizes(struct btrfs_transaction *trans)
--
fs/btrfs/volumes.c-8509-
fs/btrfs/volumes.c:8510: if (list_empty(&trans->dev_update_list))
fs/btrfs/volumes.c-8511- return;
--
fs/btrfs/volumes.c-8518- mutex_lock(&trans->fs_info->chunk_mutex);
fs/btrfs/volumes.c:8519: list_for_each_entry_safe(curr, next, &trans->dev_update_list,
fs/btrfs/volumes.c-8520- post_commit_list) {
]
|
| 96/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 120712
output: 42
thoughts: 37
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Diving into btrfs_commit_device_sizes**
Right, `btrfs_commit_device_sizes` is what I'm looking at. That function is the workhorse behind committing device size changes within Btrfs. My immediate focus is on how it handles the `dev_update_list`. Knowing that's the input, I can start forming a mental model of how it processes those size updates. Time to dig in and see the exact mechanics of that interaction. What's it doing? How are the updates coordinated?
|
| 97/2 |
2026/08/08 03:36 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_commit_device_sizes]
Results: map[SourceCode:8497: /*
8498: * Update the size and bytes used for each device where it changed. This is
8499: * delayed since we would otherwise get errors while writing out the
8500: * superblocks.
8501: *
8502: * Must be invoked during transaction commit.
8503: */
8504: void btrfs_commit_device_sizes(struct btrfs_transaction *trans)
8505: {
8506: struct btrfs_device *curr, *next;
8507:
8508: ASSERT(trans->state == TRANS_STATE_COMMIT_DOING, "state=%d" , trans->state);
8509:
8510: if (list_empty(&trans->dev_update_list))
8511: return;
8512:
8513: /*
8514: * We don't need the device_list_mutex here. This list is owned by the
8515: * transaction and the transaction must complete before the device is
8516: * released.
8517: */
8518: mutex_lock(&trans->fs_info->chunk_mutex);
8519: list_for_each_entry_safe(curr, next, &trans->dev_update_list,
8520: post_commit_list) {
8521: list_del_init(&curr->post_commit_list);
8522: curr->commit_total_bytes = curr->disk_total_bytes;
8523: curr->commit_bytes_used = curr->bytes_used;
8524: }
8525: mutex_unlock(&trans->fs_info->chunk_mutex);
8526: }
SourceFile:fs/btrfs/volumes.c]
|
| 98/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 121428
output: 40
thoughts: 1211
Thoughts: **Analyzing Btrfs Chunk Removal and System Space**
Okay, so here's what's running through my head about this btrfs chunk removal process... It seems the critical issue is the order of operations concerning device size updates and chunk tree modification. Specifically, `btrfs_commit_device_sizes` only touches `commit_total_bytes` and `commit_bytes_used` -- it *doesn't* call `btrfs_update_device`. This means `btrfs_update_device` *must* be called *before* transaction commit. And, importantly, `remove_chunk_item` *does* call `btrfs_update_device`.
Now, if we were to incorrectly call `remove_chunk_item` *before* `btrfs_remove_dev_extents`, `btrfs_update_device` would write the *old* `bytes_used` value to the chunk tree. Then, `btrfs_remove_dev_extents` decreases `bytes_used`. But, if `btrfs_update_device` isn't called again, the chunk tree will have the *wrong* `bytes_used` for the device! That's why `btrfs_remove_dev_extents` *has* to come before `remove_chunk_item`.
So, swapping the order is a no-go. Given that constraint, it is critical to ensure enough system space before removing those extents. The current approach uses `check_system_chunk`, and the main issue that I see is that it is ignoring the error that it throws. I'm thinking if we could simply return an error, we could fail gracefully and avoid catastrophic data corruption, as we are leaving an empty block group which is perfectly acceptable behavior. The cleaner thread will eventually come and remove it again. The alternative of aborting the entire transaction would have significant drawbacks.
Could we change `check_system_chunk` to return an error, and handle that in `btrfs_remove_chunk`?
```c
mutex_lock(&fs_info->chunk_mutex);
ret = check_system_chunk(trans, map->type);
mutex_unlock(&fs_info->chunk_mutex);
if (ret)
goto out;
```
That seems reasonable, *except* `check_system_chunk` is also used in `do_chunk_alloc`. If `check_system_chunk` returned an error, that would fail chunk allocation, which might not always be the right thing to do! Looking at `do_chunk_alloc` it appears that `-ENOSPC` is handled by retrying a system chunk allocation anyway. But then I'm reminded of the comment in `check_system_chunk`, which basically says ignoring the error is intentional because space *might* be unneeded due to existing COWs. This means that failing might prevent allocations that could have succeeded.
So, how can we *only* fail `btrfs_remove_chunk` if we can't allocate a system chunk?
I could add a parameter to `check_system_chunk` or `reserve_chunk_space` to indicate an error or ignore it.
Or, I could just check system space *before* `btrfs_remove_dev_extents`.
```c
/* Check if we have enough system space to remove the chunk */
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
/* Did we reserve enough space? */
if (trans->chunk_bytes_reserved < required_bytes) {
/* We didn't reserve enough space, and check_system_chunk ignored the error. */
/* We can fail gracefully here. */
}
mutex_unlock(&fs_info->chunk_mutex);
```
Okay, the more I think about it, I could just check `trans->chunk_bytes_reserved` because `check_system_chunk` *adds* reserved bytes to it when it is successful. If it fails, those bytes *aren't* added, so by checking if the value changed, I can see if a chunk allocation worked, and fail only if it didn't. Though `check_system_chunk` might not need to allocate a chunk if the space is there.
|
| 99/2 |
2026/08/08 03:36 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 100/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 124887
output: 40
thoughts: 1998
Thoughts: **Chunk Removal with Robustness**
Okay, here's the plan for improving chunk removal reliability. If `btrfs_create_chunk` fails during the process, it returns an error, such as `-ENOSPC`. This means the code won't add bytes to `chunk_block_rsv` or increment `trans->chunk_bytes_reserved`. We can leverage this to check if reservation actually succeeded.
I'll record the value of `trans->chunk_bytes_reserved` *before* calling `check_system_chunk`, which is responsible for allocating space if needed. If, after calling this function, the value of the counter *hasn't* increased, it means we failed to reserve the space required. In this case, I can return `-ENOSPC` *before* calling `btrfs_remove_dev_extents`. This prevents a potentially disastrous transaction abort. Instead, the balance operation will fail with `-ENOSPC`, the user gets a clear indication of a resource issue, and the filesystem remains consistent. This is a preferable outcome to a corrupted filesystem.
I verified that `check_system_chunk` is the only place that increments the `trans->chunk_bytes_reserved` counter. So the strategy should be safe, and can be implemented in `btrfs_remove_chunk`.
The revised flow should be:
1. **Reserve Space:** Lock the `chunk_mutex`, set `removing_chunk` to true, and call `check_system_chunk`. Check if `trans->chunk_bytes_reserved` increased as expected. If not, it means we failed to reserve the space, return `-ENOSPC`, and unlock the mutex.
2. **Remove Extents:** If space was reserved, we will unlock the `chunk_mutex`, set `removing_chunk` to false, and call `btrfs_remove_dev_extents`. If this succeeds, the chunk extents have been removed.
3. **Finish Removal:** This part is handled later and should be fine.
It's safe to hold the reserved space in `chunk_block_rsv` across the call to `btrfs_remove_dev_extents`. `trans->chunk_bytes_reserved` tracks reserved space, and this reservation is released either at the end of the transaction or upon transaction cleanup on error. This is, in fact, the behavior we want.
I am aware that `check_system_chunk` doesn't explicitly return how many bytes it reserved, but that's fine. We just check if `trans->chunk_bytes_reserved` increased.
I'm aware that a failure to reserve system chunk space doesn't necessarily mean the removal itself *couldn't* succeed. However, because chunk removal is an optimization, this is an acceptable tradeoff. If we are so low on space that we can't *even reserve* it for this operation, then it's better to fail the removal and the cleaner will try again later. If the user runs balance, it will fail with `-ENOSPC`, which indicates a full filesystem.
I've reviewed the error handling in `btrfs_remove_chunk`. If `btrfs_remove_dev_extents` fails, it *does* correctly go to an error label to unlock the `chunk_mutex` and free the map. Importantly, `btrfs_trans_release_chunk_metadata(trans)` is called as part of `btrfs_end_transaction`, which is *always* called by the caller. This ensures that the reserved chunk space is released correctly. We're safe to return `-ENOSPC` from the failure to reserve space because the reservation will be released at transaction end.
|
| 101/2 |
2026/08/08 03:36 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/transaction.c Name:btrfs_end_transaction]
Results: map[SourceCode:1136: int btrfs_end_transaction(struct btrfs_trans_handle *trans)
1137: {
1138: return __btrfs_end_transaction(trans, 0);
1139: }
SourceFile:fs/btrfs/transaction.c]
|
| 102/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 127819
output: 41
thoughts: 11
Thoughts: Okay, here's my summary, taking on the persona you've requested, targeting an expert audience:
**Analyzing `__btrfs_end_transaction`**
Alright, let's dissect this. We're looking at `__btrfs_end_transaction`. Immediately, I'm thinking about the critical role this function plays in ensuring data consistency and durability within the Btrfs filesystem. It's the point where changes are committed, and the integrity of the filesystem is cemented. We're talking about flush operations, journal synchronization, potentially checksum recalculations, and all the crucial steps before announcing a completed transaction.
My immediate concerns center on potential bottlenecks. How efficiently does it handle the metadata updates? Are we seeing any significant lock contention in this area, particularly under heavy write loads? I'm also curious about the specific strategies implemented for crash consistency. Are we relying heavily on write barriers and ordered writes, or are there more advanced techniques in play to minimize the window of vulnerability during a power loss?
Beyond the basics, I'm scrutinizing the interplay with the different transaction commit modes. Does this function differentiate between synchronous and asynchronous commits? Are there any optimizations tied to the `nobarrier` mount option? I'd expect to see a carefully crafted balance between performance and data integrity, and I'm interested in how well that is achieved.
Furthermore, let's explore recovery scenarios. How robust is the undo/redo functionality here? If a crash occurs *during* this function, what are the mechanisms in place to ensure that the filesystem remains in a consistent state upon reboot? What about the handling of partial transaction failures within this function? I would expect to see detailed logging and error handling, but I'm looking for the depth of those implementations.
Finally, I'm thinking about potential performance implications. Is there a profiling interface that would allow me to isolate `__btrfs_end_transaction` in high-throughput workloads? I'd be looking for a breakdown of time spent on different sub-operations within the function - specifically to identify optimization opportunities.
|
| 103/2 |
2026/08/08 03:36 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/transaction.c Name:__btrfs_end_transaction]
Results: map[SourceCode:1076: static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
1077: int throttle)
1078: {
1079: struct btrfs_fs_info *info = trans->fs_info;
1080: struct btrfs_transaction *cur_trans = trans->transaction;
1081: int ret = 0;
1082:
1083: if (refcount_read(&trans->use_count) > 1) {
1084: refcount_dec(&trans->use_count);
1085: trans->block_rsv = trans->orig_rsv;
1086: return 0;
1087: }
1088:
1089: btrfs_trans_release_metadata(trans);
1090: trans->block_rsv = NULL;
1091:
1092: btrfs_create_pending_block_groups(trans);
1093:
1094: btrfs_trans_release_chunk_metadata(trans);
1095:
1096: if (trans->type & __TRANS_FREEZABLE)
1097: sb_end_intwrite(info->sb);
1098:
1099: /*
1100: * Uninhibit extent buffer writeback before decrementing num_writers,
1101: * since the decrement wakes the committing thread which needs all
1102: * buffers uninhibited to write them to disk.
1103: */
1104: btrfs_uninhibit_all_eb_writeback(trans);
1105:
1106: WARN_ON(cur_trans != info->running_transaction);
1107: WARN_ON(atomic_read(&cur_trans->num_writers) < 1);
1108: atomic_dec(&cur_trans->num_writers);
1109: extwriter_counter_dec(cur_trans, trans->type);
1110:
1111: cond_wake_up(&cur_trans->writer_wait);
1112:
1113: btrfs_lockdep_release(info, btrfs_trans_num_extwriters);
1114: btrfs_lockdep_release(info, btrfs_trans_num_writers);
1115:
1116: btrfs_put_transaction(cur_trans);
1117:
1118: if (current->journal_info == trans)
1119: current->journal_info = NULL;
1120:
1121: if (throttle)
1122: btrfs_run_delayed_iputs(info);
1123:
1124: if (unlikely(TRANS_ABORTED(trans) || BTRFS_FS_ERROR(info))) {
1125: wake_up_process(info->transaction_kthread);
1126: if (TRANS_ABORTED(trans))
1127: ret = trans->aborted;
1128: else
1129: ret = -EROFS;
1130: }
1131:
1132: kmem_cache_free(btrfs_trans_handle_cachep, trans);
1133: return ret;
1134: }
SourceFile:fs/btrfs/transaction.c]
|
| 104/2 |
2026/08/08 03:36 |
llm |
debugger |
0m
Model:
Tokens:
input: 129242
output: 57
thoughts: 1121
Thoughts: **Refining the Chunk Removal Process in Btrfs**
Okay, so I'm looking at this chunk removal process in Btrfs, and trying to optimize it to avoid unnecessary space reservation, and handle potential errors gracefully. First, `__btrfs_end_transaction` calls `btrfs_trans_release_chunk_metadata`, which is what we want – we can reserve space early. Great.
The tricky part is that we have a code block:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
We want to reserve space *before* calling `check_system_chunk`, and then handle the error if reservation fails. But, hold on... if we move `check_system_chunk` to *before* `mutex_lock` like so:
```c
/*
* Reserve system space before removing the device extents.
* If we fail to reserve space, we can fail gracefully.
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
```
...then we don't call `check_system_chunk` again inside the critical section. *This is good*, we don't double reserve. But what if `remove_chunk_item` *still* fails with `-ENOSPC`? The comment is enlightening, explaining that even if our `space_info` shows enough free space, we could have an issue where incompatible profiles are present in the block groups, forcing allocation of a new system group, which leads to `check_system_space` being called, or a scrub could make a group RO after the check, leading to failure. The comment suggests retrying the chunk allocation.
So we are now at a point where *we need to consider a failure case.*
But wait, if we try to allocate a *new* system chunk in `remove_chunk_item` and it returns `-ENOSPC`, that will fail because `find_free_dev_extent` searches the commit root, and a scrubbed RO block group can't be modified. So we'd abort the transaction! Is that acceptable?
Yes, this is an incredibly rare race condition. The main problem we are solving is deterministic failures when we are out of system space. Checking `trans->chunk_bytes_reserved` early catches that case. The race condition, while still an issue, is much less common.
Now, is there a way to make `find_free_dev_extent` see the space we've freed, so we wouldn't need to abort? We want the transaction to complete if possible! Why is `find_free_dev_extent` searching the commit root in the first place? It's due to `b14c5e04bd0f` ("btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation").
...But wait, `b14c5e04bd0f` *didn't* introduce the commit root search! It was already there. We need to dig into the history to figure out when the `search_commit_root` behavior was introduced.
|
| 105/2 |
2026/08/08 03:37 |
tool |
git-log |
0m
Args: map[CodeRegexp:search_commit_root = true Count:10 SourcePath:fs/btrfs/volumes.c SymbolName:find_free_dev_extent]
Results: map[Output:b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
d7fe41044b3a btrfs: use bool type for btrfs_path members used as booleans
7fc35cc559cb btrfs: more trivial BTRFS_PATH_AUTO_FREE conversions
94cb8d714434 btrfs: use verbose ASSERT() in volumes.c
dba6ae0b439f btrfs: unify ordering of btrfs_key initializations
20218dfbaa31 btrfs: make sure to initialize start and len in find_free_dev_extent
ed8947bc73aa btrfs: merge find_free_dev_extent() and find_free_dev_extent_start()
3c538de0f2a7 btrfs: limit device extents to the device size
0ff40a910f56 btrfs: introduce btrfs_search_backwards function
1cd6121f2a38 btrfs: zoned: implement zoned chunk allocator
]
|
| 106/2 |
2026/08/08 03:37 |
llm |
debugger |
0m
Model:
Tokens:
input: 131640
output: 58
thoughts: 7
Thoughts: Okay, here's the summary, assuming I'm the expert and the "text" simply stated "Let's search further back":
**Diving Deeper into the Data**
Alright, that's the starting point. But, based on my understanding of the current trajectory and potential nuances in this area, "let's search further back" is the correct response. We need to push the boundaries of the timeframe. The more granular the data and the deeper back we go, the more opportunities we create. The immediate surface is useful, of course, but the real power lies in uncovering those initial, foundational elements that set the stage for what we're observing now. This isn't just about chronological completeness; it's about identifying the critical antecedents, the root causes, and the hidden correlations that we might be missing by staying too close to the present. The challenge, as always, will be to maintain focus. We need to stay on track and maintain a clear understanding of what we are looking for.
|
| 107/2 |
2026/08/08 03:37 |
tool |
git-log |
0m
Args: map[CodeRegexp:search_commit_root = 1 Count:10 SourcePath:fs/btrfs/volumes.c SymbolName:find_free_dev_extent]
Results: map[Output:b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
d7fe41044b3a btrfs: use bool type for btrfs_path members used as booleans
7fc35cc559cb btrfs: more trivial BTRFS_PATH_AUTO_FREE conversions
94cb8d714434 btrfs: use verbose ASSERT() in volumes.c
dba6ae0b439f btrfs: unify ordering of btrfs_key initializations
20218dfbaa31 btrfs: make sure to initialize start and len in find_free_dev_extent
ed8947bc73aa btrfs: merge find_free_dev_extent() and find_free_dev_extent_start()
3c538de0f2a7 btrfs: limit device extents to the device size
0ff40a910f56 btrfs: introduce btrfs_search_backwards function
1cd6121f2a38 btrfs: zoned: implement zoned chunk allocator
]
|
| 108/2 |
2026/08/08 03:37 |
llm |
debugger |
0m
Model:
Tokens:
input: 132198
output: 42
thoughts: 71
Thoughts: **Investigating the Origin of `search_commit_root = 1`**
Okay, so I need to figure out where this `search_commit_root = 1` thing came from. My initial instinct was that it was introduced in that commit, `1cd6121f2a38`. But something feels off, so I’m going to go back and check the `git log` to see if there are other commits that touch the relevant path and symbol.
Alright, scratch that, I'm going to quickly bypass the git log, and use grep. Let's just grep the history of `fs/btrfs/volumes.c` looking for `search_commit_root`. That'll be much faster and likely lead me to the answer, or at least a manageable set of possibilities.
|
| 109/2 |
2026/08/08 03:37 |
tool |
git-log |
2m
Args: map[CodeRegexp:search_commit_root Count:10 SourcePath:fs/btrfs/volumes.c]
Results: map[Output:3dcd50730814 btrfs: fallback to transaction csum tree on a commit root csum miss
f84f833a72c7 btrfs: use on stack backref iterator in build_backref_tree()
5254d4181add btrfs: fix zero size inode with non-zero size after log replay
e46a9f84bfe8 btrfs: allocate path on stack in load_block_group_size_class()
3dcdcb717763 btrfs: shrink the size of btrfs_bio
d7fe41044b3a btrfs: use bool type for btrfs_path members used as booleans
f07b855c56b1 btrfs: try to search for data csums in commit root
e560afc1a83d btrfs: set search_commit_root to false in iterate_inodes_from_logical()
f4d39cf1cebf btrfs: set search_commit_root on stripe io in case of relocation
d6106f0dc502 btrfs: rename btrfs_io_stripe::is_scrub to rst_search_commit_root
]
|
| 110/2 |
2026/08/08 03:40 |
llm |
debugger |
0m
Model:
Tokens:
input: 132796
output: 27
thoughts: 25
Thoughts: Here's my thought process, as if I were considering this myself:
**Tracing Down That `search_commit_root` Call**
Okay, so I need to track down exactly *when* the `search_commit_root` function was introduced into `find_free_dev_extent`. Sounds like a relatively straightforward debugging task, but let's make sure we're being precise. My goal here is probably to understand why it's there, and ideally, which related changes might have led to its addition. I'll probably want to know what problem it was originally intended to solve. Maybe there's a specific commit message or related issue that offers additional context. Right now, I'm thinking I'll start with a git blame, or potentially a git log with some clever filtering. I'll look for commits that modify the `find_free_dev_extent` function and try to narrow down the introduction of the specific function call, `search_commit_root`. I'll want to review the code changes and commit messages to understand the intention behind this addition, and if possible, track down any associated bug reports or design documents. If there's a refactoring, bug fix, or new feature involved, that would be interesting information.
|
| 111/2 |
2026/08/08 03:40 |
tool |
grepper |
0m
Error:
Args: map[Expression:search_commit_root.*find_free_dev_extent]
Results:
|
| 112/2 |
2026/08/08 03:40 |
llm |
debugger |
0m
Model:
Tokens:
input: 133125
output: 39
thoughts: 33
Thoughts: Here's my analysis of the situation:
**Investigating `find_free_dev_extent` in btrfs**
Okay, so I need to understand what's happening with the `find_free_dev_extent` function within `fs/btrfs/volumes.c`. My first instinct, as always, is to dive right into the code and see what's changed and who's been touching it recently. `git blame` is my go-to tool for this sort of exploration. I want to see the history of this specific function, pinpoint who introduced the code, and understand the context of any modifications. This will hopefully quickly illuminate the areas I need to focus on to resolve the issue at hand, and prevent a deep dive into irrelevant code.
|
| 113/2 |
2026/08/08 03:40 |
tool |
git-blame |
0m
Args: map[End:1855 File:fs/btrfs/volumes.c Start:1845]
Results: map[Output:401e29c124eac 1845) test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
7bfc837df935d 1846) ret = -ENOSPC;
6df9a95e63395 1847) goto out;
7bfc837df935d 1848) }
7bfc837df935d 1849)
e4058b54d1e44 1850) path->reada = READA_FORWARD;
d7fe41044b3ac 1851) path->search_commit_root = true;
d7fe41044b3ac 1852) path->skip_locking = true;
7bfc837df935d 1853)
0b86a832a1f38 1854) key.objectid = device->devid;
0b86a832a1f38 1855) key.type = BTRFS_DEV_EXTENT_KEY;
]
|
| 114/2 |
2026/08/08 03:40 |
llm |
debugger |
0m
Model:
Tokens:
input: 133685
output: 39
thoughts: 43
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm deep in the code:
**Debugging: The Blame Game Begins**
Alright, so `d7fe41044b3ac` just flipped a `1` to `true`. That's... interesting. Definitely not what I expect to see. This feels like a classic case of a boolean mishap. My instinct is to trace this back, to isolate what could have caused this. The first step is to establish what happened *before* this problematic commit. I need to understand what state the system was in *before* `d7fe41044b3ac` made this change. Let's see if we can identify where the problem originated. It's time to investigate what caused this.
|
| 115/2 |
2026/08/08 03:40 |
tool |
git-blame |
0m
Args: map[End:1855 File:fs/btrfs/volumes.c Start:1845]
Results: map[Output:401e29c124eac 1845) test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
7bfc837df935d 1846) ret = -ENOSPC;
6df9a95e63395 1847) goto out;
7bfc837df935d 1848) }
7bfc837df935d 1849)
e4058b54d1e44 1850) path->reada = READA_FORWARD;
d7fe41044b3ac 1851) path->search_commit_root = true;
d7fe41044b3ac 1852) path->skip_locking = true;
7bfc837df935d 1853)
0b86a832a1f38 1854) key.objectid = device->devid;
0b86a832a1f38 1855) key.type = BTRFS_DEV_EXTENT_KEY;
]
|
| 116/2 |
2026/08/08 03:40 |
llm |
debugger |
0m
Model:
Tokens:
input: 134261
output: 26
thoughts: 22
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**Identifying the Root Cause**
Alright, let's get down to brass tacks. Something's gone sideways, and I need to pinpoint the exact source of this issue. My initial gut feeling is that the problem likely stems from changes introduced prior to the commit identified as `d7fe41044b3ac`. I need to zero in on the exact commit because a bug is likely sitting in the introduced lines of code. I need to be exact here, especially when debugging. I am going to have to blame a commit before that one, to effectively root out the problem.
|
| 117/2 |
2026/08/08 03:40 |
tool |
git-show |
0m
Args: map[Commit:d7fe41044b3ac]
Results: map[Output:commit d7fe41044b3ac8f9b5965de499a13ac9ae947e79
Author: Filipe Manana <fdmanana@suse.com>
Date: Fri Nov 14 16:00:04 2025 +0000
btrfs: use bool type for btrfs_path members used as booleans
Many fields of struct btrfs_path are used as booleans but their type is
an unsigned int (of one 1 bit width to save space). Change the type to
bool keeping the :1 suffix so that they combine with the previous u8
fields in order to save space. This makes the code more clear by using
explicit true/false and more in line with the preferred style, preserving
the size of the structure.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c
index eff2d388a706..78da47a3d00e 100644
--- a/fs/btrfs/backref.c
+++ b/fs/btrfs/backref.c
@@ -1408,12 +1408,12 @@ static int find_parent_nodes(struct btrfs_backref_walk_ctx *ctx,
if (!path)
return -ENOMEM;
if (!ctx->trans) {
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
}
if (ctx->time_seq == BTRFS_SEQ_LAST)
- path->skip_locking = 1;
+ path->skip_locking = true;
again:
head = NULL;
@@ -1560,7 +1560,7 @@ static int find_parent_nodes(struct btrfs_backref_walk_ctx *ctx,
btrfs_release_path(path);
- ret = add_missing_keys(ctx->fs_info, &preftrees, path->skip_locking == 0);
+ ret = add_missing_keys(ctx->fs_info, &preftrees, !path->skip_locking);
if (ret)
goto out;
@@ -2825,8 +2825,8 @@ struct btrfs_backref_iter *btrfs_backref_iter_alloc(struct btrfs_fs_info *fs_inf
}
/* Current backref iterator only supports iteration in commit root */
- ret->path->search_commit_root = 1;
- ret->path->skip_locking = 1;
+ ret->path->search_commit_root = true;
+ ret->path->skip_locking = true;
ret->fs_info = fs_info;
return ret;
@@ -3299,8 +3299,8 @@ static int handle_indirect_tree_backref(struct btrfs_trans_handle *trans,
level = cur->level + 1;
/* Search the tree to find parent blocks referring to the block */
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
path->lowest_level = level;
ret = btrfs_search_slot(NULL, root, tree_key, path, 0, 0);
path->lowest_level = 0;
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index b964eacc1610..ebbf04501782 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -613,8 +613,8 @@ static int sample_block_group_extent_item(struct btrfs_caching_control *caching_
extent_root = btrfs_extent_root(fs_info, max_t(u64, block_group->start,
BTRFS_SUPER_INFO_OFFSET));
- path->skip_locking = 1;
- path->search_commit_root = 1;
+ path->skip_locking = true;
+ path->search_commit_root = true;
path->reada = READA_FORWARD;
search_offset = index * div_u64(block_group->length, max_index);
@@ -744,8 +744,8 @@ static int load_extent_tree_free(struct btrfs_caching_control *caching_ctl)
* root to add free space. So we skip locking and search the commit
* root, since its read-only
*/
- path->skip_locking = 1;
- path->search_commit_root = 1;
+ path->skip_locking = true;
+ path->search_commit_root = true;
path->reada = READA_FORWARD;
key.objectid = last;
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 46262939e873..51dc8e0bc9c1 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -1709,9 +1709,9 @@ static struct extent_buffer *btrfs_search_slot_get_root(struct btrfs_root *root,
level = btrfs_header_level(b);
/*
* Ensure that all callers have set skip_locking when
- * p->search_commit_root = 1.
+ * p->search_commit_root is true.
*/
- ASSERT(p->skip_locking == 1);
+ ASSERT(p->skip_locking);
goto out;
}
@@ -3860,10 +3860,10 @@ static noinline int setup_leaf_for_split(struct btrfs_trans_handle *trans,
}
btrfs_release_path(path);
- path->keep_locks = 1;
- path->search_for_split = 1;
+ path->keep_locks = true;
+ path->search_for_split = true;
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
- path->search_for_split = 0;
+ path->search_for_split = false;
if (ret > 0)
ret = -EAGAIN;
if (ret < 0)
@@ -3890,11 +3890,11 @@ static noinline int setup_leaf_for_split(struct btrfs_trans_handle *trans,
if (ret)
goto err;
- path->keep_locks = 0;
+ path->keep_locks = false;
btrfs_unlock_up_safe(path, 1);
return 0;
err:
- path->keep_locks = 0;
+ path->keep_locks = false;
return ret;
}
@@ -4610,11 +4610,11 @@ int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key,
u32 nritems;
int level;
int ret = 1;
- int keep_locks = path->keep_locks;
+ const bool keep_locks = path->keep_locks;
ASSERT(!path->nowait);
ASSERT(path->lowest_level == 0);
- path->keep_locks = 1;
+ path->keep_locks = true;
again:
cur = btrfs_read_lock_root_node(root);
level = btrfs_header_level(cur);
@@ -4704,7 +4704,7 @@ int btrfs_search_forward(struct btrfs_root *root, struct btrfs_key *min_key,
* 0 is returned if another key is found, < 0 if there are any errors
* and 1 is returned if there are no higher keys in the tree
*
- * path->keep_locks should be set to 1 on the search made before
+ * path->keep_locks should be set to true on the search made before
* calling this function.
*/
int btrfs_find_next_key(struct btrfs_root *root, struct btrfs_path *path,
@@ -4803,13 +4803,13 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path,
next = NULL;
btrfs_release_path(path);
- path->keep_locks = 1;
+ path->keep_locks = true;
if (time_seq) {
ret = btrfs_search_old_slot(root, &key, path, time_seq);
} else {
if (path->need_commit_sem) {
- path->need_commit_sem = 0;
+ path->need_commit_sem = false;
need_commit_sem = true;
if (path->nowait) {
if (!down_read_trylock(&fs_info->commit_root_sem)) {
@@ -4822,7 +4822,7 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path,
}
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
}
- path->keep_locks = 0;
+ path->keep_locks = false;
if (ret < 0)
goto done;
@@ -4961,7 +4961,7 @@ int btrfs_next_old_leaf(struct btrfs_root *root, struct btrfs_path *path,
if (need_commit_sem) {
int ret2;
- path->need_commit_sem = 1;
+ path->need_commit_sem = true;
ret2 = finish_need_commit_sem_search(path);
up_read(&fs_info->commit_root_sem);
if (ret2)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 16dd11c48531..692370fc07b2 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -65,21 +65,21 @@ struct btrfs_path {
* set by btrfs_split_item, tells search_slot to keep all locks
* and to force calls to keep space in the nodes
*/
- unsigned int search_for_split:1;
+ bool search_for_split:1;
/* Keep some upper locks as we walk down. */
- unsigned int keep_locks:1;
- unsigned int skip_locking:1;
- unsigned int search_commit_root:1;
- unsigned int need_commit_sem:1;
- unsigned int skip_release_on_error:1;
+ bool keep_locks:1;
+ bool skip_locking:1;
+ bool search_commit_root:1;
+ bool need_commit_sem:1;
+ bool skip_release_on_error:1;
/*
* Indicate that new item (btrfs_search_slot) is extending already
* existing item and ins_len contains only the data size and not item
* header (ie. sizeof(struct btrfs_item) is not included).
*/
- unsigned int search_for_extension:1;
+ bool search_for_extension:1;
/* Stop search if any locks need to be taken (for read) */
- unsigned int nowait:1;
+ bool nowait:1;
};
#define BTRFS_PATH_AUTO_FREE(path_name) \
diff --git a/fs/btrfs/defrag.c b/fs/btrfs/defrag.c
index a4cc1bc63562..2e3c011d410a 100644
--- a/fs/btrfs/defrag.c
+++ b/fs/btrfs/defrag.c
@@ -472,7 +472,7 @@ static int btrfs_defrag_leaves(struct btrfs_trans_handle *trans,
memcpy(&key, &root->defrag_progress, sizeof(key));
}
- path->keep_locks = 1;
+ path->keep_locks = true;
ret = btrfs_search_forward(root, &key, path, BTRFS_OLDEST_GENERATION);
if (ret < 0)
@@ -515,7 +515,7 @@ static int btrfs_defrag_leaves(struct btrfs_trans_handle *trans,
/*
* Now that we reallocated the node we can find the next key. Note that
* btrfs_find_next_key() can release our path and do another search
- * without COWing, this is because even with path->keep_locks = 1,
+ * without COWing, this is because even with path->keep_locks == true,
* btrfs_search_slot() / ctree.c:unlock_up() does not keeps a lock on a
* node when path->slots[node_level - 1] does not point to the last
* item or a slot beyond the last item (ctree.c:unlock_up()). Therefore
diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index a4eaef60549e..b6c7da8e1bc8 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -489,8 +489,8 @@ static int mark_block_group_to_copy(struct btrfs_fs_info *fs_info,
}
path->reada = READA_FORWARD;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
key.objectid = src_dev->devid;
key.type = BTRFS_DEV_EXTENT_KEY;
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 86004b8daa96..819e0a15e8e7 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -789,7 +789,7 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans,
want = extent_ref_type(parent, owner);
if (insert) {
extra_size = btrfs_extent_inline_ref_size(want);
- path->search_for_extension = 1;
+ path->search_for_extension = true;
} else
extra_size = -1;
@@ -955,7 +955,7 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans,
if (!path->keep_locks) {
btrfs_release_path(path);
- path->keep_locks = 1;
+ path->keep_locks = true;
goto again;
}
@@ -976,11 +976,11 @@ int lookup_inline_extent_backref(struct btrfs_trans_handle *trans,
*ref_ret = (struct btrfs_extent_inline_ref *)ptr;
out:
if (path->keep_locks) {
- path->keep_locks = 0;
+ path->keep_locks = false;
btrfs_unlock_up_safe(path, 1);
}
if (insert)
- path->search_for_extension = 0;
+ path->search_for_extension = false;
return ret;
}
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
index e7c219e83ff0..b17632ea085f 100644
--- a/fs/btrfs/file-item.c
+++ b/fs/btrfs/file-item.c
@@ -394,8 +394,8 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio)
* between reading the free space cache and updating the csum tree.
*/
if (btrfs_is_free_space_inode(inode)) {
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
}
/*
@@ -423,8 +423,8 @@ int btrfs_lookup_bio_sums(struct btrfs_bio *bbio)
* from across transactions.
*/
if (bbio->csum_search_commit_root) {
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
down_read(&fs_info->commit_root_sem);
}
@@ -1177,10 +1177,10 @@ int btrfs_csum_file_blocks(struct btrfs_trans_handle *trans,
}
btrfs_release_path(path);
- path->search_for_extension = 1;
+ path->search_for_extension = true;
ret = btrfs_search_slot(trans, root, &file_key, path,
csum_size, 1);
- path->search_for_extension = 0;
+ path->search_for_extension = false;
if (ret < 0)
goto out;
diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index 6ccb492eae8e..f0f72850fab2 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
@@ -968,8 +968,8 @@ int load_free_space_cache(struct btrfs_block_group *block_group)
path = btrfs_alloc_path();
if (!path)
return 0;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
/*
* We must pass a path with search_commit_root set to btrfs_iget in
diff --git a/fs/btrfs/free-space-tree.c b/fs/btrfs/free-space-tree.c
index 26eae347739f..47745ae23c7d 100644
--- a/fs/btrfs/free-space-tree.c
+++ b/fs/btrfs/free-space-tree.c
@@ -1694,8 +1694,8 @@ int btrfs_load_free_space_tree(struct btrfs_caching_control *caching_ctl)
* Just like caching_thread() doesn't want to deadlock on the extent
* tree, we don't want to deadlock on the free space tree.
*/
- path->skip_locking = 1;
- path->search_commit_root = 1;
+ path->skip_locking = true;
+ path->search_commit_root = true;
path->reada = READA_FORWARD;
info = btrfs_search_free_space_info(NULL, block_group, path, 0);
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index 1bd73b80f9fa..98dacfd03234 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -312,7 +312,7 @@ int btrfs_insert_inode_ref(struct btrfs_trans_handle *trans,
if (!path)
return -ENOMEM;
- path->skip_release_on_error = 1;
+ path->skip_release_on_error = true;
ret = btrfs_insert_empty_item(trans, root, path, &key,
ins_len);
if (ret == -EEXIST) {
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 1a0c380ef464..fc0f0c46ab22 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -7111,8 +7111,8 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
* point the commit_root has everything we need.
*/
if (btrfs_is_free_space_inode(inode)) {
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
}
ret = btrfs_lookup_file_extent(NULL, root, path, objectid, start, 0);
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 1956e4bf2302..58fb55644be5 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -3834,8 +3834,8 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work *work)
* Rescan should only search for commit root, and any later difference
* should be recorded by qgroup
*/
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
while (!ret && !(stopped = rescan_should_stop(fs_info))) {
trans = btrfs_start_transaction(fs_info->fs_root, 0);
diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c
index f5c616115254..2987cb7c686e 100644
--- a/fs/btrfs/raid-stripe-tree.c
+++ b/fs/btrfs/raid-stripe-tree.c
@@ -388,8 +388,8 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info,
return -ENOMEM;
if (stripe->rst_search_commit_root) {
- path->skip_locking = 1;
- path->search_commit_root = 1;
+ path->skip_locking = true;
+ path->search_commit_root = true;
}
ret = btrfs_search_slot(NULL, stripe_root, &stripe_key, path, 0, 0);
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 739fca944296..5bfefc3e9c06 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -3165,8 +3165,8 @@ static int __add_tree_block(struct reloc_control *rc,
key.offset = blocksize;
}
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
ret = btrfs_search_slot(NULL, rc->extent_root, &key, path, 0, 0);
if (ret < 0)
return ret;
@@ -3358,8 +3358,8 @@ int find_next_extent(struct reloc_control *rc, struct btrfs_path *path,
key.type = BTRFS_EXTENT_ITEM_KEY;
key.offset = 0;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
ret = btrfs_search_slot(NULL, rc->extent_root, &key, path,
0, 0);
if (ret < 0)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 7e521d21ad40..f6c2196322c6 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -463,10 +463,10 @@ static noinline_for_stack struct scrub_ctx *scrub_setup_ctx(
refcount_set(&sctx->refs, 1);
sctx->is_dev_replace = is_dev_replace;
sctx->fs_info = fs_info;
- sctx->extent_path.search_commit_root = 1;
- sctx->extent_path.skip_locking = 1;
- sctx->csum_path.search_commit_root = 1;
- sctx->csum_path.skip_locking = 1;
+ sctx->extent_path.search_commit_root = true;
+ sctx->extent_path.skip_locking = true;
+ sctx->csum_path.search_commit_root = true;
+ sctx->csum_path.skip_locking = true;
for (i = 0; i < SCRUB_TOTAL_STRIPES; i++) {
int ret;
@@ -2202,10 +2202,10 @@ static int scrub_raid56_parity_stripe(struct scrub_ctx *sctx,
* as the data stripe bytenr may be smaller than previous extent. Thus
* we have to use our own extent/csum paths.
*/
- extent_path.search_commit_root = 1;
- extent_path.skip_locking = 1;
- csum_path.search_commit_root = 1;
- csum_path.skip_locking = 1;
+ extent_path.search_commit_root = true;
+ extent_path.skip_locking = true;
+ csum_path.search_commit_root = true;
+ csum_path.skip_locking = true;
for (int i = 0; i < data_stripes; i++) {
int stripe_index;
@@ -2688,8 +2688,8 @@ int scrub_enumerate_chunks(struct scrub_ctx *sctx,
return -ENOMEM;
path->reada = READA_FORWARD;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
key.objectid = scrub_dev->devid;
key.type = BTRFS_DEV_EXTENT_KEY;
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index fa94105e139a..3d437024e8bc 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -633,9 +633,9 @@ static struct btrfs_path *alloc_path_for_send(void)
path = btrfs_alloc_path();
if (!path)
return NULL;
- path->search_commit_root = 1;
- path->skip_locking = 1;
- path->need_commit_sem = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
+ path->need_commit_sem = true;
return path;
}
@@ -7622,10 +7622,10 @@ static int btrfs_compare_trees(struct btrfs_root *left_root,
goto out;
}
- left_path->search_commit_root = 1;
- left_path->skip_locking = 1;
- right_path->search_commit_root = 1;
- right_path->skip_locking = 1;
+ left_path->search_commit_root = true;
+ left_path->skip_locking = true;
+ right_path->search_commit_root = true;
+ right_path->skip_locking = true;
/*
* Strategy: Go to the first items of both trees. Then do
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index e40e1d746381..cc27f87c4904 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -602,9 +602,9 @@ static int overwrite_item(struct walk_control *wc)
insert:
btrfs_release_path(wc->subvol_path);
/* try to insert the key into the destination tree */
- wc->subvol_path->skip_release_on_error = 1;
+ wc->subvol_path->skip_release_on_error = true;
ret = btrfs_insert_empty_item(trans, root, wc->subvol_path, &wc->log_key, item_size);
- wc->subvol_path->skip_release_on_error = 0;
+ wc->subvol_path->skip_release_on_error = false;
dst_eb = wc->subvol_path->nodes[0];
dst_slot = wc->subvol_path->slots[0];
@@ -5706,8 +5706,8 @@ static int btrfs_check_ref_name_override(struct extent_buffer *eb,
search_path = btrfs_alloc_path();
if (!search_path)
return -ENOMEM;
- search_path->search_commit_root = 1;
- search_path->skip_locking = 1;
+ search_path->search_commit_root = true;
+ search_path->skip_locking = true;
while (cur_offset < item_size) {
u64 parent;
@@ -6026,8 +6026,8 @@ static int conflicting_inode_is_dir(struct btrfs_root *root, u64 ino,
key.type = BTRFS_INODE_ITEM_KEY;
key.offset = 0;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (WARN_ON_ONCE(ret > 0)) {
@@ -6047,8 +6047,8 @@ static int conflicting_inode_is_dir(struct btrfs_root *root, u64 ino,
}
btrfs_release_path(path);
- path->search_commit_root = 0;
- path->skip_locking = 0;
+ path->search_commit_root = false;
+ path->skip_locking = false;
return ret;
}
@@ -7169,8 +7169,8 @@ static int btrfs_log_all_parents(struct btrfs_trans_handle *trans,
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
- path->skip_locking = 1;
- path->search_commit_root = 1;
+ path->skip_locking = true;
+ path->search_commit_root = true;
key.objectid = ino;
key.type = BTRFS_INODE_REF_KEY;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 75a34ed95c74..e6a3f3ceb74b 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1710,8 +1710,8 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
}
path->reada = READA_FORWARD;
- path->search_commit_root = 1;
- path->skip_locking = 1;
+ path->search_commit_root = true;
+ path->skip_locking = true;
key.objectid = device->devid;
key.type = BTRFS_DEV_EXTENT_KEY;
@@ -7448,7 +7448,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)
* chunk tree, to keep it simple, just skip locking on the chunk tree.
*/
ASSERT(!test_bit(BTRFS_FS_OPEN, &fs_info->flags));
- path->skip_locking = 1;
+ path->skip_locking = true;
/*
* Read all device items, and then all the chunk items. All
diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c
index 3d27eb1e2f74..98d6aa3b7d6a 100644
--- a/fs/btrfs/xattr.c
+++ b/fs/btrfs/xattr.c
@@ -85,7 +85,7 @@ int btrfs_setxattr(struct btrfs_trans_handle *trans, struct inode *inode,
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
- path->skip_release_on_error = 1;
+ path->skip_release_on_error = true;
if (!value) {
di = btrfs_lookup_xattr(trans, root, path,
]
|
| 118/2 |
2026/08/08 03:40 |
llm |
debugger |
0m
Model:
Tokens:
input: 143954
output: 45
thoughts: 76
Thoughts: Okay, here's my take on those thoughts, framed as my own:
**Investigating a Btrfs Bug**
Right, so the value changed for this variable, `d7fe41044b3ac` – I see it went from being `1` to `true`. That's a potentially significant change, and I need to understand why. I'm curious when this `search_commit_root = 1` first entered the code base. The straightforward approach here is a grep within the relevant `fs/btrfs/volumes.c` source file. That should quickly reveal the introduction point and surrounding context.
|
| 119/2 |
2026/08/08 03:40 |
tool |
git-log |
4m
Args: map[CodeRegexp:search_commit_root = 1 Count:10 SourcePath:fs/btrfs/volumes.c]
Results: map[Output:d7fe41044b3a btrfs: use bool type for btrfs_path members used as booleans
f07b855c56b1 btrfs: try to search for data csums in commit root
119474bdbac0 btrfs: remove NULL transaction support for btrfs_lookup_extent_info()
afcb80624f5b btrfs: remove search_commit parameter from btrfs_lookup_csums_list()
9acaa64187f9 btrfs: scrub: implement raid stripe tree support
3c771c194402 btrfs: scrub: avoid unnecessary csum tree search preparing stripes
1dc4888e725d btrfs: scrub: avoid unnecessary extent tree search preparing stripes
17353a344792 btrfs: scrub: remove unused btrfs_path in scrub_simple_mirror()
5dc96f8d5de9 btrfs: scrub: remove scrub_parity structure
b979547513ff btrfs: scrub: introduce helper to find and fill sector info for a scrub_stripe
]
|
| 120/2 |
2026/08/08 03:44 |
llm |
debugger |
0m
Model:
Tokens:
input: 144507
output: 39
thoughts: 63
Thoughts: Here's my summary, as the thought process unfolds:
**Digging into that `find_free_dev_extent` Issue**
Okay, wait a minute... I was *certain* that `search_commit_root = 1` had been introduced within `find_free_dev_extent` itself during one of these recent commits. But the diffs don't seem to bear that out. I need to be absolutely sure. This is where I start using my intuition and experience to trace changes. No, no it wasn't! Right, better double-check the history of that function to understand where this flag came from. Let's get the definitive answer using `git blame`. I'll run it specifically on the `find_free_dev_extent` function within `fs/btrfs/volumes.c`. That should give me the precise context for how and when that flag first appeared in relation to this function. Then I'll have a clear line of causality and know how to proceed.
|
| 121/2 |
2026/08/08 03:44 |
tool |
git-blame |
0m
Args: map[End:1855 File:fs/btrfs/volumes.c Start:1845]
Results: map[Output:401e29c124eac 1845) test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
7bfc837df935d 1846) ret = -ENOSPC;
6df9a95e63395 1847) goto out;
7bfc837df935d 1848) }
7bfc837df935d 1849)
e4058b54d1e44 1850) path->reada = READA_FORWARD;
d7fe41044b3ac 1851) path->search_commit_root = true;
d7fe41044b3ac 1852) path->skip_locking = true;
7bfc837df935d 1853)
0b86a832a1f38 1854) key.objectid = device->devid;
0b86a832a1f38 1855) key.type = BTRFS_DEV_EXTENT_KEY;
]
|
| 122/2 |
2026/08/08 03:45 |
llm |
debugger |
0m
Model:
Tokens:
input: 145149
output: 32
thoughts: 72
Thoughts: Here's my summary, as I understand the situation:
**Tracing a Logic Change**
Okay, so I'm looking at a pretty specific issue here. Something in the system that's likely related to storage or extent management. The value of `path->search_commit_root` just flipped to `true`. And I know, from the logs or perhaps some prior analysis, that before this change, it was set to `1`. That's a very specific transition, which makes it easier to track down the root cause.
My immediate thought is, I need to figure out *which* commit, specifically, introduced the line `path->search_commit_root = 1` within the `find_free_dev_extent` function. That's my target. Finding that specific commit, let's say a commit that introduced the initial change that later became `1`, is key to understanding the logic change and, hopefully, the reason for this unexpected `true` value. I'm going to start digging into the version control history, focusing on that function and that specific variable, looking for the commit that first set things in motion.
|
| 123/2 |
2026/08/08 03:45 |
tool |
grepper |
0m
Error:
Args: map[Expression:path->search_commit_root = 1.*find_free_dev_extent]
Results:
|
| 124/2 |
2026/08/08 03:45 |
llm |
debugger |
0m
Model:
Tokens:
input: 145507
output: 44
thoughts: 3398
Thoughts: **Refining the Chunk Removal Process in Btrfs**
Okay, so I've been digging into this `btrfs_remove_chunk` function, and I think I've spotted a crucial area for improvement. My goal here is to make this operation more robust in the face of potential space reservation failures, and ideally, fail *gracefully*.
My initial thought was to use `git log -p fs/btrfs/volumes.c` and search for `search_commit_root` in `find_free_dev_extent`, but that doesn't matter here. The key is that the code intentionally searches the commit root to avoid reusing freed space in the current transaction. And if we can't get that space, we MUST fail cleanly.
My first thought was to add some code:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
/*
* If we failed to reserve system space, we can fail gracefully.
* We must do this before removing the device extents, because if we
* remove the device extents and then fail to remove the chunk item,
* we would have to abort the transaction.
*/
/* Wait, check_system_chunk doesn't return an error. How to check? */
```
Then I started to think, how do I actually check if `check_system_chunk` fails? I thought of checking `trans->chunk_bytes_reserved`. But I needed to know what the expected reservation was. That lead me to:
```c
u64 expected_reserve = btrfs_calc_metadata_size(fs_info, get_profile_num_devs(fs_info, map->type)) +
btrfs_calc_insert_metadata_size(fs_info, 1);
```
Then, another thought: `check_system_chunk` only reserves if `left < bytes`! If it succeeds, it reserves. If it fails it DOES NOT reserve!
So, to check if it reserves, check if `trans->chunk_bytes_reserved` increases:
```c
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
/* Failed to reserve system space, fail gracefully */
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = -ENOSPC;
goto out;
}
```
Okay, so my thinking went on to make sure this check is perfectly robust. Can `bytes` be zero? Nope. Can `trans->chunk_bytes_reserved` increase by less than `bytes`? Nope, it increments by exactly `bytes`. Can `check_system_chunk` succeed and `btrfs_block_rsv_add` fail? Yes, if `left >= bytes` and that SHOULD NOT happen! If it does, we fail gracefully!
The core change:
```c
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3488,11 +3488,20 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
* safe by just acquiring the chunk_mutex.
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ ret = -ENOSPC;
+ goto out;
+ }
```
Now, I realized something: *order matters*. The `check_system_chunk` call needs to happen *before* `btrfs_remove_dev_extents`. Right now it happens after! We need to move it:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->
|
| 125/2 |
2026/08/08 03:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/transaction.c Name:btrfs_trans_release_chunk_metadata]
Results: map[SourceCode: 248: /*
249: * To be called after doing the chunk btree updates right after allocating a new
250: * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
251: * chunk after all chunk btree updates and after finishing the second phase of
252: * chunk allocation (btrfs_create_pending_block_groups()) in case some block
253: * group had its chunk item insertion delayed to the second phase.
254: */
255: void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
256: {
257: struct btrfs_fs_info *fs_info = trans->fs_info;
258:
259: if (!trans->chunk_bytes_reserved)
260: return;
261:
262: btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
263: trans->chunk_bytes_reserved, NULL);
264: trans->chunk_bytes_reserved = 0;
265: }
SourceFile:fs/btrfs/transaction.c]
|
| 126/2 |
2026/08/08 03:45 |
llm |
debugger |
0m
Model:
Tokens:
input: 150327
output: 50
thoughts: 1027
Thoughts: **Analyzing Chunk Removal and System Space Reservation in Btrfs**
Okay, so I've been digging into this `btrfs_remove_chunk` function, trying to understand a potential transaction abort issue. My initial thought was about releasing `chunk_bytes_reserved`. The code is supposed to release *all* reserved bytes, right? I started by confirming that the transaction initiated by `btrfs_relocate_chunk_finish`, which calls `btrfs_remove_chunk`, is a *new* transaction dedicated solely to the chunk removal. This means the transaction's `chunk_bytes_reserved` only contains reservations for this specific operation and nothing else.
So, calling `btrfs_trans_release_chunk_metadata` *should* be safe after chunk removal in this dedicated transaction. But, wait, I realized something. If `btrfs_remove_dev_extents` errors, it *aborts* the transaction. The transaction cleanup will automatically release the space through `btrfs_end_transaction`, so calling `btrfs_trans_release_chunk_metadata` on an error path is actually redundant.
Now, let's consider another edge case: the failure of `check_system_chunk` to reserve space. If `check_system_chunk` fails, `trans->chunk_bytes_reserved` won't increase. In this case, we simply return `-ENOSPC`, because there's nothing that needs releasing. No reservation happened.
The solution is clear: **move `check_system_chunk` *before* `btrfs_remove_dev_extents` and check for its success.** Then, if `check_system_chunk` fails, we can gracefully return `-ENOSPC` *before* the device tree is modified. That's a critical step to preventing data corruption.
So let's break down the *root cause* and the *fix* based on my investigation.
**Root Cause:**
1. `btrfs_remove_chunk` removes device extents from the device tree via `btrfs_remove_dev_extents`.
2. Then it tries to reserve system space for removing the chunk item via `check_system_chunk`. `check_system_chunk` can attempt to allocate a new system chunk if the system space is full.
3. `find_free_dev_extent` in `check_system_chunk` searches the commit root of the device tree to avoid reusing space freed in the current transaction. This is problematic, because since we just removed the device extents, the freed space is not visible.
4. If the device is full, `find_free_dev_extent` returns `-ENOSPC`, and `check_system_chunk` fails to allocate a system chunk, but ignores this error and reserves zero space.
5. Then, `remove_chunk_item` fails with `-ENOSPC` because the reservation was unsuccessful.
6. `btrfs_remove_chunk` catches this `-ENOSPC` and retries, failing again.
7. Since `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully. It must abort the transaction to prevent filesystem corruption.
**Fix:**
1. Move the `check_system_chunk` call *before* `btrfs_remove_dev_extents`.
2. Check whether `check_system_chunk` has successfully reserved space using something like a check of `trans->chunk_bytes_reserved` *before* and *after* the call.
3. If the check shows that the reservation failed, gracefully return `-ENOSPC` *before* modifying any data structures.
This change allows the balance operation to fail with `-ENOSPC` rather than forcing a transaction abort and possible data loss.
Now, I'm checking if there are other places where `check_system_chunk` is called and errors are ignored, leading to transaction aborts. Let's look at `do_chunk_alloc`.
In `do_chunk_alloc`:
* `check_system_chunk(trans, flags);`
* `bg = btrfs_alloc_chunk(trans, flags);`
If `check_system_chunk` fails, `btrfs_alloc_chunk` is then called. If `btrfs_alloc_chunk` *also* fails, it returns `-ENOSPC`, and the error propagates up the call stack to `btrfs_chunk_alloc` and eventually to the calling function, which *handles* the `-ENOSPC` error gracefully. No transaction abort happens here. The caller receives `-ENOSPC`
|
| 127/2 |
2026/08/08 03:45 |
tool |
codeexpert |
2m
Args: map[Question:Is it safe to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`?]
Results: map[Answer:No, it is **not safe** to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`. Doing so would introduce severe deadlock risks, spurious transaction aborts, and racy space reservations.
Here is a detailed breakdown of why this order is critical:
### 1. Deadlock on `fs_info->chunk_mutex`
* **Lock Requirement**: `check_system_chunk` requires holding `fs_info->chunk_mutex` (asserted by `lockdep_assert_held(&fs_info->chunk_mutex)` in `reserve_chunk_space`).
* **The Deadlock Path**: If we hold `chunk_mutex` while calling `btrfs_remove_dev_extents`, we can deadlock. `btrfs_remove_dev_extents` deletes device extent items from the device tree (`fs_info->dev_root`), which is a regular btree. Deleting items from this btree requires COWing extent buffers. If the metadata space is low, COWing these buffers can trigger a metadata chunk allocation (`btrfs_chunk_alloc`), which will attempt to acquire `fs_info->chunk_mutex`. Since the current thread already holds `chunk_mutex`, this results in a deadlock.
### 2. Spurious `-ENOSPC` and Transaction Aborts via `trans->removing_chunk`
* To prevent concurrent tasks from deadlocking on `chunk_mutex` during chunk removal, `btrfs_remove_chunk` sets `trans->removing_chunk = true` while holding `chunk_mutex`.
* If `trans->removing_chunk` is set to `true` before `btrfs_remove_dev_extents` is called, any metadata chunk allocation triggered during `btrfs_remove_dev_extents` (due to COWing device tree buffers) will immediately fail. This is because `btrfs_chunk_alloc` returns `-ENOSPC` when `trans->removing_chunk` is `true`. This would lead to premature and spurious `-ENOSPC` errors and transaction aborts.
### 3. Ineffective and Racy System Space Reservation
* If we do not hold `chunk_mutex` across `btrfs_remove_dev_extents` but instead acquire `chunk_mutex`, call `check_system_chunk`, release `chunk_mutex`, and then call `btrfs_remove_dev_extents`, the reservation is no longer safe:
1. Calling `check_system_chunk` without holding `chunk_mutex` violates lockdep assertions and is unsafe because it can allocate a system chunk, which itself modifies the chunk btree.
2. During the execution of `btrfs_remove_dev_extents` (which can be slow as it deletes device extents for all stripes), other concurrent tasks can allocate chunks and consume the reserved system space or modify the system block groups. By the time we actually modify the chunk btree in `remove_chunk_item`, the system space reservation might have been invalidated or exhausted, defeating the purpose of `check_system_chunk` (which is to guarantee enough system space for the chunk btree modifications).
3. If `btrfs_remove_dev_extents` itself triggers a chunk allocation, it would accumulate multiple system space reservations on the same transaction handle, leading to resource bloat or potential exhaustion.
Therefore, `btrfs_remove_dev_extents` must be executed first, and only after it succeeds can we set `trans->removing_chunk = true`, acquire `fs_info->chunk_mutex`, and call `check_system_chunk` to safely reserve space immediately before modifying the chunk btree.]
|
| 128/3 |
2026/08/08 03:45 |
agent |
codeexpert |
2m
Model:
Results: map[Answer:No, it is **not safe** to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`. Doing so would introduce severe deadlock risks, spurious transaction aborts, and racy space reservations.
Here is a detailed breakdown of why this order is critical:
### 1. Deadlock on `fs_info->chunk_mutex`
* **Lock Requirement**: `check_system_chunk` requires holding `fs_info->chunk_mutex` (asserted by `lockdep_assert_held(&fs_info->chunk_mutex)` in `reserve_chunk_space`).
* **The Deadlock Path**: If we hold `chunk_mutex` while calling `btrfs_remove_dev_extents`, we can deadlock. `btrfs_remove_dev_extents` deletes device extent items from the device tree (`fs_info->dev_root`), which is a regular btree. Deleting items from this btree requires COWing extent buffers. If the metadata space is low, COWing these buffers can trigger a metadata chunk allocation (`btrfs_chunk_alloc`), which will attempt to acquire `fs_info->chunk_mutex`. Since the current thread already holds `chunk_mutex`, this results in a deadlock.
### 2. Spurious `-ENOSPC` and Transaction Aborts via `trans->removing_chunk`
* To prevent concurrent tasks from deadlocking on `chunk_mutex` during chunk removal, `btrfs_remove_chunk` sets `trans->removing_chunk = true` while holding `chunk_mutex`.
* If `trans->removing_chunk` is set to `true` before `btrfs_remove_dev_extents` is called, any metadata chunk allocation triggered during `btrfs_remove_dev_extents` (due to COWing device tree buffers) will immediately fail. This is because `btrfs_chunk_alloc` returns `-ENOSPC` when `trans->removing_chunk` is `true`. This would lead to premature and spurious `-ENOSPC` errors and transaction aborts.
### 3. Ineffective and Racy System Space Reservation
* If we do not hold `chunk_mutex` across `btrfs_remove_dev_extents` but instead acquire `chunk_mutex`, call `check_system_chunk`, release `chunk_mutex`, and then call `btrfs_remove_dev_extents`, the reservation is no longer safe:
1. Calling `check_system_chunk` without holding `chunk_mutex` violates lockdep assertions and is unsafe because it can allocate a system chunk, which itself modifies the chunk btree.
2. During the execution of `btrfs_remove_dev_extents` (which can be slow as it deletes device extents for all stripes), other concurrent tasks can allocate chunks and consume the reserved system space or modify the system block groups. By the time we actually modify the chunk btree in `remove_chunk_item`, the system space reservation might have been invalidated or exhausted, defeating the purpose of `check_system_chunk` (which is to guarantee enough system space for the chunk btree modifications).
3. If `btrfs_remove_dev_extents` itself triggers a chunk allocation, it would accumulate multiple system space reservations on the same transaction handle, leading to resource bloat or potential exhaustion.
Therefore, `btrfs_remove_dev_extents` must be executed first, and only after it succeeds can we set `trans->removing_chunk = true`, acquire `fs_info->chunk_mutex`, and call `check_system_chunk` to safely reserve space immediately before modifying the chunk btree.]
Instruction:
You are a capable Linux kernel developer tasked with researching complex questions
about kernel source code. You will be given a concrete question, and need to provide
a concrete answer.
Use tools extensively while researching the question. Don't make assumptions,
or rely on your previous knowledge about the kernel source code, use available tools
to access the actual source code.
Use all available sources of information:
- kernel source code
- documentation in the Documentation dir in the source tree
- git commits descriptions, git blame
Do not guess file names or file paths and attempt to read them without
verifying their existence first using content search or directory listing tools.
If a file, symbol, or directory is not found via content search (grepper) or
directory listing (codesearch-dir-index), treat it as completely absent.
Do not attempt to guess alternative names, extensions, or directories.
Tool Selection Guidelines:
1. For C functions, structs, and variables: Prefer codesearch-definition-source
or codesearch-struct-layout FIRST to retrieve clean, exact definitions
without line-number guessing.
2. If symbol lookup fails (e.g., preprocessor macros, macro-generated code, or
disabled #ifdef branches), fall back to read-file or grepper.
3. For file headers, #include directives, preprocessor macro definitions, and
non-C files (Kconfig, Makefiles, docs): Use read-file or grepper directly.
Do NOT use git-log to search for the presence or existence of files in the
repository. git-log is only for tracing commit history of files that are
already present in the current checkout. If a file does not exist in the
current checkout, it cannot be used for reproduction.
Avoid running broad git-log queries (such as searches on the entire repo)
without a specific 'PathPrefix' to restrict the scope, as these are very
expensive and will time out.
If a git-log tool call times out, do not retry the query with the same broad
scope. You must specify a tighter, narrower 'PathPrefix' for subsequent queries.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Is it safe to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`?
|
| 129/4 |
2026/08/08 03:45 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 3903
output: 78
thoughts: 90
Thoughts: Okay, here's my summary, framed as my internal thought process:
**Analyzing the Potential Move: `check_system_chunk` and `btrfs_remove_dev_extents`**
Hmm, the user is wondering about reordering some code within `btrfs_remove_chunk`. Specifically, they're questioning whether it's safe to move the call to `check_system_chunk` *before* the call to `btrfs_remove_dev_extents`. That's an interesting question, and one that deserves a proper investigation.
My initial approach is always to get a firm grasp of the context. First things first: I need to refresh my memory on what these functions *do* and how they interact. I'll need to see the actual code. So, I need to find the definitions and sources for both `btrfs_remove_chunk` and `check_system_chunk` within the btrfs codebase. A quick search should do the trick. I'll probably reach for a tool like `codesearch-definition-source` or maybe `grepper` – whatever's quickest and most reliable for the current environment. That'll let me get a good look at the code paths and dependencies before I make any judgements. I need to understand the impact of the call order.
|
| 130/4 |
2026/08/08 03:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_chunk]
Results: map[SourceCode:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
3553: }
3554: }
3555:
3556: btrfs_update_per_profile_avail(fs_info);
3557: mutex_unlock(&fs_info->chunk_mutex);
3558: trans->removing_chunk = false;
3559:
3560: /*
3561: * We are done with chunk btree updates and deletions, so release the
3562: * system space we previously reserved (with check_system_chunk()).
3563: */
3564: btrfs_trans_release_chunk_metadata(trans);
3565:
3566: /* On error, btrfs_remove_block_group() aborts the transaction. */
3567: ret = btrfs_remove_block_group(trans, map);
3568: if (unlikely(ret))
3569: ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
3570:
3571: out:
3572: if (trans->removing_chunk) {
3573: mutex_unlock(&fs_info->chunk_mutex);
3574: trans->removing_chunk = false;
3575: }
3576: /* once for us */
3577: btrfs_free_chunk_map(map);
3578: return ret;
3579: }
SourceFile:fs/btrfs/volumes.c]
|
| 131/4 |
2026/08/08 03:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 132/4 |
2026/08/08 03:45 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 6770
output: 43
thoughts: 45
Thoughts: Okay, let's dive into this code.
**Analyzing `check_system_chunk` and `btrfs_remove_dev_extents`**
My initial focus will be on the interaction between `check_system_chunk` and `btrfs_remove_dev_extents`. I need to understand how they relate and what data or functionality is passed between them. It’s clear they're related, but the specific relationship needs investigation. Right now, I want to fully understand `btrfs_remove_dev_extents`. Let me see what that particular function is actually responsible for. What are its inputs? What's its goal? What side effects does it have? I'll probably start by looking at its implementation and any relevant documentation. From there I'll better understand the larger picture.
|
| 133/4 |
2026/08/08 03:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 134/4 |
2026/08/08 03:45 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 7901
output: 40
thoughts: 211
Thoughts: **Analyzing the Locking in `btrfs_remove_chunk`**
Okay, let's break down the locking and order of operations within the `btrfs_remove_chunk` function. First, we have the initial removal of device extents. Seems straightforward. But then, we see this line: `trans->removing_chunk = true;` followed by `mutex_lock(&fs_info->chunk_mutex);`.
Hold on. Looking further down, `check_system_chunk(trans, map->type)` is called. And what's this? The documentation explicitly states that `check_system_chunk` *requires* holding `fs_info->chunk_mutex`. Now, why is that? We need to understand the relationship. It is crucial to examine why `check_system_chunk` needs to be protected by this mutex. It makes sense that this function must be in sequence with the removal because the comment states that this mutex is to be held while reserving or removing a chunk. Time to dive into `reserve_chunk_space`.
|
| 135/4 |
2026/08/08 03:45 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 136/4 |
2026/08/08 03:45 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 9639
output: 34
thoughts: 913
Thoughts: **Analyzing a Potential Deadlock in Btrfs Device Removal**
Okay, so I'm looking at `reserve_chunk_space` and it's asserting that the `chunk_mutex` is held. The immediate thought is, if I move the call to `check_system_chunk` before `btrfs_remove_dev_extents`, I'd need to hold that mutex *before* calling the latter. But, if I call `check_system_chunk` before `btrfs_remove_dev_extents`, I'm going to need to acquire the `chunk_mutex`.
Aha! Let's revisit `btrfs_remove_dev_extents`: There's a critical comment right there! It explicitly states: "Note that here we do not acquire the chunk_mutex because that can result in a deadlock when deleting the device extent items from the devices btree - COWing an extent buffer from the btree may result in allocating a new metadata chunk, which would attempt to lock again fs_info->chunk_mutex."
Right. Let's trace this carefully: if I hold `chunk_mutex` (either because I've called `check_system_chunk` first *or* because I'm calling it before `btrfs_remove_dev_extents`), then:
1. I'm holding `chunk_mutex`.
2. I call `btrfs_remove_dev_extents`.
3. Inside `btrfs_remove_dev_extents`, I delete device extent items from the devices btree using `btrfs_free_dev_extent`.
4. This deletion from the devices btree *might* require COWing an extent buffer, a common Btrfs operation.
5. COWing an extent buffer could then trigger a metadata chunk allocation if we're short on space.
6. That chunk allocation, potentially via `btrfs_chunk_alloc` or similar, will *itself* attempt to acquire `chunk_mutex`.
7. Boom! Deadlock, because I already *have* the mutex!
So, what about calling `check_system_chunk` before `btrfs_remove_dev_extents` *without* holding `chunk_mutex`? But the `reserve_chunk_space` function, crucial to the functioning of `check_system_chunk`, insists on holding `chunk_mutex`. The comment explains, it's needed "because we can end up allocating a system chunk and for an atomic and race free space reservation in the chunk block reserve." If I don't hold the mutex, calling `check_system_chunk` becomes unsafe because it can allocate a system chunk (using `btrfs_create_chunk` and `btrfs_chunk_alloc_add_chunk_item`, etc.), which fundamentally alters the chunk btree and, as expected, requires the mutex. So, calling it without the mutex introduces a race/bug, and calling it *with* the mutex creates the potential deadlock!
Seems that I'm back to the initial problem. I need to figure out why this is done the way it is done. Let me look at the git log. Let me use `git-log` to check the commits for `check_system_chunk` or `btrfs_remove_dev_extents` or `btrfs_remove_chunk`.
|
| 137/4 |
2026/08/08 03:46 |
tool |
git-log |
0m
Args: map[CodeRegexp:check_system_chunk PathPrefix:fs/btrfs/]
Results: map[Output:ecd84d54674a btrfs: update comments for chunk allocation -ENOSPC cases
2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
07730d87ac78 btrfs: migrate the chunk allocation code
26ce2095e03c btrfs: migrate inc/dec_block_group_ro code
e3e0520b32bc btrfs: migrate the block group removal code
451a2c130342 btrfs: Remove fs_info from check_system_chunk
2ff7e61e0d30 btrfs: take an fs_info directly when the root is not used otherwise
7fd01182d1a1 Btrfs: fix the number of transaction units needed to remove a block group
4617ea3a52cf Btrfs: fix necessary chunk tree space calculation when allocating a chunk
]
|
| 138/4 |
2026/08/08 03:46 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 11588
output: 25
thoughts: 57
Thoughts: Okay, let's dive into this. I'm going to examine commit `2bb2e00ed978`, specifically the one titled "btrfs: fix deadlock between chunk allocation and chunk btree modifications". I need to understand this, and `git show` seems like the perfect tool to unpack the details. Let's get the full picture, line by line. I anticipate this will involve a deep dive into the btrfs internals. I'm expecting to see modifications related to locking or transaction handling within the chunk allocation and B-tree interaction. I'll be looking for the specific conditions that led to the deadlock and how the fix resolved it, likely with revised mutex usage or ordering. Hopefully, the commit message provides sufficient context, but I'm prepared to follow the code's threads to fully grasp the problem and the implemented solution. This is going to be an interesting study.
|
| 139/4 |
2026/08/08 03:46 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed Oct 13 10:12:49 2021 +0100
btrfs: fix deadlock between chunk allocation and chunk btree modifications
When a task is doing some modification to the chunk btree and it is not in
the context of a chunk allocation or a chunk removal, it can deadlock with
another task that is currently allocating a new data or metadata chunk.
These contexts are the following:
* When relocating a system chunk, when we need to COW the extent buffers
that belong to the chunk btree;
* When adding a new device (ioctl), where we need to add a new device item
to the chunk btree;
* When removing a device (ioctl), where we need to remove a device item
from the chunk btree;
* When resizing a device (ioctl), where we need to update a device item in
the chunk btree and may need to relocate a system chunk that lies beyond
the new device size when shrinking a device.
The problem happens due to a sequence of steps like the following:
1) Task A starts a data or metadata chunk allocation and it locks the
chunk mutex;
2) Task B is relocating a system chunk, and when it needs to COW an extent
buffer of the chunk btree, it has locked both that extent buffer as
well as its parent extent buffer;
3) Since there is not enough available system space, either because none
of the existing system block groups have enough free space or because
the only one with enough free space is in RO mode due to the relocation,
task B triggers a new system chunk allocation. It blocks when trying to
acquire the chunk mutex, currently held by task A;
4) Task A enters btrfs_chunk_alloc_add_chunk_item(), in order to insert
the new chunk item into the chunk btree and update the existing device
items there. But in order to do that, it has to lock the extent buffer
that task B locked at step 2, or its parent extent buffer, but task B
is waiting on the chunk mutex, which is currently locked by task A,
therefore resulting in a deadlock.
One example report when the deadlock happens with system chunk relocation:
INFO: task kworker/u9:5:546 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u9:5 state:D stack:25936 pid: 546 ppid: 2 flags:0x00004000
Workqueue: events_unbound btrfs_async_reclaim_metadata_space
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
rwsem_down_read_slowpath+0x4ee/0x9d0 kernel/locking/rwsem.c:993
__down_read_common kernel/locking/rwsem.c:1214 [inline]
__down_read kernel/locking/rwsem.c:1223 [inline]
down_read_nested+0xe6/0x440 kernel/locking/rwsem.c:1590
__btrfs_tree_read_lock+0x31/0x350 fs/btrfs/locking.c:47
btrfs_tree_read_lock fs/btrfs/locking.c:54 [inline]
btrfs_read_lock_root_node+0x8a/0x320 fs/btrfs/locking.c:191
btrfs_search_slot_get_root fs/btrfs/ctree.c:1623 [inline]
btrfs_search_slot+0x13b4/0x2140 fs/btrfs/ctree.c:1728
btrfs_update_device+0x11f/0x500 fs/btrfs/volumes.c:2794
btrfs_chunk_alloc_add_chunk_item+0x34d/0xea0 fs/btrfs/volumes.c:5504
do_chunk_alloc fs/btrfs/block-group.c:3408 [inline]
btrfs_chunk_alloc+0x84d/0xf50 fs/btrfs/block-group.c:3653
flush_space+0x54e/0xd80 fs/btrfs/space-info.c:670
btrfs_async_reclaim_metadata_space+0x396/0xa90 fs/btrfs/space-info.c:953
process_one_work+0x9df/0x16d0 kernel/workqueue.c:2297
worker_thread+0x90/0xed0 kernel/workqueue.c:2444
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
INFO: task syz-executor:9107 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor state:D stack:23200 pid: 9107 ppid: 7792 flags:0x00004004
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
__mutex_lock_common kernel/locking/mutex.c:669 [inline]
__mutex_lock+0xc96/0x1680 kernel/locking/mutex.c:729
btrfs_chunk_alloc+0x31a/0xf50 fs/btrfs/block-group.c:3631
find_free_extent_update_loop fs/btrfs/extent-tree.c:3986 [inline]
find_free_extent+0x25cb/0x3a30 fs/btrfs/extent-tree.c:4335
btrfs_reserve_extent+0x1f1/0x500 fs/btrfs/extent-tree.c:4415
btrfs_alloc_tree_block+0x203/0x1120 fs/btrfs/extent-tree.c:4813
__btrfs_cow_block+0x412/0x1620 fs/btrfs/ctree.c:415
btrfs_cow_block+0x2f6/0x8c0 fs/btrfs/ctree.c:570
btrfs_search_slot+0x1094/0x2140 fs/btrfs/ctree.c:1768
relocate_tree_block fs/btrfs/relocation.c:2694 [inline]
relocate_tree_blocks+0xf73/0x1770 fs/btrfs/relocation.c:2757
relocate_block_group+0x47e/0xc70 fs/btrfs/relocation.c:3673
btrfs_relocate_block_group+0x48a/0xc60 fs/btrfs/relocation.c:4070
btrfs_relocate_chunk+0x96/0x280 fs/btrfs/volumes.c:3181
__btrfs_balance fs/btrfs/volumes.c:3911 [inline]
btrfs_balance+0x1f03/0x3cd0 fs/btrfs/volumes.c:4301
btrfs_ioctl_balance+0x61e/0x800 fs/btrfs/ioctl.c:4137
btrfs_ioctl+0x39ea/0x7b70 fs/btrfs/ioctl.c:4949
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:874 [inline]
__se_sys_ioctl fs/ioctl.c:860 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:860
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
So fix this by making sure that whenever we try to modify the chunk btree
and we are neither in a chunk allocation context nor in a chunk remove
context, we reserve system space before modifying the chunk btree.
Reported-by: Hao Sun <sunhao.th@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/CACkBjsax51i4mu6C0C3vJqQN3NR_iVuucoeG3U1HXjrgzn5FFQ@mail.gmail.com/
Fixes: 79bd37120b1495 ("btrfs: rework chunk allocation to avoid exhaustion of the system chunk array")
CC: stable@vger.kernel.org # 5.14+
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index de9aeb3733cf..f971d043469c 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3425,25 +3425,6 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
goto out;
}
- /*
- * If this is a system chunk allocation then stop right here and do not
- * add the chunk item to the chunk btree. This is to prevent a deadlock
- * because this system chunk allocation can be triggered while COWing
- * some extent buffer of the chunk btree and while holding a lock on a
- * parent extent buffer, in which case attempting to insert the chunk
- * item (or update the device item) would result in a deadlock on that
- * parent extent buffer. In this case defer the chunk btree updates to
- * the second phase of chunk allocation and keep our reservation until
- * the second phase completes.
- *
- * This is a rare case and can only be triggered by the very few cases
- * we have where we need to touch the chunk btree outside chunk allocation
- * and chunk removal. These cases are basically adding a device, removing
- * a device or resizing a device.
- */
- if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
- return 0;
-
ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
/*
* Normally we are not expected to fail with -ENOSPC here, since we have
@@ -3576,14 +3557,14 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
* This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
* the system chunk array due to concurrent allocations") provides more details.
*
- * For allocation of system chunks, we defer the updates and insertions into the
- * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
- * if the chunk allocation is triggered while COWing an extent buffer of the
- * chunk btree, we are holding a lock on the parent of that extent buffer and
- * doing the chunk btree updates and insertions can require locking that parent.
- * This is for the very few and rare cases where we update the chunk btree that
- * are not chunk allocation or chunk removal: adding a device, removing a device
- * or resizing a device.
+ * Allocation of system chunks does not happen through this function. A task that
+ * needs to update the chunk btree (the only btree that uses system chunks), must
+ * preallocate chunk space by calling either check_system_chunk() or
+ * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
+ * metadata chunk or when removing a chunk, while the later is used before doing
+ * a modification to the chunk btree - use cases for the later are adding,
+ * removing and resizing a device as well as relocation of a system chunk.
+ * See the comment below for more details.
*
* The reservation of system space, done through check_system_chunk(), as well
* as all the updates and insertions into the chunk btree must be done while
@@ -3620,11 +3601,27 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
if (trans->allocating_chunk)
return -ENOSPC;
/*
- * If we are removing a chunk, don't re-enter or we would deadlock.
- * System space reservation and system chunk allocation is done by the
- * chunk remove operation (btrfs_remove_chunk()).
+ * Allocation of system chunks can not happen through this path, as we
+ * could end up in a deadlock if we are allocating a data or metadata
+ * chunk and there is another task modifying the chunk btree.
+ *
+ * This is because while we are holding the chunk mutex, we will attempt
+ * to add the new chunk item to the chunk btree or update an existing
+ * device item in the chunk btree, while the other task that is modifying
+ * the chunk btree is attempting to COW an extent buffer while holding a
+ * lock on it and on its parent - if the COW operation triggers a system
+ * chunk allocation, then we can deadlock because we are holding the
+ * chunk mutex and we may need to access that extent buffer or its parent
+ * in order to add the chunk item or update a device item.
+ *
+ * Tasks that want to modify the chunk tree should reserve system space
+ * before updating the chunk btree, by calling either
+ * btrfs_reserve_chunk_metadata() or check_system_chunk().
+ * It's possible that after a task reserves the space, it still ends up
+ * here - this happens in the cases described above at do_chunk_alloc().
+ * The task will have to either retry or fail.
*/
- if (trans->removing_chunk)
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
@@ -3723,17 +3720,14 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-/*
- * Reserve space in the system space for allocating or removing a chunk
- */
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+static void reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
- u64 thresh;
int ret = 0;
- u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -3746,19 +3740,13 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
- num_devs = get_profile_num_devs(fs_info, type);
-
- /* num_devs device items to update and 1 chunk item to add or remove */
- thresh = btrfs_calc_metadata_size(fs_info, num_devs) +
- btrfs_calc_insert_metadata_size(fs_info, 1);
-
- if (left < thresh && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
- left, thresh, type);
+ left, bytes, type);
btrfs_dump_space_info(fs_info, info, 0, 0);
}
- if (left < thresh) {
+ if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
@@ -3767,21 +3755,20 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
- *
- * Also, if our caller is allocating a system chunk, do not
- * attempt to insert the chunk item in the chunk btree, as we
- * could deadlock on an extent buffer since our caller may be
- * COWing an extent buffer from the chunk btree.
*/
bg = btrfs_create_chunk(trans, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
- } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ } else {
/*
* If we fail to add the chunk item here, we end up
* trying again at phase 2 of chunk allocation, at
* btrfs_create_pending_block_groups(). So ignore
- * any error here.
+ * any error here. An ENOSPC here could happen, due to
+ * the cases described at do_chunk_alloc() - the system
+ * block group we just created was just turned into RO
+ * mode by a scrub for example, or a running discard
+ * temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
@@ -3790,12 +3777,61 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (!ret) {
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
- thresh, BTRFS_RESERVE_NO_FLUSH);
+ bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
- trans->chunk_bytes_reserved += thresh;
+ trans->chunk_bytes_reserved += bytes;
}
}
+/*
+ * Reserve space in the system space for allocating or removing a chunk.
+ * The caller must be holding fs_info->chunk_mutex.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ const u64 num_devs = get_profile_num_devs(fs_info, type);
+ u64 bytes;
+
+ /* num_devs device items to update and 1 chunk item to add or remove. */
+ bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
+ btrfs_calc_insert_metadata_size(fs_info, 1);
+
+ reserve_chunk_space(trans, bytes, type);
+}
+
+/*
+ * Reserve space in the system space, if needed, for doing a modification to the
+ * chunk btree.
+ *
+ * @trans: A transaction handle.
+ * @is_item_insertion: Indicate if the modification is for inserting a new item
+ * in the chunk btree or if it's for the deletion or update
+ * of an existing item.
+ *
+ * This is used in a context where we need to update the chunk btree outside
+ * block group allocation and removal, to avoid a deadlock with a concurrent
+ * task that is allocating a metadata or data block group and therefore needs to
+ * update the chunk btree while holding the chunk mutex. After the update to the
+ * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
+ *
+ */
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ u64 bytes;
+
+ if (is_item_insertion)
+ bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
+ else
+ bytes = btrfs_calc_metadata_size(fs_info, 1);
+
+ mutex_lock(&fs_info->chunk_mutex);
+ reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
+ mutex_unlock(&fs_info->chunk_mutex);
+}
+
void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
{
struct btrfs_block_group *block_group;
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 07f977d3816c..5878b7ce3b78 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -293,6 +293,8 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index fed823596248..33a0ee7ac590 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -2692,8 +2692,12 @@ static int relocate_tree_block(struct btrfs_trans_handle *trans,
list_add_tail(&node->list, &rc->backref_cache.changed);
} else {
path->lowest_level = node->level;
+ if (root == root->fs_info->chunk_root)
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, key, path, 0, 1);
btrfs_release_path(path);
+ if (root == root->fs_info->chunk_root)
+ btrfs_trans_release_chunk_metadata(trans);
if (ret > 0)
ret = 0;
}
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index debba6f04858..9eab8a741166 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1847,8 +1847,10 @@ static int btrfs_add_dev_item(struct btrfs_trans_handle *trans,
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, true);
ret = btrfs_insert_empty_item(trans, trans->fs_info->chunk_root, path,
&key, sizeof(*dev_item));
+ btrfs_trans_release_chunk_metadata(trans);
if (ret)
goto out;
@@ -1921,7 +1923,9 @@ static int btrfs_rm_dev_item(struct btrfs_device *device)
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret) {
if (ret > 0)
ret = -ENOENT;
@@ -2513,7 +2517,9 @@ static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
key.type = BTRFS_DEV_ITEM_KEY;
while (1) {
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0)
goto error;
@@ -2862,6 +2868,7 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
struct btrfs_super_block *super_copy = fs_info->super_copy;
u64 old_total;
u64 diff;
+ int ret;
if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state))
return -EACCES;
@@ -2890,7 +2897,11 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
&trans->transaction->dev_update_list);
mutex_unlock(&fs_info->chunk_mutex);
- return btrfs_update_device(trans, device);
+ btrfs_reserve_chunk_metadata(trans, false);
+ ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
}
static int btrfs_free_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
@@ -4925,8 +4936,10 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
round_down(old_total - diff, fs_info->sectorsize));
mutex_unlock(&fs_info->chunk_mutex);
+ btrfs_reserve_chunk_metadata(trans, false);
/* Now btrfs_update_device() will change the on-disk size. */
ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0) {
btrfs_abort_transaction(trans, ret);
btrfs_end_transaction(trans);
]
|
| 140/4 |
2026/08/08 03:46 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 19381
output: 39
thoughts: 62
Thoughts: Okay, let's see what we can dig up.
**Hunting Down Related Btrfs Code Changes**
Alright, I need to get a handle on the landscape around these specific btrfs functions, particularly `btrfs_remove_dev_extents`, `check_system_chunk`, and `btrfs_remove_chunk`. My initial thought is to run a code search through the commit history. I'll start with `git log` and specifically look for commits mentioning `btrfs_remove_dev_extents` directly. That should give me a solid base to then explore related modifications or bug fixes in the vicinity. Once I've got that initial search done, I'll branch out to see if I can find the related chunk code. I'll need to do it methodically to avoid missing something.
|
| 141/4 |
2026/08/08 03:46 |
tool |
git-log |
0m
Args: map[CodeRegexp:btrfs_remove_dev_extents PathPrefix:fs/btrfs/]
Results: map[Output:979e1dc3d69e btrfs: handle deletions from remapped block group
]
|
| 142/4 |
2026/08/08 03:46 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 19694
output: 26
thoughts: 18
Thoughts: Okay, here's my interpretation of what's going through my mind when I encounter that commit hash, presented as a first-person summary, and assuming expert-level knowledge in the field.
**Diving into Commit 979e1dc3d69e**
Alright, *979e1dc3d69e*... let's see what we've got. The immediate context is, of course, critical. Where does this commit sit in the broader history? What branch is it on? What am I *expecting* to find here based on recent activity, the bug reports, or the tasks I’ve been juggling? Is it a regression fix, a new feature, a refactor, or something more obscure like a documentation update?
My first instinct is to pull up the diff. I'm not going to be reading the code line by line, unless something immediately jumps out at me as a potential problem or opportunity. I’m looking for structural changes. Did they introduce a new library? Update a critical dependency? Change an architectural pattern? Where are the core changes located and how do they relate to the bigger picture? I'm glancing at the file names and thinking about the modules these files belong to. If it's a code change, I need to understand the intent. Based on the file paths, I already have an idea of the domain the changes apply to.
I'll be paying close attention to the commit message too, of course. Is it informative and well-written? Does it clearly articulate the problem, the solution, and the reasoning behind the changes? Does it refer to any relevant issues, tickets, or pull requests? The commit message should align with my intuition after reviewing the diff. If something feels off, I'll need to dig deeper. I want to see if there is any documentation on the changes.
Based on initial read, I need to quickly determine the impact. Does this affect the core logic, or is it isolated to some ancillary feature? Are there any potential performance implications, security vulnerabilities, or backward compatibility issues? Given my prior knowledge of this codebase, I know which areas are more sensitive or prone to problems. I'll focus my attention there.
Finally, assuming all checks out, I’ll consider implications for testing. Does this commit come with unit tests, integration tests, or end-to-end tests? Do the tests adequately cover the changes? Are there any gaps that need to be addressed? If the tests aren't sufficient, I may need to create a test plan, or simply make a note to add tests in the future.
|
| 143/4 |
2026/08/08 03:46 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1029 lines.
[commit 979e1dc3d69e4c825eec05d05d9567b251f6ec23
Author: Mark Harmstone <mark@harmstone.com>
Date: Wed Jan 7 14:09:10 2026 +0000
btrfs: handle deletions from remapped block group
Handle the case where we free an extent from a block group that has the
REMAPPED flag set. Because the remap tree is orthogonal to the extent
tree, for data this may be within any number of identity remaps or
actual remaps. If we're freeing a metadata node, this will be wholly
inside one or the other.
btrfs_remove_extent_from_remap_tree() searches the remap tree for the
remaps that cover the range in question, then calls
remove_range_from_remap_tree() for each one, to punch a hole in the
remap and adjust the free-space tree.
For an identity remap, remove_range_from_remap_tree() will adjust the
block group's `identity_remap_count` if this changes. If it reaches
zero we mark the block group as fully remapped.
For an identity remap, remove_range_from_remap_tree() will adjust the
block group's `identity_remap_count` if this changes. If it reaches
zero we mark the block group as fully remapped.
Fully remapped block groups have their chunk stripes removed and their
device extents freed, which makes the disk space available again to the
chunk allocator. This happens asynchronously: in the cleaner thread for
sync discard and nodiscard, and (in a later patch) in the discard worker
for async discard.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a1ab513fa8ea..3b8a750d8519 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -1067,6 +1067,24 @@ static int remove_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_del_item(trans, root, path);
}
+void btrfs_remove_bg_from_sinfo(struct btrfs_block_group *bg)
+{
+ int factor = btrfs_bg_type_to_factor(bg->flags);
+
+ spin_lock(&bg->space_info->lock);
+ if (btrfs_test_opt(bg->fs_info, ENOSPC_DEBUG)) {
+ WARN_ON(bg->space_info->total_bytes < bg->length);
+ WARN_ON(bg->space_info->bytes_readonly < bg->length - bg->zone_unusable);
+ WARN_ON(bg->space_info->bytes_zone_unusable < bg->zone_unusable);
+ WARN_ON(bg->space_info->disk_total < bg->length * factor);
+ }
+ bg->space_info->total_bytes -= bg->length;
+ bg->space_info->bytes_readonly -= (bg->length - bg->zone_unusable);
+ btrfs_space_info_update_bytes_zone_unusable(bg->space_info, -bg->zone_unusable);
+ bg->space_info->disk_total -= bg->length * factor;
+ spin_unlock(&bg->space_info->lock);
+}
+
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct btrfs_chunk_map *map)
{
@@ -1078,7 +1096,6 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct kobject *kobj = NULL;
int ret;
int index;
- int factor;
struct btrfs_caching_control *caching_ctl = NULL;
bool remove_map;
bool remove_rsv = false;
@@ -1087,7 +1104,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
if (!block_group)
return -ENOENT;
- BUG_ON(!block_group->ro);
+ BUG_ON(!block_group->ro && !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED));
trace_btrfs_remove_block_group(block_group);
/*
@@ -1099,7 +1116,6 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
block_group->length);
index = btrfs_bg_flags_to_raid_index(block_group->flags);
- factor = btrfs_bg_type_to_factor(block_group->flags);
/* make sure this block group isn't part of an allocation cluster */
cluster = &fs_info->data_alloc_cluster;
@@ -1223,26 +1239,11 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
spin_lock(&block_group->space_info->lock);
list_del_init(&block_group->ro_list);
-
- if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
- WARN_ON(block_group->space_info->total_bytes
- < block_group->length);
- WARN_ON(block_group->space_info->bytes_readonly
- < block_group->length - block_group->zone_unusable);
- WARN_ON(block_group->space_info->bytes_zone_unusable
- < block_group->zone_unusable);
- WARN_ON(block_group->space_info->disk_total
- < block_group->length * factor);
- }
- block_group->space_info->total_bytes -= block_group->length;
- block_group->space_info->bytes_readonly -=
- (block_group->length - block_group->zone_unusable);
- btrfs_space_info_update_bytes_zone_unusable(block_group->space_info,
- -block_group->zone_unusable);
- block_group->space_info->disk_total -= block_group->length * factor;
-
spin_unlock(&block_group->space_info->lock);
+ if (!(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED))
+ btrfs_remove_bg_from_sinfo(block_group);
+
/*
* Remove the free space for the block group from the free space tree
* and the block group's item from the extent tree before marking the
@@ -1575,8 +1576,10 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
spin_lock(&space_info->lock);
spin_lock(&block_group->lock);
- if (btrfs_is_block_group_used(block_group) || block_group->ro ||
- list_is_singular(&block_group->list)) {
+ if (btrfs_is_block_group_used(block_group) ||
+ (block_group->ro && !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED)) ||
+ list_is_singular(&block_group->list) ||
+ test_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &block_group->runtime_flags)) {
/*
* We want to bail if we made new allocations or have
* outstanding allocations in this block group. We do
@@ -1617,9 +1620,10 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
* needing to allocate extents from the block group.
*/
used = btrfs_space_info_used(space_info, true);
- if ((space_info->total_bytes - block_group->length < used &&
- block_group->zone_unusable < block_group->length) ||
- has_unwritten_metadata(block_group)) {
+ if (((space_info->total_bytes - block_group->length < used &&
+ block_group->zone_unusable < block_group->length) ||
+ has_unwritten_metadata(block_group)) &&
+ !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED)) {
/*
* Add a reference for the list, compensate for the ref
* drop under the "next" label for the
@@ -1784,6 +1788,9 @@ void btrfs_mark_bg_unused(struct btrfs_block_group *bg)
btrfs_get_block_group(bg);
trace_btrfs_add_unused_block_group(bg);
list_add_tail(&bg->bg_list, &fs_info->unused_bgs);
+ } else if (bg->flags & BTRFS_BLOCK_GROUP_REMAPPED &&
+ bg->identity_remap_count == 0) {
+ /* Leave fully remapped block groups on the fully_remapped_bgs list. */
} else if (!test_bit(BLOCK_GROUP_FLAG_NEW, &bg->runtime_flags)) {
/* Pull out the block group from the reclaim_bgs list. */
trace_btrfs_add_unused_block_group(bg);
@@ -4581,6 +4588,13 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
list_del_init(&block_group->bg_list);
btrfs_put_block_group(block_group);
}
+
+ while (!list_empty(&info->fully_remapped_bgs)) {
+ block_group = list_first_entry(&info->fully_remapped_bgs,
+ struct btrfs_block_group, bg_list);
+ list_del_init(&block_group->bg_list);
+ btrfs_put_block_group(block_group);
+ }
spin_unlock(&info->unused_bgs_lock);
spin_lock(&info->zone_active_bgs_lock);
@@ -4768,3 +4782,24 @@ bool btrfs_block_group_should_use_size_class(const struct btrfs_block_group *bg)
return false;
return true;
}
+
+void btrfs_mark_bg_fully_remapped(struct btrfs_block_group *bg,
+ struct btrfs_trans_handle *trans)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ /*
+ * The block group might already be on the unused_bgs list, remove it
+ * if it is. It'll get readded after the async discard worker finishes,
+ * or in btrfs_handle_fully_remapped_bgs() if we're not using async
+ * discard.
+ */
+ if (!list_empty(&bg->bg_list))
+ list_del(&bg->bg_list);
+ else
+ btrfs_get_block_group(bg);
+
+ list_add_tail(&bg->bg_list, &fs_info->fully_remapped_bgs);
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index ecabb1a9fc0e..f5c15c7f6cc7 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -92,6 +92,7 @@ enum btrfs_block_group_flags {
* transaction.
*/
BLOCK_GROUP_FLAG_NEW,
+ BLOCK_GROUP_FLAG_FULLY_REMAPPED,
};
enum btrfs_caching_type {
@@ -340,6 +341,7 @@ int btrfs_add_new_free_space(struct btrfs_block_group *block_group,
struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
struct btrfs_fs_info *fs_info,
const u64 chunk_offset);
+void btrfs_remove_bg_from_sinfo(struct btrfs_block_group *bg);
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct btrfs_chunk_map *map);
void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info);
@@ -411,5 +413,7 @@ int btrfs_use_block_group_size_class(struct btrfs_block_group *bg,
enum btrfs_block_group_size_class size_class,
bool force_wrong_size_class);
bool btrfs_block_group_should_use_size_class(const struct btrfs_block_group *bg);
+void btrfs_mark_bg_fully_remapped(struct btrfs_block_group *bg,
+ struct btrfs_trans_handle *trans);
#endif /* BTRFS_BLOCK_GROUP_H */
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index c69734c74c26..627282613eee 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1473,6 +1473,10 @@ static int cleaner_kthread(void *arg)
*/
btrfs_run_defrag_inodes(fs_info);
+ if (btrfs_fs_incompat(fs_info, REMAP_TREE) &&
+ !btrfs_test_opt(fs_info, DISCARD_ASYNC))
+ btrfs_handle_fully_remapped_bgs(fs_info);
+
/*
* Acquires fs_info->reclaim_bgs_lock to avoid racing
* with relocation (btrfs_relocate_chunk) and relocation
@@ -2807,6 +2811,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
INIT_LIST_HEAD(&fs_info->tree_mod_seq_list);
INIT_LIST_HEAD(&fs_info->unused_bgs);
INIT_LIST_HEAD(&fs_info->reclaim_bgs);
+ INIT_LIST_HEAD(&fs_info->fully_remapped_bgs);
INIT_LIST_HEAD(&fs_info->zone_active_bgs);
#ifdef CONFIG_BTRFS_DEBUG
INIT_LIST_HEAD(&fs_info->allocated_roots);
@@ -2862,6 +2867,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
mutex_init(&fs_info->chunk_mutex);
mutex_init(&fs_info->transaction_kthread_mutex);
mutex_init(&fs_info->cleaner_mutex);
+ mutex_init(&fs_info->remap_mutex);
mutex_init(&fs_info->ro_block_group_mutex);
init_rwsem(&fs_info->commit_root_sem);
init_rwsem(&fs_info->cleanup_work_sem);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index ce4bda1f37ad..5e3e9f18b263 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -41,6 +41,7 @@
#include "tree-checker.h"
#include "raid-stripe-tree.h"
#include "delayed-inode.h"
+#include "relocation.h"
#undef SCRAMBLE_DELAYED_REFS
@@ -2848,6 +2849,73 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info,
return 0;
}
+/*
+ * Complete the remapping of a block group by removing its chunk stripes and
+ * device extents, and adding it to the unused list if there's no longer any
+ * extents nominally within it.
+ */
+int btrfs_complete_bg_remapping(struct btrfs_block_group *bg)
+{
+ struct btrfs_fs_info *fs_info = bg->fs_info;
+ struct btrfs_chunk_map *map;
+ int ret;
+
+ map = btrfs_get_chunk_map(fs_info, bg->start, 1);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ ret = btrfs_last_identity_remap_gone(map, bg);
+ if (ret) {
+ btrfs_free_chunk_map(map);
+ return ret;
+ }
+
+ /*
+ * Set num_stripes to 0, so that btrfs_remove_dev_extents() won't run a
+ * second time.
+ */
+ map->num_stripes = 0;
+
+ btrfs_free_chunk_map(map);
+
+ if (bg->used == 0) {
+ spin_lock(&fs_info->unused_bgs_lock);
+ if (!list_empty(&bg->bg_list)) {
+ list_del_init(&bg->bg_list);
+ btrfs_put_block_group(bg);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+
+ btrfs_mark_bg_unused(bg);
+ }
+
+ return 0;
+}
+
+void btrfs_handle_fully_remapped_bgs(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ while (!list_empty(&fs_info->fully_remapped_bgs)) {
+ bg = list_first_entry(&fs_info->fully_remapped_bgs,
+ struct btrfs_block_group, bg_list);
+ list_del_init(&bg->bg_list);
+ spin_unlock(&fs_info->unused_bgs_lock);
+
+ ret = btrfs_complete_bg_remapping(bg);
+ if (ret) {
+ btrfs_put_block_group(bg);
+ return;
+ }
+
+ btrfs_put_block_group(bg);
+ spin_lock(&fs_info->unused_bgs_lock);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
+
int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3000,11 +3068,22 @@ u64 btrfs_get_extent_owner_root(struct btrfs_fs_info *fs_info,
}
static int do_free_extent_accounting(struct btrfs_trans_handle *trans,
- u64 bytenr, struct btrfs_squota_delta *delta)
+ u64 bytenr, struct btrfs_squota_delta *delta,
+ struct btrfs_path *path)
{
int ret;
+ bool remapped = false;
u64 num_bytes = delta->num_bytes;
+ /* Returns 1 on success and 0 on no-op. */
+ ret = btrfs_remove_extent_from_remap_tree(trans, path, bytenr, num_bytes);
+ if (unlikely(ret < 0)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ } else if (ret == 1) {
+ remapped = true;
+ }
+
if (delta->is_data) {
struct btrfs_root *csum_root;
@@ -3028,10 +3107,13 @@ static int do_free_extent_accounting(struct btrfs_trans_handle *trans,
return ret;
}
- ret = btrfs_add_to_free_space_tree(trans, bytenr, num_bytes);
- if (unlikely(ret)) {
- btrfs_abort_transaction(trans, ret);
- return ret;
+ /* If remapped, FST has already been taken care of in remove_range_from_remap_tree(). */
+ if (!remapped) {
+ ret = btrfs_add_to_free_space_tree(trans, bytenr, num_bytes);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
}
ret = btrfs_update_block_group(trans, bytenr, num_bytes, false);
@@ -3390,7 +3472,7 @@ static int __btrfs_free_extent(struct btrfs_trans_handle *trans,
}
btrfs_release_path(path);
- ret = do_free_extent_accounting(trans, bytenr, &delta);
+ ret = do_free_extent_accounting(trans, bytenr, &delta, path);
}
btrfs_release_path(path);
diff --git a/fs/btrfs/extent-tree.h b/fs/btrfs/extent-tree.h
index 71bb8109c969..d7b6aeb63656 100644
--- a/fs/btrfs/extent-tree.h
+++ b/fs/btrfs/extent-tree.h
@@ -163,5 +163,7 @@ void btrfs_error_unpin_extent_range(struct btrfs_fs_info *fs_info, u64 start, u6
int btrfs_discard_extent(struct btrfs_fs_info *fs_info, u64 bytenr,
u64 num_bytes, u64 *actual_bytes);
int btrfs_trim_fs(struct btrfs_fs_info *fs_info, struct fstrim_range *range);
+void btrfs_handle_fully_remapped_bgs(struct btrfs_fs_info *fs_info);
+int btrfs_complete_bg_remapping(struct btrfs_block_group *bg);
#endif
diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h
index 13b0aa0b9da9..5bbc993a66e1 100644
--- a/fs/btrfs/fs.h
+++ b/fs/btrfs/fs.h
@@ -587,6 +587,7 @@ struct btrfs_fs_info {
struct mutex transaction_kthread_mutex;
struct mutex cleaner_mutex;
struct mutex chunk_mutex;
+ struct mutex remap_mutex;
/*
* This is taken to make sure we don't set block groups ro after the
@@ -840,10 +841,11 @@ struct btrfs_fs_info {
struct list_head reclaim_bgs;
int bg_reclaim_threshold;
- /* Protects the lists unused_bgs and reclaim_bgs. */
+ /* Protects the lists unused_bgs, reclaim_bgs, and fully_remapped_bgs. */
spinlock_t unused_bgs_lock;
/* Protected by unused_bgs_lock. */
struct list_head unused_bgs;
+ struct list_head fully_remapped_bgs;
struct mutex unused_bg_unpin_mutex;
/* Protect block groups that are going to be deleted */
struct mutex reclaim_bgs_lock;
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 6de508323dbd..e0558b2cd0b4 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -37,6 +37,7 @@
#include "super.h"
#include "tree-checker.h"
#include "raid-stripe-tree.h"
+#include "free-space-tree.h"
/*
* Relocation overview
@@ -3859,6 +3860,177 @@ static const char *stage_to_string(enum reloc_stage stage)
return "unknown";
}
+static void adjust_block_group_remap_bytes(struct btrfs_trans_handle *trans,
+ struct btrfs_block_group *bg, s64 diff)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ bool bg_already_dirty = true;
+ bool mark_unused = false;
+
+ spin_lock(&bg->lock);
+ bg->remap_bytes += diff;
+ if (bg->used == 0 && bg->remap_bytes == 0)
+ mark_unused = true;
+ spin_unlock(&bg->lock);
+
+ if (mark_unused)
+ btrfs_mark_bg_unused(bg);
+
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ if (list_empty(&bg->dirty_list)) {
+ list_add_tail(&bg->dirty_list, &trans->transaction->dirty_bgs);
+ bg_already_dirty = false;
+ btrfs_get_block_group(bg);
+ }
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+
+ /* Modified block groups are accounted for in the delayed_refs_rsv. */
+ if (!bg_already_dirty)
+ btrfs_inc_delayed_refs_rsv_bg_updates(fs_info);
+}
+
+static int remove_chunk_stripes(struct btrfs_trans_handle *trans,
+ struct btrfs_chunk_map *chunk_map,
+ struct btrfs_path *path)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key;
+ struct extent_buffer *leaf;
+ struct btrfs_chunk *chunk;
+ int ret;
+
+ key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+ key.offset = chunk_map->start;
+
+ btrfs_reserve_chunk_metadata(trans, false);
+
+ ret = btrfs_search_slot(trans, fs_info->chunk_root, &key, path, 0, 1);
+ if (ret) {
+ if (ret == 1) {
+ btrfs_release_path(path);
+ ret = -ENOENT;
+ }
+ btrfs_trans_release_chunk_metadata(trans);
+ return ret;
+ }
+
+ leaf = path->nodes[0];
+
+ chunk = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_chunk);
+ btrfs_set_chunk_num_stripes(leaf, chunk, 0);
+ btrfs_set_chunk_sub_stripes(leaf, chunk, 0);
+
+ btrfs_truncate_item(trans, path, offsetof(struct btrfs_chunk, stripe), 1);
+
+ btrfs_mark_buffer_dirty(trans, leaf);
+
+ btrfs_release_path(path);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return 0;
+}
+
+int btrfs_last_identity_remap_gone(struct btrfs_chunk_map *chunk_map,
+ struct btrfs_block_group *bg)
+{
+ struct btrfs_fs_info *fs_info = bg->fs_info;
+ struct btrfs_trans_handle *trans;
+ int ret;
+ unsigned int num_items;
+ BTRFS_PATH_AUTO_FREE(path);
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ /*
+ * One item for each entry we're removing in the dev extents tree, and
+ * another for each device. DUP chunks are all on one device,
+ * everything else has one device per stripe.
+ */
+ if (bg->flags & BTRFS_BLOCK_GROUP_DUP)
+ num_items = chunk_map->num_stripes + 1;
+ else
+ num_items = 2 * chunk_map->num_stripes;
+
+ trans = btrfs_start_transaction_fallback_global_rsv(fs_info->tree_root, num_items);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+
+ ret = btrfs_remove_dev_extents(trans, chunk_map);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+
+ mutex_lock(&trans->fs_info->chunk_mutex);
+ for (unsigned int i = 0; i < chunk_map->num_stripes; i++) {
+ ret = btrfs_update_device(trans, chunk_map->stripes[i].dev);
+ if (unlikely(ret)) {
+ mutex_unlock(&trans->fs_info->chunk_mutex);
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+ }
+ mutex_unlock(&trans->fs_info->chunk_mutex);
+
+ write_lock(&trans->fs_info->mapping_tree_lock);
+ btrfs_chunk_map_device_clear_bits(chunk_map, CHUNK_ALLOCATED);
+ write_unlock(&trans->fs_info->mapping_tree_lock);
+
+ btrfs_remove_bg_from_sinfo(bg);
+
+ ret = remove_chunk_stripes(trans, chunk_map, path);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+
+ ret = btrfs_commit_transaction(trans);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void adjust_identity_remap_count(struct btrfs_trans_handle *trans,
+ struct btrfs_block_group *bg, int delta)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ bool bg_already_dirty = true;
+ bool mark_fully_remapped = false;
+
+ WARN_ON(delta < 0 && -delta > bg->identity_remap_count);
+
+ spin_lock(&bg->lock);
+
+ bg->identity_remap_count += delta;
+
+ if (bg->identity_remap_count == 0 &&
+ !test_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &bg->runtime_flags)) {
+ set_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &bg->runtime_flags);
+ mark_fully_remapped = true;
+ }
+
+ spin_unlock(&bg->lock);
+
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ if (list_empty(&bg->dirty_list)) {
+ list_add_tail(&bg->dirty_list, &trans->transaction->dirty_bgs);
+ bg_already_dirty = false;
+ btrfs_get_block_group(bg);
+ }
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+
+ /* Modified block groups are accounted for in the delayed_refs_rsv. */
+ if (!bg_already_dirty)
+ btrfs_inc_delayed_refs_rsv_bg_updates(fs_info);
+
+ if (mark_fully_remapped)
+ btrfs_mark_bg_fully_remapped(bg, trans);
+}
+
int btrfs_translate_remap(struct btrfs_fs_info *fs_info, u64 *logical, u64 *length)
{
int ret;
@@ -4463,3 +4635,260 @@ u64 btrfs_get_reloc_bg_bytenr(const struct btrfs_fs_info *fs_info)
logical = fs_info->reloc_ctl->block_group->start;
return logical;
}
+
+static int insert_remap_item(struct btrfs_trans_handle *trans, struct btrfs_path *path,
+ u64 old_addr, u64 length, u64 new_addr)
+{
+ int ret;
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key;
+ struct btrfs_remap_item remap = { 0 };
+
+ if (old_addr == new_addr) {
+ /* Add new identity remap item. */
+ key.objectid = old_addr;
+ key.type = BTRFS_IDENTITY_REMAP_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root, path,
+ &key, 0);
+ if (ret)
+ return ret;
+ } else {
+ /* Add new remap item. */
+ key.objectid = old_addr;
+ key.type = BTRFS_REMAP_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root,
+ path, &key, sizeof(struct btrfs_remap_item));
+ if (ret)
+ return ret;
+
+ btrfs_set_stack_remap_address(&remap, new_addr);
+
+ write_extent_buffer(path->nodes[0], &remap,
+ btrfs_item_ptr_offset(path->nodes[0], path->slots[0]),
+ sizeof(struct btrfs_remap_item));
+
+ btrfs_release_path(path);
+
+ /* Add new backref item. */
+ key.objectid = new_addr;
+ key.type = BTRFS_REMAP_BACKREF_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root,
+ path, &key,
+ sizeof(struct btrfs_remap_item));
+ if (ret)
+ return ret;
+
+ btrfs_set_stack_remap_address(&remap, old_addr);
+
+ write_extent_buffer(path->nodes[0], &remap,
+ btrfs_item_ptr_offset(path->nodes[0], path->slots[0]),
+ sizeof(struct btrfs_remap_item));
+ }
+
+ btrfs_release_path(path);
+
+ return 0;
+}
+
+/*
+ * Punch a hole in the remap item or identity remap item pointed to by path,
+ * for the range [hole_start, hole_start + hole_length).
+ */
+static int remove_range_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ struct btrfs_block_group *bg,
+ u64 hole_start, u64 hole_length)
+{
+ int ret;
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct extent_buffer *leaf = path->nodes[0];
+ struct btrfs_key key;
+ u64 hole_end, new_addr, remap_start, remap_length, remap_end;
+ u64 overlap_length;
+ bool is_identity_remap;
+ int identity_count_delta = 0;
+
+ hole_end = hole_start + hole_length;
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+
+ is_identity_remap = (key.type == BTRFS_IDENTITY_REMAP_KEY);
+
+ remap_start = key.objectid;
+ remap_length = key.offset;
+ remap_end = remap_start + remap_length;
+
+ if (is_identity_remap) {
+ new_addr = remap_start;
+ } else {
+ struct btrfs_remap_item *remap_ptr;
+
+ remap_ptr = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_remap_item);
+ new_addr = btrfs_remap_address(leaf, remap_ptr);
+ }
+
+ /* Delete old item. */
+ ret = btrfs_del_item(trans, fs_info->remap_root, path);
+ btrfs_release_path(path);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap) {
+ identity_count_delta = -1;
+ } else {
+ /* Remove backref. */
+ key.objectid = new_addr;
+ key.type = BTRFS_REMAP_BACKREF_KEY;
+ key.offset = remap_length;
+
+ ret = btrfs_search_slot(trans, fs_info->remap_root, &key, path, -1, 1);
+ if (ret) {
+ if (ret == 1) {
+ btrfs_release_path(path);
+ ret = -ENOENT;
+ }
+ return ret;
+ }
+
+ ret = btrfs_del_item(trans, fs_info->remap_root, path);
+
+ btrfs_release_path(path);
+
+ if (ret)
+ return ret;
+ }
+
+ /* If hole_start > remap_start, re-add the start of the remap item. */
+ if (hole_start > remap_start) {
+ ret = insert_remap_item(trans, path, remap_start,
+ hole_start - remap_start, new_addr);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap)
+ identity_count_delta++;
+ }
+
+ /* If hole_end < remap_end, re-add the end of the remap item. */
+ if (hole_end < remap_end) {
+ ret = insert_remap_item(trans, path, hole_end,
+ remap_end - hole_end,
+ hole_end - remap_start + new_addr);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap)
+ identity_count_delta++;
+ }
+
+ if (identity_count_delta != 0)
+ adjust_identity_remap_count(trans, bg, identity_count_delta);
+
+ overlap_length = min_t(u64, hole_end, remap_end) -
+ max_t(u64, hole_start, remap_start);
+
+ if (!is_identity_remap) {
+ struct btrfs_block_group *dest_bg;
+
+ dest_bg = btrfs_lookup_block_group(fs_info, new_addr);
+ adjust_block_group_remap_bytes(trans, dest_bg, -overlap_length);
+ btrfs_put_block_group(dest_bg);
+ ret = btrfs_add_to_free_space_tree(trans,
+ hole_start - remap_start + new_addr,
+ overlap_length);
+ if (ret)
+ return ret;
+ }
+
+ ret = overlap_length;
+
+ return ret;
+}
+
+/*
+ * Return 1 if remove_range_from_remap_tree() has been called successfully,
+ * 0 if block group wasn't remapped, and a negative number on error.
+ */
+int btrfs_remove_extent_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ u64 bytenr, u64 num_bytes)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key, found_key;
+ struct extent_buffer *leaf;
+ struct btrfs_block_group *bg;
+ int ret, length;
+
+ if (!(btrfs_super_incompat_flags(fs_info->super_copy) &
+ BTRFS_FEATURE_INCOMPAT_REMAP_TREE))
+ return 0;
+
+ bg = btrfs_lookup_block_group(fs_info, bytenr);
+ if (!bg)
+ return 0;
+
+ mutex_lock(&fs_info->remap_mutex);
+
+ if (!(bg->flags & BTRFS_BLOCK_GROUP_REMAPPED)) {
+ mutex_unlock(&fs_info->remap_mutex);
+ btrfs_put_block_group(bg);
+ return 0;
+ }
+
+ do {
+ key.objectid = bytenr;
+ key.type = (u8)-1;
+ key.offset = (u64)-1;
+
+ ret = btrfs_search_slot(trans, fs_info->remap_root, &key, path, -1, 1);
+ if (ret < 0)
+ goto end;
+
+ leaf = path->nodes[0];
+ if (path->slots[0] == 0) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ path->slots[0]--;
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.type != BTRFS_IDENTITY_REMAP_KEY &&
+ found_key.type != BTRFS_REMAP_KEY) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ if (bytenr < found_key.objectid ||
+ bytenr >= found_key.objectid + found_key.offset) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ length = remove_range_from_remap_tree(trans, path, bg, bytenr, num_bytes);
+ if (length < 0) {
+ ret = length;
+ goto end;
+ }
+
+ bytenr += length;
+ num_bytes -= length;
+ } while (num_bytes > 0);
+
+ ret = 1;
+
+end:
+ mutex_unlock(&fs_info->remap_mutex);
+
+ btrfs_put_block_group(bg);
+ btrfs_release_path(path);
+
+ return ret;
+}
diff --git a/fs/btrfs/relocation.h b/fs/btrfs/relocation.h
index c0ee26004fc1..3afb6f85b722 100644
--- a/fs/btrfs/relocation.h
+++ b/fs/btrfs/relocation.h
@@ -32,5 +32,10 @@ struct btrfs_root *find_reloc_root(struct btrfs_fs_info *fs_info, u64 bytenr);
bool btrfs_should_ignore_reloc_root(const struct btrfs_root *root);
u64 btrfs_get_reloc_bg_bytenr(const struct btrfs_fs_info *fs_info);
int btrfs_translate_remap(struct btrfs_fs_info *fs_info, u64 *logical, u64 *length);
+int btrfs_remove_extent_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ u64 bytenr, u64 num_bytes);
+int btrfs_last_identity_remap_gone(struct btrfs_chunk_map *chunk_map,
+ struct btrfs_block_group *bg);
#endif
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 2a4bda452d85..4884c7b62c61 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2925,8 +2925,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path
return ret;
}
-static noinline int btrfs_update_device(struct btrfs_trans_handle *trans,
- struct btrfs_device *device)
+int btrfs_update_device(struct btrfs_trans_handle *trans, struct btrfs_device *device)
{
int ret;
BTRFS_PATH_AUTO_FREE(path);
@@ -3224,25 +3223,12 @@ static int remove_chunk_item(struct btrfs_trans_handle *trans,
return btrfs_free_chunk(trans, chunk_offset);
}
-int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
+int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_chunk_map *map;
+ struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
u64 dev_extent_len = 0;
int i, ret = 0;
- struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
-
- map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
- if (IS_ERR(map)) {
- /*
- * This is a logic error, but we don't want to just rely on the
- * user having built with ASSERT enabled, so if ASSERT doesn't
- * do anything we still error out.
- */
- DEBUG_WARN("errr %ld reading chunk map at offset %llu",
- PTR_ERR(map), chunk_offset);
- return PTR_ERR(map);
- }
/*
* First delete the device extent items from the devices btree.
@@ -3263,7 +3249,7 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
if (unlikely(ret)) {
mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
- goto out;
+ return ret;
}
if (device->bytes_used > 0) {
@@ -3283,6 +3269,26 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
mutex_unlock(&fs_devices->device_list_mutex);
+ return 0;
+}
+
+int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_chunk_map *map;
+ int ret;
+
+ map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
+ if (IS_ERR(map)) {
+ DEBUG_WARN("errr %ld reading chunk map at offset %llu",
+ PTR_ERR(map), chunk_offset);
+ return PTR_ERR(map);
+ }
+
+ ret = btrfs_remove_dev_extents(trans, map);
+ if (ret)
+ goto out;
+
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
*
@@ -5419,7 +5425,7 @@ static void chunk_map_device_set_bits(struct btrfs_chunk_map *map, unsigned int
}
}
-static void chunk_map_device_clear_bits(struct btrfs_chunk_map *map, unsigned int bits)
+void btrfs_chunk_map_device_clear_bits(struct btrfs_chunk_map *map, unsigned int bits)
{
for (int i = 0; i < map->num_stripes; i++) {
struct btrfs_io_stripe *stripe = &map->stripes[i];
@@ -5436,7 +5442,7 @@ void btrfs_remove_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_ma
write_lock(&fs_info->mapping_tree_lock);
rb_erase_cached(&map->rb_node, &fs_info->mapping_tree);
RB_CLEAR_NODE(&map->rb_node);
- chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
+ btrfs_chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
write_unlock(&fs_info->mapping_tree_lock);
/* Once for the tree reference. */
@@ -5472,7 +5478,7 @@ int btrfs_add_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_map *m
return -EEXIST;
}
chunk_map_device_set_bits(map, CHUNK_ALLOCATED);
- chunk_map_device_clear_bits(map, CHUNK_TRIMMED);
+ btrfs_chunk_map_device_clear_bits(map, CHUNK_TRIMMED);
write_unlock(&fs_info->mapping_tree_lock);
return 0;
@@ -5828,7 +5834,7 @@ void btrfs_mapping_tree_free(struct btrfs_fs_info *fs_info)
]
]
|
| 144/4 |
2026/08/08 03:46 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 33959
output: 42
thoughts: 730
Thoughts: **Analyzing the Deadlock in `btrfs_remove_chunk`**
Okay, let's dissect this deadlock scenario in `btrfs_remove_chunk`. I'm going through the code again, specifically focusing on the sequence. I was thinking of moving `check_system_chunk` before `btrfs_remove_dev_extents`, but let's see where that takes me.
If I rearrange the call order, I have two choices: call `check_system_chunk` either *without* or *with* holding `chunk_mutex`. The first option is immediately problematic. `check_system_chunk` ultimately calls `reserve_chunk_space`, and that function has a hard assertion requiring `fs_info->chunk_mutex` to be held. It's unsafe to violate that; `reserve_chunk_space` itself might allocate a system chunk via `btrfs_create_chunk` and `btrfs_chunk_alloc_add_chunk_item`, which directly modifies the chunk btree and device items. That's a textbook race condition and synchronization violation if I don't hold the mutex.
So, the only feasible path is to acquire `chunk_mutex` before calling `check_system_chunk`. Then I'd call `btrfs_remove_dev_extents` *while* holding `chunk_mutex`. But that's where the red flag pops up. The comments within `btrfs_remove_dev_extents` (or was it in the original `btrfs_remove_chunk`?) scream about this exact scenario. It highlights how we should *not* hold `chunk_mutex` while calling `btrfs_remove_dev_extents`. The comment warns about a deadlock possibility when deleting device extent items from the devices btree. The comment explicitly notes that the COWing of an extent buffer from the btree might allocate a new metadata chunk, and attempting to acquire `fs_info->chunk_mutex` again within that process will lead to a deadlock.
Let's trace this step-by-step.
1. I acquire `chunk_mutex`.
2. I call `check_system_chunk`, which reserves space.
3. I call `btrfs_remove_dev_extents`.
4. Inside `btrfs_remove_dev_extents`, I call `btrfs_free_dev_extent` to delete device extent items. *This is where the btree comes in*, but now, where are device extents stored? I'm pretty sure it's the chunk btree, but I'm going to quickly confirm this. To confirm this, let's check. Let's use `codesearch-definition-source` or `grepper` to find the exact location of `btrfs_free_dev_extent` for a quick refresher!
|
| 145/4 |
2026/08/08 03:46 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_free_dev_extent]
Results: map[SourceCode:1954: static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
1955: struct btrfs_device *device,
1956: u64 start, u64 *dev_extent_len)
1957: {
1958: struct btrfs_fs_info *fs_info = device->fs_info;
1959: struct btrfs_root *root = fs_info->dev_root;
1960: int ret;
1961: BTRFS_PATH_AUTO_FREE(path);
1962: struct btrfs_key key;
1963: struct btrfs_key found_key;
1964: struct extent_buffer *leaf = NULL;
1965: struct btrfs_dev_extent *extent = NULL;
1966:
1967: path = btrfs_alloc_path();
1968: if (!path)
1969: return -ENOMEM;
1970:
1971: key.objectid = device->devid;
1972: key.type = BTRFS_DEV_EXTENT_KEY;
1973: key.offset = start;
1974: again:
1975: ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
1976: if (ret > 0) {
1977: ret = btrfs_previous_item(root, path, key.objectid,
1978: BTRFS_DEV_EXTENT_KEY);
1979: if (ret)
1980: return ret;
1981: leaf = path->nodes[0];
1982: btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
1983: extent = btrfs_item_ptr(leaf, path->slots[0],
1984: struct btrfs_dev_extent);
1985: BUG_ON(found_key.offset > start || found_key.offset +
1986: btrfs_dev_extent_length(leaf, extent) < start);
1987: key = found_key;
1988: btrfs_release_path(path);
1989: goto again;
1990: } else if (ret == 0) {
1991: leaf = path->nodes[0];
1992: extent = btrfs_item_ptr(leaf, path->slots[0],
1993: struct btrfs_dev_extent);
1994: } else {
1995: return ret;
1996: }
1997:
1998: *dev_extent_len = btrfs_dev_extent_length(leaf, extent);
1999:
2000: ret = btrfs_del_item(trans, root, path);
2001: if (ret == 0)
2002: set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
2003: return ret;
2004: }
SourceFile:fs/btrfs/volumes.c]
|
| 146/4 |
2026/08/08 03:46 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 36244
output: 36
thoughts: 1549
Thoughts: **Analyzing Potential Deadlock in Btrfs Device Removal**
Okay, so `btrfs_free_dev_extent` modifies `fs_info->dev_root`, which is the device tree. That's a standard metadata tree. The act of modifying `fs_info->dev_root`, like calling `btrfs_search_slot` with `cow = 1` and `btrfs_del_item`, requires COWing extent buffers within that device tree. Since this is a metadata tree, COWing involves allocating new metadata extents using `btrfs_alloc_tree_block` -> `btrfs_reserve_extent` -> `find_free_extent`. If there's no free space in existing metadata block groups, `find_free_extent` triggers a new metadata chunk allocation via `btrfs_chunk_alloc`.
Now, if we are in a situation where `btrfs_chunk_alloc` calls `fs_info->chunk_mutex`, and we *already* hold `fs_info->chunk_mutex` (e.g., from an earlier `check_system_chunk` call before `btrfs_remove_dev_extents`), we're going to deadlock. A standard mutex isn't recursive, so it's a guaranteed deadlock.
The comment in `btrfs_remove_dev_extents` explicitly addresses this. It notes that COWing an extent buffer during device extent item deletion *may* lead to allocating a new metadata chunk, which *would* attempt to re-acquire `fs_info->chunk_mutex`, and thus deadlock. The comment says "devices btree" here refers to the device tree (`dev_root`).
Okay, so holding `chunk_mutex` during `btrfs_remove_dev_extents` *is* a problem. But what if we *don't* hold `chunk_mutex` during `btrfs_remove_dev_extents` but we still call `check_system_chunk` first? `check_system_chunk` has to be called under the protection of `chunk_mutex`.
The question becomes: Is this safe:
```c
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
mutex_lock(&fs_info->chunk_mutex);
```
If we release `chunk_mutex` after `check_system_chunk` and then call `btrfs_remove_dev_extents`, what are the consequences?
`check_system_chunk` reserves system space in `fs_info->chunk_block_rsv` and adds `bytes` to `trans->chunk_bytes_reserved`. But if we release the `chunk_mutex` can another task allocate and consume this reserved space?
Wait! `chunk_block_rsv` is a block reserve and the space reserved in it is tied to the transaction handle `trans` (via `trans->chunk_bytes_reserved`).
Wait, `check_system_chunk` *can* allocate a new system chunk if the `space_info` doesn't have enough space by calling `btrfs_create_chunk`, which then calls `btrfs_chunk_alloc_add_chunk_item` which *modifies the chunk btree!* This is the real problem! If we call `check_system_chunk` before `btrfs_remove_dev_extents` and it needs to allocate a system chunk, this will insert a chunk item into the chunk btree while we are not holding the chunk mutex!
If `btrfs_remove_dev_extents` fails after `check_system_chunk` has created a chunk, we're left with an orphaned chunk, which isn't ideal. But a more serious question is whether a concurrent operation could modify the chunk btree while `btrfs_remove_dev_extents` is running. This makes the potential race far more serious.
The key question is: Why is `check_system_chunk` being called *before* `btrfs_remove_dev_extents`? What's the historical reasoning behind this order? I need to review the history of `btrfs_remove_chunk` to understand why. Let's look at the git log.
|
| 147/4 |
2026/08/08 03:47 |
tool |
git-log |
0m
Args: map[CodeRegexp:btrfs_remove_chunk PathPrefix:fs/btrfs/]
Results: map[Output:b56f35560b82 btrfs: handle setting up relocation of block group with remap-tree
979e1dc3d69e btrfs: handle deletions from remapped block group
7dc66abb5a47 btrfs: use a dedicated data structure for chunk maps
2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
b3470b5dbe13 btrfs: add dedicated members for start and length of a block group
e3e0520b32bc btrfs: migrate the block group removal code
97aff912a2fa btrfs: Remove fs_info from btrfs_finish_chunk_alloc
5b4aacefb8fb btrfs: call functions that overwrite their root parameter with fs_info
19c4d2f99478 Revert "btrfs: let btrfs_delete_unused_bgs() to clean relocated bgs"
]
|
| 148/4 |
2026/08/08 03:47 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 39083
output: 26
thoughts: 60
Thoughts: Okay, let's dive into this. I'm taking a look at commit `79bd37120b14`, which, from its description, seems to deal with a fundamental problem in Btrfs chunk allocation – preventing the system chunk array from being completely exhausted. My gut tells me this could be a fairly significant performance and stability enhancement, depending on *how* they've reworked the allocation.
I'll pull up the actual changes using `git show` on that commit hash. I need to see the code diff. I'm expecting to see modifications to the allocation algorithms, probably some new heuristics, and hopefully some improved error handling or resource management around the chunk array. The devil is always in the details with storage management, especially when dealing with the Btrfs complexities. Let's see what's really happening under the hood. Hopefully, this commit addresses the underlying cause of the exhaustion, and isn't just a band-aid. Time to get my hands dirty.
|
| 149/4 |
2026/08/08 03:47 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1281 lines.
[commit 79bd37120b149532af5b21953643ed74af69654f
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:06 2021 +0100
btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
Commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations") fixed a problem that resulted in
exhausting the system chunk array in the superblock when there are many
tasks allocating chunks in parallel. Basically too many tasks enter the
first phase of chunk allocation without previous tasks having finished
their second phase of allocation, resulting in too many system chunks
being allocated. That was originally observed when running the fallocate
tests of stress-ng on a PowerPC machine, using a node size of 64K.
However that commit also introduced a deadlock where a task in phase 1 of
the chunk allocation waited for another task that had allocated a system
chunk to finish its phase 2, but that other task was waiting on an extent
buffer lock held by the first task, therefore resulting in both tasks not
making any progress. That change was later reverted by a patch with the
subject "btrfs: fix deadlock with concurrent chunk allocations involving
system chunks", since there is no simple and short solution to address it
and the deadlock is relatively easy to trigger on zoned filesystems, while
the system chunk array exhaustion is not so common.
This change reworks the chunk allocation to avoid the system chunk array
exhaustion. It accomplishes that by making the first phase of chunk
allocation do the updates of the device items in the chunk btree and the
insertion of the new chunk item in the chunk btree. This is done while
under the protection of the chunk mutex (fs_info->chunk_mutex), in the
same critical section that checks for available system space, allocates
a new system chunk if needed and reserves system chunk space. This way
we do not have chunk space reserved until the second phase completes.
The same logic is applied to chunk removal as well, since it keeps
reserved system space long after it is done updating the chunk btree.
For direct allocation of system chunks, the previous behaviour remains,
because otherwise we would deadlock on extent buffers of the chunk btree.
Changes to the chunk btree are by large done by chunk allocation and chunk
removal, which first reserve chunk system space and then later do changes
to the chunk btree. The other remaining cases are uncommon and correspond
to adding a device, removing a device and resizing a device. All these
other cases do not pre-reserve system space, they modify the chunk btree
right away, so they don't hold reserved space for a long period like chunk
allocation and chunk removal do.
The diff of this change is huge, but more than half of it is just addition
of comments describing both how things work regarding chunk allocation and
removal, including both the new behavior and the parts of the old behavior
that did not change.
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a26209f98279..c557327b4545 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2207,6 +2207,13 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
return ret;
}
+/*
+ * This function, insert_block_group_item(), belongs to the phase 2 of chunk
+ * allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
static int insert_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_block_group *block_group)
{
@@ -2229,15 +2236,19 @@ static int insert_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_insert_item(trans, root, &key, &bgi, sizeof(bgi));
}
+/*
+ * This function, btrfs_create_pending_block_groups(), belongs to the phase 2 of
+ * chunk allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *block_group;
int ret = 0;
- if (!trans->can_flush_pending_bgs)
- return;
-
while (!list_empty(&trans->new_bgs)) {
int index;
@@ -2252,6 +2263,13 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
ret = insert_block_group_item(trans, block_group);
if (ret)
btrfs_abort_transaction(trans, ret);
+ if (!block_group->chunk_item_inserted) {
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, block_group);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret)
+ btrfs_abort_transaction(trans, ret);
+ }
ret = btrfs_finish_chunk_alloc(trans, block_group->start,
block_group->length);
if (ret)
@@ -2275,8 +2293,9 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
btrfs_trans_release_chunk_metadata(trans);
}
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size)
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *cache;
@@ -2286,7 +2305,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
cache = btrfs_create_block_group_cache(fs_info, chunk_offset);
if (!cache)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
cache->length = size;
set_free_space_tree_thresholds(cache);
@@ -2300,7 +2319,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
ret = btrfs_load_block_group_zone_info(cache, true);
if (ret) {
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
ret = exclude_super_stripes(cache);
@@ -2308,7 +2327,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
/* We may have excluded something, so call this just in case */
btrfs_free_excluded_extents(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
add_new_free_space(cache, chunk_offset, chunk_offset + size);
@@ -2335,7 +2354,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
if (ret) {
btrfs_remove_free_space_cache(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
/*
@@ -2354,7 +2373,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
btrfs_update_delayed_refs_rsv(trans);
set_avail_alloc_bits(fs_info, type);
- return 0;
+ return cache;
}
/*
@@ -3232,11 +3251,203 @@ int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type)
return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
+static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ /*
+ * Check if we have enough space in the system space info because we
+ * will need to update device items in the chunk btree and insert a new
+ * chunk item in the chunk btree as well. This will allocate a new
+ * system block group if needed.
+ */
+ check_system_chunk(trans, flags);
+
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ goto out;
+ }
+
+ /*
+ * If this is a system chunk allocation then stop right here and do not
+ * add the chunk item to the chunk btree. This is to prevent a deadlock
+ * because this system chunk allocation can be triggered while COWing
+ * some extent buffer of the chunk btree and while holding a lock on a
+ * parent extent buffer, in which case attempting to insert the chunk
+ * item (or update the device item) would result in a deadlock on that
+ * parent extent buffer. In this case defer the chunk btree updates to
+ * the second phase of chunk allocation and keep our reservation until
+ * the second phase completes.
+ *
+ * This is a rare case and can only be triggered by the very few cases
+ * we have where we need to touch the chunk btree outside chunk allocation
+ * and chunk removal. These cases are basically adding a device, removing
+ * a device or resizing a device.
+ */
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ return 0;
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ /*
+ * Normally we are not expected to fail with -ENOSPC here, since we have
+ * previously reserved space in the system space_info and allocated one
+ * new system chunk if necessary. However there are two exceptions:
+ *
+ * 1) We may have enough free space in the system space_info but all the
+ * existing system block groups have a profile which can not be used
+ * for extent allocation.
+ *
+ * This happens when mounting in degraded mode. For example we have a
+ * RAID1 filesystem with 2 devices, lose one device and mount the fs
+ * using the other device in degraded mode. If we then allocate a chunk,
+ * we may have enough free space in the existing system space_info, but
+ * none of the block groups can be used for extent allocation since they
+ * have a RAID1 profile, and because we are in degraded mode with a
+ * single device, we are forced to allocate a new system chunk with a
+ * SINGLE profile. Making check_system_chunk() iterate over all system
+ * block groups and check if they have a usable profile and enough space
+ * can be slow on very large filesystems, so we tolerate the -ENOSPC and
+ * try again after forcing allocation of a new system chunk. Like this
+ * we avoid paying the cost of that search in normal circumstances, when
+ * we were not mounted in degraded mode;
+ *
+ * 2) We had enough free space info the system space_info, and one suitable
+ * block group to allocate from when we called check_system_chunk()
+ * above. However right after we called it, the only system block group
+ * with enough free space got turned into RO mode by a running scrub,
+ * and in this case we have to allocate a new one and retry. We only
+ * need do this allocate and retry once, since we have a transaction
+ * handle and scrub uses the commit root to search for block groups.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+out:
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
+}
+
/*
- * If force is CHUNK_ALLOC_FORCE:
+ * Chunk allocation is done in 2 phases:
+ *
+ * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
+ * the chunk, the chunk mapping, create its block group and add the items
+ * that belong in the chunk btree to it - more specifically, we need to
+ * update device items in the chunk btree and add a new chunk item to it.
+ *
+ * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
+ * group item to the extent btree and the device extent items to the devices
+ * btree.
+ *
+ * This is done to prevent deadlocks. For example when COWing a node from the
+ * extent btree we are holding a write lock on the node's parent and if we
+ * trigger chunk allocation and attempted to insert the new block group item
+ * in the extent btree right way, we could deadlock because the path for the
+ * insertion can include that parent node. At first glance it seems impossible
+ * to trigger chunk allocation after starting a transaction since tasks should
+ * reserve enough transaction units (metadata space), however while that is true
+ * most of the time, chunk allocation may still be triggered for several reasons:
+ *
+ * 1) When reserving metadata, we check if there is enough free space in the
+ * metadata space_info and therefore don't trigger allocation of a new chunk.
+ * However later when the task actually tries to COW an extent buffer from
+ * the extent btree or from the device btree for example, it is forced to
+ * allocate a new block group (chunk) because the only one that had enough
+ * free space was just turned to RO mode by a running scrub for example (or
+ * device replace, block group reclaim thread, etc), so we can not use it
+ * for allocating an extent and end up being forced to allocate a new one;
+ *
+ * 2) Because we only check that the metadata space_info has enough free bytes,
+ * we end up not allocating a new metadata chunk in that case. However if
+ * the filesystem was mounted in degraded mode, none of the existing block
+ * groups might be suitable for extent allocation due to their incompatible
+ * profile (for e.g. mounting a 2 devices filesystem, where all block groups
+ * use a RAID1 profile, in degraded mode using a single device). In this case
+ * when the task attempts to COW some extent buffer of the extent btree for
+ * example, it will trigger allocation of a new metadata block group with a
+ * suitable profile (SINGLE profile in the example of the degraded mount of
+ * the RAID1 filesystem);
+ *
+ * 3) The task has reserved enough transaction units / metadata space, but when
+ * it attempts to COW an extent buffer from the extent or device btree for
+ * example, it does not find any free extent in any metadata block group,
+ * therefore forced to try to allocate a new metadata block group.
+ * This is because some other task allocated all available extents in the
+ * meanwhile - this typically happens with tasks that don't reserve space
+ * properly, either intentionally or as a bug. One example where this is
+ * done intentionally is fsync, as it does not reserve any transaction units
+ * and ends up allocating a variable number of metadata extents for log
+ * tree extent buffers.
+ *
+ * We also need this 2 phases setup when adding a device to a filesystem with
+ * a seed device - we must create new metadata and system chunks without adding
+ * any of the block group items to the chunk, extent and device btrees. If we
+ * did not do it this way, we would get ENOSPC when attempting to update those
+ * btrees, since all the chunks from the seed device are read-only.
+ *
+ * Phase 1 does the updates and insertions to the chunk btree because if we had
+ * it done in phase 2 and have a thundering herd of tasks allocating chunks in
+ * parallel, we risk having too many system chunks allocated by many tasks if
+ * many tasks reach phase 1 without the previous ones completing phase 2. In the
+ * extreme case this leads to exhaustion of the system chunk array in the
+ * superblock. This is easier to trigger if using a btree node/leaf size of 64K
+ * and with RAID filesystems (so we have more device items in the chunk btree).
+ * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
+ * the system chunk array due to concurrent allocations") provides more details.
+ *
+ * For allocation of system chunks, we defer the updates and insertions into the
+ * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
+ * if the chunk allocation is triggered while COWing an extent buffer of the
+ * chunk btree, we are holding a lock on the parent of that extent buffer and
+ * doing the chunk btree updates and insertions can require locking that parent.
+ * This is for the very few and rare cases where we update the chunk btree that
+ * are not chunk allocation or chunk removal: adding a device, removing a device
+ * or resizing a device.
+ *
+ * The reservation of system space, done through check_system_chunk(), as well
+ * as all the updates and insertions into the chunk btree must be done while
+ * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
+ * an extent buffer from the chunks btree we never trigger allocation of a new
+ * system chunk, which would result in a deadlock (trying to lock twice an
+ * extent buffer of the chunk btree, first time before triggering the chunk
+ * allocation and the second time during chunk allocation while attempting to
+ * update the chunks btree). The system chunk array is also updated while holding
+ * that mutex. The same logic applies to removing chunks - we must reserve system
+ * space, update the chunk btree and the system chunk array in the superblock
+ * while holding fs_info->chunk_mutex.
+ *
+ * This function, btrfs_chunk_alloc(), belongs to phase 1.
+ *
+ * If @force is CHUNK_ALLOC_FORCE:
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
+ * If @force is NOT CHUNK_ALLOC_FORCE:
* - return 0 if it doesn't need to allocate a new chunk,
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
@@ -3253,6 +3464,13 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
@@ -3316,13 +3534,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
force_metadata_allocation(fs_info);
}
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
+ ret = do_chunk_alloc(trans, flags);
trans->allocating_chunk = false;
spin_lock(&space_info->lock);
@@ -3341,22 +3553,6 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
space_info->chunk_alloc = 0;
spin_unlock(&space_info->lock);
mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
return ret;
}
@@ -3409,14 +3605,31 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *bg;
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
+ *
+ * Also, if our caller is allocating a system chunk, do not
+ * attempt to insert the chunk item in the chunk btree, as we
+ * could deadlock on an extent buffer since our caller may be
+ * COWing an extent buffer from the chunk btree.
*/
- ret = btrfs_alloc_chunk(trans, flags);
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ /*
+ * If we fail to add the chunk item here, we end up
+ * trying again at phase 2 of chunk allocation, at
+ * btrfs_create_pending_block_groups(). So ignore
+ * any error here.
+ */
+ btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ }
}
if (!ret) {
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 7b927425dc71..c72a71efcb18 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -97,6 +97,7 @@ struct btrfs_block_group {
unsigned int removed:1;
unsigned int to_copy:1;
unsigned int relocating_repair:1;
+ unsigned int chunk_item_inserted:1;
int disk_cache_state;
@@ -268,8 +269,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work);
void btrfs_reclaim_bgs(struct btrfs_fs_info *fs_info);
void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg);
int btrfs_read_block_groups(struct btrfs_fs_info *info);
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size);
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
bool do_chunk_alloc);
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 4bc3ca2cbd7d..c5c08c87e130 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -364,49 +364,6 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
return 0;
}
-static struct extent_buffer *alloc_tree_block_no_bg_flush(
- struct btrfs_trans_handle *trans,
- struct btrfs_root *root,
- u64 parent_start,
- const struct btrfs_disk_key *disk_key,
- int level,
- u64 hint,
- u64 empty_size,
- enum btrfs_lock_nesting nest)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct extent_buffer *ret;
-
- /*
- * If we are COWing a node/leaf from the extent, chunk, device or free
- * space trees, make sure that we do not finish block group creation of
- * pending block groups. We do this to avoid a deadlock.
- * COWing can result in allocation of a new chunk, and flushing pending
- * block groups (btrfs_create_pending_block_groups()) can be triggered
- * when finishing allocation of a new chunk. Creation of a pending block
- * group modifies the extent, chunk, device and free space trees,
- * therefore we could deadlock with ourselves since we are holding a
- * lock on an extent buffer that btrfs_create_pending_block_groups() may
- * try to COW later.
- * For similar reasons, we also need to delay flushing pending block
- * groups when splitting a leaf or node, from one of those trees, since
- * we are holding a write lock on it and its parent or when inserting a
- * new root node for one of those trees.
- */
- if (root == fs_info->extent_root ||
- root == fs_info->chunk_root ||
- root == fs_info->dev_root ||
- root == fs_info->free_space_root)
- trans->can_flush_pending_bgs = false;
-
- ret = btrfs_alloc_tree_block(trans, root, parent_start,
- root->root_key.objectid, disk_key, level,
- hint, empty_size, nest);
- trans->can_flush_pending_bgs = true;
-
- return ret;
-}
-
/*
* does the dirty work in cow of a single block. The parent block (if
* supplied) is updated to point to the new cow copy. The new buffer is marked
@@ -455,8 +412,9 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
if ((root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) && parent)
parent_start = parent->start;
- cow = alloc_tree_block_no_bg_flush(trans, root, parent_start, &disk_key,
- level, search_start, empty_size, nest);
+ cow = btrfs_alloc_tree_block(trans, root, parent_start,
+ root->root_key.objectid, &disk_key, level,
+ search_start, empty_size, nest);
if (IS_ERR(cow))
return PTR_ERR(cow);
@@ -2458,9 +2416,9 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
else
btrfs_node_key(lower, &lower_key, 0);
- c = alloc_tree_block_no_bg_flush(trans, root, 0, &lower_key, level,
- root->node->start, 0,
- BTRFS_NESTING_NEW_ROOT);
+ c = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &lower_key, level, root->node->start, 0,
+ BTRFS_NESTING_NEW_ROOT);
if (IS_ERR(c))
return PTR_ERR(c);
@@ -2589,8 +2547,9 @@ static noinline int split_node(struct btrfs_trans_handle *trans,
mid = (c_nritems + 1) / 2;
btrfs_node_key(c, &disk_key, mid);
- split = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, level,
- c->start, 0, BTRFS_NESTING_SPLIT);
+ split = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, level, c->start, 0,
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(split))
return PTR_ERR(split);
@@ -3381,10 +3340,10 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans,
* BTRFS_NESTING_SPLIT_THE_SPLITTENING if we need to, but for now just
* use BTRFS_NESTING_NEW_ROOT.
*/
- right = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, 0,
- l->start, 0, num_doubles ?
- BTRFS_NESTING_NEW_ROOT :
- BTRFS_NESTING_SPLIT);
+ right = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, 0, l->start, 0,
+ num_doubles ? BTRFS_NESTING_NEW_ROOT :
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(right))
return PTR_ERR(right);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 443c348bc6f3..14b9fdc8aaa9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -254,8 +254,11 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
}
/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
+ * To be called after doing the chunk btree updates right after allocating a new
+ * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
+ * chunk after all chunk btree updates and after finishing the second phase of
+ * chunk allocation (btrfs_create_pending_block_groups()) in case some block
+ * group had its chunk item insertion delayed to the second phase.
*/
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
@@ -264,8 +267,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
if (!trans->chunk_bytes_reserved)
return;
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
trans->chunk_bytes_reserved = 0;
@@ -696,7 +697,6 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,
h->fs_info = root->fs_info;
h->type = type;
- h->can_flush_pending_bgs = true;
INIT_LIST_HEAD(&h->new_bgs);
smp_mb();
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index a18d67796b54..ba45065f9451 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -132,7 +132,7 @@ struct btrfs_trans_handle {
short aborted;
bool adding_csums;
bool allocating_chunk;
- bool can_flush_pending_bgs;
+ bool removing_chunk;
bool reloc_reserved;
bool in_fsync;
struct btrfs_root *root;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 782e16795bc4..c6c14315b1c9 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1745,19 +1745,14 @@ static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_handle_fs_error(fs_info, ret, "Slot search failed");
goto out;
}
*dev_extent_len = btrfs_dev_extent_length(leaf, extent);
ret = btrfs_del_item(trans, root, path);
- if (ret) {
- btrfs_handle_fs_error(fs_info, ret,
- "Failed to remove dev extent item");
- } else {
+ if (ret == 0)
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
- }
out:
btrfs_free_path(path);
return ret;
@@ -2942,7 +2937,7 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
u32 cur;
struct btrfs_key key;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
array_size = btrfs_super_sys_array_size(super_copy);
ptr = super_copy->sys_chunk_array;
@@ -2972,7 +2967,6 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
cur += len;
}
}
- mutex_unlock(&fs_info->chunk_mutex);
return ret;
}
@@ -3012,6 +3006,29 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,
return em;
}
+static int remove_chunk_item(struct btrfs_trans_handle *trans,
+ struct map_lookup *map, u64 chunk_offset)
+{
+ int i;
+
+ /*
+ * Removing chunk items and updating the device items in the chunks btree
+ * requires holding the chunk_mutex.
+ * See the comment at btrfs_chunk_alloc() for the details.
+ */
+ lockdep_assert_held(&trans->fs_info->chunk_mutex);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ int ret;
+
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ if (ret)
+ return ret;
+ }
+
+ return btrfs_free_chunk(trans, chunk_offset);
+}
+
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3032,14 +3049,16 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(em);
}
map = em->map_lookup;
- mutex_lock(&fs_info->chunk_mutex);
- check_system_chunk(trans, map->type);
- mutex_unlock(&fs_info->chunk_mutex);
/*
- * Take the device list mutex to prevent races with the final phase of
- * a device replace operation that replaces the device object associated
- * with map stripes (dev-replace.c:btrfs_dev_replace_finishing()).
+ * First delete the device extent items from the devices btree.
+ * We take the device_list_mutex to avoid racing with the finishing phase
+ * of a device replace operation. See the comment below before acquiring
+ * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
+ * because that can result in a deadlock when deleting the device extent
+ * items from the devices btree - COWing an extent buffer from the btree
+ * may result in allocating a new metadata chunk, which would attempt to
+ * lock again fs_info->chunk_mutex.
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
@@ -3061,18 +3080,73 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
btrfs_clear_space_info_full(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
}
+ }
+ mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_update_device(trans, device);
+ /*
+ * We acquire fs_info->chunk_mutex for 2 reasons:
+ *
+ * 1) Just like with the first phase of the chunk allocation, we must
+ * reserve system space, do all chunk btree updates and deletions, and
+ * update the system chunk array in the superblock while holding this
+ * mutex. This is for similar reasons as explained on the comment at
+ * the top of btrfs_chunk_alloc();
+ *
+ * 2) Prevent races with the final phase of a device replace operation
+ * that replaces the device object associated with the map's stripes,
+ * because the device object's id can change at any time during that
+ * final phase of the device replace operation
+ * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
+ * replaced device and then see it with an ID of
+ * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
+ * the device item, which does not exists on the chunk btree.
+ * The finishing phase of device replace acquires both the
+ * device_list_mutex and the chunk_mutex, in that order, so we are
+ * safe by just acquiring the chunk_mutex.
+ */
+ trans->removing_chunk = true;
+ mutex_lock(&fs_info->chunk_mutex);
+
+ check_system_chunk(trans, map->type);
+
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ /*
+ * Normally we should not get -ENOSPC since we reserved space before
+ * through the call to check_system_chunk().
+ *
+ * Despite our system space_info having enough free space, we may not
+ * be able to allocate extents from its block groups, because all have
+ * an incompatible profile, which will force us to allocate a new system
+ * block group with the right profile, or right after we called
+ * check_system_space() above, a scrub turned the only system block group
+ * with enough free space into RO mode.
+ * This is explained with more detail at do_chunk_alloc().
+ *
+ * So if we get -ENOSPC, allocate a new system chunk and retry once.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
goto out;
}
- }
- mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_free_chunk(trans, chunk_offset);
- if (ret) {
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -3087,6 +3161,15 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
}
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+
+ /*
+ * We are done with chunk btree updates and deletions, so release the
+ * system space we previously reserved (with check_system_chunk()).
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+
ret = btrfs_remove_block_group(trans, chunk_offset, em);
if (ret) {
btrfs_abort_transaction(trans, ret);
@@ -3094,6 +3177,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
out:
+ if (trans->removing_chunk) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ }
/* once for us */
free_extent_map(em);
return ret;
@@ -4860,13 +4947,12 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
u32 array_size;
u8 *ptr;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
+
array_size = btrfs_super_sys_array_size(super_copy);
if (array_size + item_size + sizeof(disk_key)
- > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
- mutex_unlock(&fs_info->chunk_mutex);
+ > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
return -EFBIG;
- }
ptr = super_copy->sys_chunk_array + array_size;
btrfs_cpu_key_to_disk(&disk_key, key);
@@ -4875,7 +4961,6 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
memcpy(ptr, chunk, item_size);
item_size += sizeof(disk_key);
btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
- mutex_unlock(&fs_info->chunk_mutex);
return 0;
}
@@ -5225,13 +5310,14 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
}
}
-static int create_chunk(struct btrfs_trans_handle *trans,
+static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
{
struct btrfs_fs_info *info = trans->fs_info;
struct map_lookup *map = NULL;
struct extent_map_tree *em_tree;
+ struct btrfs_block_group *block_group;
struct extent_map *em;
u64 start = ctl->start;
u64 type = ctl->type;
@@ -5241,7 +5327,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
map = kmalloc(map_lookup_size(ctl->num_stripes), GFP_NOFS);
if (!map)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
map->num_stripes = ctl->num_stripes;
for (i = 0; i < ctl->ndevs; ++i) {
@@ -5263,7 +5349,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
em = alloc_extent_map();
if (!em) {
kfree(map);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
set_bit(EXTENT_FLAG_FS_MAPPING, &em->flags);
em->map_lookup = map;
@@ -5279,12 +5365,12 @@ static int create_chunk(struct btrfs_trans_handle *trans,
if (ret) {
write_unlock(&em_tree->lock);
free_extent_map(em);
- return ret;
+ return ERR_PTR(ret);
}
write_unlock(&em_tree->lock);
- ret = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
- if (ret)
+ block_group = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
+ if (IS_ERR(block_group))
goto error_del_extent;
for (i = 0; i < map->num_stripes; i++) {
@@ -5304,7 +5390,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
check_raid56_incompat_flag(info, type);
check_raid1c34_incompat_flag(info, type);
- return 0;
+ return block_group;
error_del_extent:
write_lock(&em_tree->lock);
@@ -5316,34 +5402,36 @@ static int create_chunk(struct btrfs_trans_handle *trans,
/* One for the tree reference */
free_extent_map(em);
- return ret;
+ return block_group;
}
-int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
+struct btrfs_block_group *btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ u64 type)
{
struct btrfs_fs_info *info = trans->fs_info;
struct btrfs_fs_devices *fs_devices = info->fs_devices;
struct btrfs_device_info *devices_info = NULL;
struct alloc_chunk_ctl ctl;
+ struct btrfs_block_group *block_group;
int ret;
lockdep_assert_held(&info->chunk_mutex);
if (!alloc_profile_is_valid(type, 0)) {
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (list_empty(&fs_devices->alloc_list)) {
if (btrfs_test_opt(info, ENOSPC_DEBUG))
btrfs_debug(info, "%s: no writable device", __func__);
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
}
if (!(type & BTRFS_BLOCK_GROUP_TYPE_MASK)) {
btrfs_err(info, "invalid chunk type 0x%llx requested", type);
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
ctl.start = find_next_chunk(info);
@@ -5353,46 +5441,43 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
devices_info = kcalloc(fs_devices->rw_devices, sizeof(*devices_info),
GFP_NOFS);
if (!devices_info)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = gather_device_info(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
ret = decide_stripe_size(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
- ret = create_chunk(trans, &ctl, devices_info);
+ block_group = create_chunk(trans, &ctl, devices_info);
out:
kfree(devices_info);
- return ret;
+ return block_group;
}
/*
- * Chunk allocation falls into two parts. The first part does work
]
]
|
| 150/4 |
2026/08/08 03:47 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 53416
output: 26
thoughts: 4728
Thoughts: **Deep Dive into Btrfs Chunk Removal Logic**
Okay, this is intriguing. This commit, `79bd37120b14`, radically changed the chunk removal process in Btrfs, specifically to mitigate system chunk array exhaustion. I'm focusing on the change in `btrfs_remove_chunk`. Before the commit, `check_system_chunk` was called *before* `btrfs_remove_dev_extents`. The revised code does the opposite and it took some digging to understand *why*.
The key seems to be managing reserved system space and avoiding deadlocks. Initially, my thinking was, "Well, *why* move it? Reserve the space first, right?". But that's where the nuance kicks in. The problem is holding a reservation for too long. If `check_system_chunk` is called before `btrfs_remove_dev_extents`, we reserve system space. Then `btrfs_remove_dev_extents` runs, which can take a considerable amount of time because it involves modifying the device tree.
The crucial point is that this lengthy operation can trigger a *new* metadata chunk allocation by a secondary transaction, which calls `btrfs_chunk_alloc`. If the reservation from the *first* transaction is held during the whole process, it leads to system space being held unnecessarily, possibly causing exhaustion problems, or, more importantly, deadlock scenarios.
The original intent of moving `check_system_chunk` was to prevent this, and here's why. `btrfs_remove_dev_extents` is called, and is responsible for removing device extents from the device tree and because of this, it can trigger a new chunk allocation in the secondary transaction. If this happens, a new chunk allocation will be needed, which will also need system space and updates to the chunk btree.
The solution is intricate. The commit cleverly introduces the `trans->removing_chunk = true` flag. It is vital because if it were set too early (before `btrfs_remove_dev_extents`), `btrfs_chunk_alloc` would immediately return `-ENOSPC` when called by `btrfs_remove_dev_extents`. This would prevent needed allocations. If set before, there are numerous issues and a potentially much harder to debug issue.
The revised logic:
1. Device extents are removed. Chunk allocation can occur.
2. `trans->removing_chunk = true` is set *before* calling `check_system_chunk` to ensure no conflicting chunk operations are done, and system space is reserved at the last possible minute.
Now, by calling `check_system_chunk` after the device tree operation, we ensure the system space is checked *just* before we modify the chunk btree, thereby making our reservation "fresh". Also, since `check_system_chunk` is in the same critical section as `remove_chunk_item`, this means system space is reserved when it's absolutely needed. The commit ensures it's safe to run while `trans->removing_chunk == true` because `check_system_chunk` is in the `chunk_mutex` critical section.
This design avoids reserving system space *unnecessarily* long and avoids a deadlock in chunk allocation. I now fully understand the brilliance of setting `trans->removing_chunk` *after* `btrfs_remove_dev_extents`. It is incredibly subtle and absolutely necessary. I'll need to remember this pattern for future development in this area!
|
| 151/4 |
2026/08/08 03:47 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1163 lines.
[commit e3e0520b32bc3dbc64110536d171bfb334ac7a2a
Author: Josef Bacik <josef@toxicpanda.com>
Date: Thu Jun 20 15:37:55 2019 -0400
btrfs: migrate the block group removal code
This is the removal code and the unused bgs code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: David Sterba <dsterba@suse.com>
[ refresh, move clear_incompat_bg_bits ]
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 643a2f16603b..a27f814b86bd 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -6,6 +6,10 @@
#include "disk-io.h"
#include "free-space-cache.h"
#include "free-space-tree.h"
+#include "disk-io.h"
+#include "volumes.h"
+#include "transaction.h"
+#include "ref-verify.h"
void btrfs_get_block_group(struct btrfs_block_group_cache *cache)
{
@@ -660,3 +664,539 @@ int btrfs_cache_block_group(struct btrfs_block_group_cache *cache,
return ret;
}
+
+static void clear_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
+{
+ u64 extra_flags = chunk_to_extended(flags) &
+ BTRFS_EXTENDED_PROFILE_MASK;
+
+ write_seqlock(&fs_info->profiles_lock);
+ if (flags & BTRFS_BLOCK_GROUP_DATA)
+ fs_info->avail_data_alloc_bits &= ~extra_flags;
+ if (flags & BTRFS_BLOCK_GROUP_METADATA)
+ fs_info->avail_metadata_alloc_bits &= ~extra_flags;
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ fs_info->avail_system_alloc_bits &= ~extra_flags;
+ write_sequnlock(&fs_info->profiles_lock);
+}
+
+/*
+ * Clear incompat bits for the following feature(s):
+ *
+ * - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
+ * in the whole filesystem
+ */
+static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
+{
+ if (flags & BTRFS_BLOCK_GROUP_RAID56_MASK) {
+ struct list_head *head = &fs_info->space_info;
+ struct btrfs_space_info *sinfo;
+
+ list_for_each_entry_rcu(sinfo, head, list) {
+ bool found = false;
+
+ down_read(&sinfo->groups_sem);
+ if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
+ found = true;
+ if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
+ found = true;
+ up_read(&sinfo->groups_sem);
+
+ if (found)
+ return;
+ }
+ btrfs_clear_fs_incompat(fs_info, RAID56);
+ }
+}
+
+int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
+ u64 group_start, struct extent_map *em)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_root *root = fs_info->extent_root;
+ struct btrfs_path *path;
+ struct btrfs_block_group_cache *block_group;
+ struct btrfs_free_cluster *cluster;
+ struct btrfs_root *tree_root = fs_info->tree_root;
+ struct btrfs_key key;
+ struct inode *inode;
+ struct kobject *kobj = NULL;
+ int ret;
+ int index;
+ int factor;
+ struct btrfs_caching_control *caching_ctl = NULL;
+ bool remove_em;
+ bool remove_rsv = false;
+
+ block_group = btrfs_lookup_block_group(fs_info, group_start);
+ BUG_ON(!block_group);
+ BUG_ON(!block_group->ro);
+
+ trace_btrfs_remove_block_group(block_group);
+ /*
+ * Free the reserved super bytes from this block group before
+ * remove it.
+ */
+ btrfs_free_excluded_extents(block_group);
+ btrfs_free_ref_tree_range(fs_info, block_group->key.objectid,
+ block_group->key.offset);
+
+ memcpy(&key, &block_group->key, sizeof(key));
+ index = btrfs_bg_flags_to_raid_index(block_group->flags);
+ factor = btrfs_bg_type_to_factor(block_group->flags);
+
+ /* make sure this block group isn't part of an allocation cluster */
+ cluster = &fs_info->data_alloc_cluster;
+ spin_lock(&cluster->refill_lock);
+ btrfs_return_cluster_to_free_space(block_group, cluster);
+ spin_unlock(&cluster->refill_lock);
+
+ /*
+ * make sure this block group isn't part of a metadata
+ * allocation cluster
+ */
+ cluster = &fs_info->meta_alloc_cluster;
+ spin_lock(&cluster->refill_lock);
+ btrfs_return_cluster_to_free_space(block_group, cluster);
+ spin_unlock(&cluster->refill_lock);
+
+ path = btrfs_alloc_path();
+ if (!path) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ /*
+ * get the inode first so any iput calls done for the io_list
+ * aren't the final iput (no unlinks allowed now)
+ */
+ inode = lookup_free_space_inode(block_group, path);
+
+ mutex_lock(&trans->transaction->cache_write_mutex);
+ /*
+ * Make sure our free space cache IO is done before removing the
+ * free space inode
+ */
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ if (!list_empty(&block_group->io_list)) {
+ list_del_init(&block_group->io_list);
+
+ WARN_ON(!IS_ERR(inode) && inode != block_group->io_ctl.inode);
+
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+ btrfs_wait_cache_io(trans, block_group, path);
+ btrfs_put_block_group(block_group);
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ }
+
+ if (!list_empty(&block_group->dirty_list)) {
+ list_del_init(&block_group->dirty_list);
+ remove_rsv = true;
+ btrfs_put_block_group(block_group);
+ }
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+ mutex_unlock(&trans->transaction->cache_write_mutex);
+
+ if (!IS_ERR(inode)) {
+ ret = btrfs_orphan_add(trans, BTRFS_I(inode));
+ if (ret) {
+ btrfs_add_delayed_iput(inode);
+ goto out;
+ }
+ clear_nlink(inode);
+ /* One for the block groups ref */
+ spin_lock(&block_group->lock);
+ if (block_group->iref) {
+ block_group->iref = 0;
+ block_group->inode = NULL;
+ spin_unlock(&block_group->lock);
+ iput(inode);
+ } else {
+ spin_unlock(&block_group->lock);
+ }
+ /* One for our lookup ref */
+ btrfs_add_delayed_iput(inode);
+ }
+
+ key.objectid = BTRFS_FREE_SPACE_OBJECTID;
+ key.offset = block_group->key.objectid;
+ key.type = 0;
+
+ ret = btrfs_search_slot(trans, tree_root, &key, path, -1, 1);
+ if (ret < 0)
+ goto out;
+ if (ret > 0)
+ btrfs_release_path(path);
+ if (ret == 0) {
+ ret = btrfs_del_item(trans, tree_root, path);
+ if (ret)
+ goto out;
+ btrfs_release_path(path);
+ }
+
+ spin_lock(&fs_info->block_group_cache_lock);
+ rb_erase(&block_group->cache_node,
+ &fs_info->block_group_cache_tree);
+ RB_CLEAR_NODE(&block_group->cache_node);
+
+ if (fs_info->first_logical_byte == block_group->key.objectid)
+ fs_info->first_logical_byte = (u64)-1;
+ spin_unlock(&fs_info->block_group_cache_lock);
+
+ down_write(&block_group->space_info->groups_sem);
+ /*
+ * we must use list_del_init so people can check to see if they
+ * are still on the list after taking the semaphore
+ */
+ list_del_init(&block_group->list);
+ if (list_empty(&block_group->space_info->block_groups[index])) {
+ kobj = block_group->space_info->block_group_kobjs[index];
+ block_group->space_info->block_group_kobjs[index] = NULL;
+ clear_avail_alloc_bits(fs_info, block_group->flags);
+ }
+ up_write(&block_group->space_info->groups_sem);
+ clear_incompat_bg_bits(fs_info, block_group->flags);
+ if (kobj) {
+ kobject_del(kobj);
+ kobject_put(kobj);
+ }
+
+ if (block_group->has_caching_ctl)
+ caching_ctl = btrfs_get_caching_control(block_group);
+ if (block_group->cached == BTRFS_CACHE_STARTED)
+ btrfs_wait_block_group_cache_done(block_group);
+ if (block_group->has_caching_ctl) {
+ down_write(&fs_info->commit_root_sem);
+ if (!caching_ctl) {
+ struct btrfs_caching_control *ctl;
+
+ list_for_each_entry(ctl,
+ &fs_info->caching_block_groups, list)
+ if (ctl->block_group == block_group) {
+ caching_ctl = ctl;
+ refcount_inc(&caching_ctl->count);
+ break;
+ }
+ }
+ if (caching_ctl)
+ list_del_init(&caching_ctl->list);
+ up_write(&fs_info->commit_root_sem);
+ if (caching_ctl) {
+ /* Once for the caching bgs list and once for us. */
+ btrfs_put_caching_control(caching_ctl);
+ btrfs_put_caching_control(caching_ctl);
+ }
+ }
+
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ WARN_ON(!list_empty(&block_group->dirty_list));
+ WARN_ON(!list_empty(&block_group->io_list));
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+
+ btrfs_remove_free_space_cache(block_group);
+
+ spin_lock(&block_group->space_info->lock);
+ list_del_init(&block_group->ro_list);
+
+ if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ WARN_ON(block_group->space_info->total_bytes
+ < block_group->key.offset);
+ WARN_ON(block_group->space_info->bytes_readonly
+ < block_group->key.offset);
+ WARN_ON(block_group->space_info->disk_total
+ < block_group->key.offset * factor);
+ }
+ block_group->space_info->total_bytes -= block_group->key.offset;
+ block_group->space_info->bytes_readonly -= block_group->key.offset;
+ block_group->space_info->disk_total -= block_group->key.offset * factor;
+
+ spin_unlock(&block_group->space_info->lock);
+
+ memcpy(&key, &block_group->key, sizeof(key));
+
+ mutex_lock(&fs_info->chunk_mutex);
+ spin_lock(&block_group->lock);
+ block_group->removed = 1;
+ /*
+ * At this point trimming can't start on this block group, because we
+ * removed the block group from the tree fs_info->block_group_cache_tree
+ * so no one can't find it anymore and even if someone already got this
+ * block group before we removed it from the rbtree, they have already
+ * incremented block_group->trimming - if they didn't, they won't find
+ * any free space entries because we already removed them all when we
+ * called btrfs_remove_free_space_cache().
+ *
+ * And we must not remove the extent map from the fs_info->mapping_tree
+ * to prevent the same logical address range and physical device space
+ * ranges from being reused for a new block group. This is because our
+ * fs trim operation (btrfs_trim_fs() / btrfs_ioctl_fitrim()) is
+ * completely transactionless, so while it is trimming a range the
+ * currently running transaction might finish and a new one start,
+ * allowing for new block groups to be created that can reuse the same
+ * physical device locations unless we take this special care.
+ *
+ * There may also be an implicit trim operation if the file system
+ * is mounted with -odiscard. The same protections must remain
+ * in place until the extents have been discarded completely when
+ * the transaction commit has completed.
+ */
+ remove_em = (atomic_read(&block_group->trimming) == 0);
+ spin_unlock(&block_group->lock);
+
+ mutex_unlock(&fs_info->chunk_mutex);
+
+ ret = remove_block_group_free_space(trans, block_group);
+ if (ret)
+ goto out;
+
+ btrfs_put_block_group(block_group);
+ btrfs_put_block_group(block_group);
+
+ ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ if (ret > 0)
+ ret = -EIO;
+ if (ret < 0)
+ goto out;
+
+ ret = btrfs_del_item(trans, root, path);
+ if (ret)
+ goto out;
+
+ if (remove_em) {
+ struct extent_map_tree *em_tree;
+
+ em_tree = &fs_info->mapping_tree;
+ write_lock(&em_tree->lock);
+ remove_extent_mapping(em_tree, em);
+ write_unlock(&em_tree->lock);
+ /* once for the tree */
+ free_extent_map(em);
+ }
+out:
+ if (remove_rsv)
+ btrfs_delayed_refs_rsv_release(fs_info, 1);
+ btrfs_free_path(path);
+ return ret;
+}
+
+struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
+ struct btrfs_fs_info *fs_info, const u64 chunk_offset)
+{
+ struct extent_map_tree *em_tree = &fs_info->mapping_tree;
+ struct extent_map *em;
+ struct map_lookup *map;
+ unsigned int num_items;
+
+ read_lock(&em_tree->lock);
+ em = lookup_extent_mapping(em_tree, chunk_offset, 1);
+ read_unlock(&em_tree->lock);
+ ASSERT(em && em->start == chunk_offset);
+
+ /*
+ * We need to reserve 3 + N units from the metadata space info in order
+ * to remove a block group (done at btrfs_remove_chunk() and at
+ * btrfs_remove_block_group()), which are used for:
+ *
+ * 1 unit for adding the free space inode's orphan (located in the tree
+ * of tree roots).
+ * 1 unit for deleting the block group item (located in the extent
+ * tree).
+ * 1 unit for deleting the free space item (located in tree of tree
+ * roots).
+ * N units for deleting N device extent items corresponding to each
+ * stripe (located in the device tree).
+ *
+ * In order to remove a block group we also need to reserve units in the
+ * system space info in order to update the chunk tree (update one or
+ * more device items and remove one chunk item), but this is done at
+ * btrfs_remove_chunk() through a call to check_system_chunk().
+ */
+ map = em->map_lookup;
+ num_items = 3 + map->num_stripes;
+ free_extent_map(em);
+
+ return btrfs_start_transaction_fallback_global_rsv(fs_info->extent_root,
+ num_items, 1);
+}
+
+/*
+ * Process the unused_bgs list and remove any that don't have any allocated
+ * space inside of them.
+ */
+void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_block_group_cache *block_group;
+ struct btrfs_space_info *space_info;
+ struct btrfs_trans_handle *trans;
+ int ret = 0;
+
+ if (!test_bit(BTRFS_FS_OPEN, &fs_info->flags))
+ return;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ while (!list_empty(&fs_info->unused_bgs)) {
+ u64 start, end;
+ int trimming;
+
+ block_group = list_first_entry(&fs_info->unused_bgs,
+ struct btrfs_block_group_cache,
+ bg_list);
+ list_del_init(&block_group->bg_list);
+
+ space_info = block_group->space_info;
+
+ if (ret || btrfs_mixed_space_info(space_info)) {
+ btrfs_put_block_group(block_group);
+ continue;
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+
+ mutex_lock(&fs_info->delete_unused_bgs_mutex);
+
+ /* Don't want to race with allocators so take the groups_sem */
+ down_write(&space_info->groups_sem);
+ spin_lock(&block_group->lock);
+ if (block_group->reserved || block_group->pinned ||
+ btrfs_block_group_used(&block_group->item) ||
+ block_group->ro ||
+ list_is_singular(&block_group->list)) {
+ /*
+ * We want to bail if we made new allocations or have
+ * outstanding allocations in this block group. We do
+ * the ro check in case balance is currently acting on
+ * this block group.
+ */
+ trace_btrfs_skip_unused_block_group(block_group);
+ spin_unlock(&block_group->lock);
+ up_write(&space_info->groups_sem);
+ goto next;
+ }
+ spin_unlock(&block_group->lock);
+
+ /* We don't want to force the issue, only flip if it's ok. */
+ ret = __btrfs_inc_block_group_ro(block_group, 0);
+ up_write(&space_info->groups_sem);
+ if (ret < 0) {
+ ret = 0;
+ goto next;
+ }
+
+ /*
+ * Want to do this before we do anything else so we can recover
+ * properly if we fail to join the transaction.
+ */
+ trans = btrfs_start_trans_remove_block_group(fs_info,
+ block_group->key.objectid);
+ if (IS_ERR(trans)) {
+ btrfs_dec_block_group_ro(block_group);
+ ret = PTR_ERR(trans);
+ goto next;
+ }
+
+ /*
+ * We could have pending pinned extents for this block group,
+ * just delete them, we don't care about them anymore.
+ */
+ start = block_group->key.objectid;
+ end = start + block_group->key.offset - 1;
+ /*
+ * Hold the unused_bg_unpin_mutex lock to avoid racing with
+ * btrfs_finish_extent_commit(). If we are at transaction N,
+ * another task might be running finish_extent_commit() for the
+ * previous transaction N - 1, and have seen a range belonging
+ * to the block group in freed_extents[] before we were able to
+ * clear the whole block group range from freed_extents[]. This
+ * means that task can lookup for the block group after we
+ * unpinned it from freed_extents[] and removed it, leading to
+ * a BUG_ON() at btrfs_unpin_extent_range().
+ */
+ mutex_lock(&fs_info->unused_bg_unpin_mutex);
+ ret = clear_extent_bits(&fs_info->freed_extents[0], start, end,
+ EXTENT_DIRTY);
+ if (ret) {
+ mutex_unlock(&fs_info->unused_bg_unpin_mutex);
+ btrfs_dec_block_group_ro(block_group);
+ goto end_trans;
+ }
+ ret = clear_extent_bits(&fs_info->freed_extents[1], start, end,
+ EXTENT_DIRTY);
+ if (ret) {
+ mutex_unlock(&fs_info->unused_bg_unpin_mutex);
+ btrfs_dec_block_group_ro(block_group);
+ goto end_trans;
+ }
+ mutex_unlock(&fs_info->unused_bg_unpin_mutex);
+
+ /* Reset pinned so btrfs_put_block_group doesn't complain */
+ spin_lock(&space_info->lock);
+ spin_lock(&block_group->lock);
+
+ btrfs_space_info_update_bytes_pinned(fs_info, space_info,
+ -block_group->pinned);
+ space_info->bytes_readonly += block_group->pinned;
+ percpu_counter_add_batch(&space_info->total_bytes_pinned,
+ -block_group->pinned,
+ BTRFS_TOTAL_BYTES_PINNED_BATCH);
+ block_group->pinned = 0;
+
+ spin_unlock(&block_group->lock);
+ spin_unlock(&space_info->lock);
+
+ /* DISCARD can flip during remount */
+ trimming = btrfs_test_opt(fs_info, DISCARD);
+
+ /* Implicit trim during transaction commit. */
+ if (trimming)
+ btrfs_get_block_group_trimming(block_group);
+
+ /*
+ * Btrfs_remove_chunk will abort the transaction if things go
+ * horribly wrong.
+ */
+ ret = btrfs_remove_chunk(trans, block_group->key.objectid);
+
+ if (ret) {
+ if (trimming)
+ btrfs_put_block_group_trimming(block_group);
+ goto end_trans;
+ }
+
+ /*
+ * If we're not mounted with -odiscard, we can just forget
+ * about this block group. Otherwise we'll need to wait
+ * until transaction commit to do the actual discard.
+ */
+ if (trimming) {
+ spin_lock(&fs_info->unused_bgs_lock);
+ /*
+ * A concurrent scrub might have added us to the list
+ * fs_info->unused_bgs, so use a list_move operation
+ * to add the block group to the deleted_bgs list.
+ */
+ list_move(&block_group->bg_list,
+ &trans->transaction->deleted_bgs);
+ spin_unlock(&fs_info->unused_bgs_lock);
+ btrfs_get_block_group(block_group);
+ }
+end_trans:
+ btrfs_end_transaction(trans);
+next:
+ mutex_unlock(&fs_info->delete_unused_bgs_mutex);
+ btrfs_put_block_group(block_group);
+ spin_lock(&fs_info->unused_bgs_lock);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
+
+void btrfs_mark_bg_unused(struct btrfs_block_group_cache *bg)
+{
+ struct btrfs_fs_info *fs_info = bg->fs_info;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ if (list_empty(&bg->bg_list)) {
+ btrfs_get_block_group(bg);
+ trace_btrfs_add_unused_block_group(bg);
+ list_add_tail(&bg->bg_list, &fs_info->unused_bgs);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 143baaa54684..f1fe14ba2702 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -176,6 +176,13 @@ struct btrfs_caching_control *btrfs_get_caching_control(
struct btrfs_block_group_cache *cache);
u64 add_new_free_space(struct btrfs_block_group_cache *block_group,
u64 start, u64 end);
+struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
+ struct btrfs_fs_info *fs_info,
+ const u64 chunk_offset);
+int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
+ u64 group_start, struct extent_map *em);
+void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info);
+void btrfs_mark_bg_unused(struct btrfs_block_group_cache *bg);
static inline int btrfs_block_group_cache_done(
struct btrfs_block_group_cache *cache)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 17eb4c91f0e1..aedee3f66764 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2532,12 +2532,6 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info);
int btrfs_make_block_group(struct btrfs_trans_handle *trans,
u64 bytes_used, u64 type, u64 chunk_offset,
u64 size);
-struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
- struct btrfs_fs_info *fs_info,
- const u64 chunk_offset);
-int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
- u64 group_start, struct extent_map *em);
-void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info);
void btrfs_get_block_group_trimming(struct btrfs_block_group_cache *cache);
void btrfs_put_block_group_trimming(struct btrfs_block_group_cache *cache);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
@@ -2618,7 +2612,6 @@ int btrfs_start_write_no_snapshotting(struct btrfs_root *root);
void btrfs_end_write_no_snapshotting(struct btrfs_root *root);
void btrfs_wait_for_snapshot_creation(struct btrfs_root *root);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
-void btrfs_mark_bg_unused(struct btrfs_block_group_cache *bg);
/* ctree.c */
int btrfs_bin_search(struct extent_buffer *eb, const struct btrfs_key *key,
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 08bd67169590..775d78a101b0 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -7501,530 +7501,6 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
return 0;
}
-static void clear_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
-{
- u64 extra_flags = chunk_to_extended(flags) &
- BTRFS_EXTENDED_PROFILE_MASK;
-
- write_seqlock(&fs_info->profiles_lock);
- if (flags & BTRFS_BLOCK_GROUP_DATA)
- fs_info->avail_data_alloc_bits &= ~extra_flags;
- if (flags & BTRFS_BLOCK_GROUP_METADATA)
- fs_info->avail_metadata_alloc_bits &= ~extra_flags;
- if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
- fs_info->avail_system_alloc_bits &= ~extra_flags;
- write_sequnlock(&fs_info->profiles_lock);
-}
-
-/*
- * Clear incompat bits for the following feature(s):
- *
- * - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
- * in the whole filesystem
- */
-static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
-{
- if (flags & BTRFS_BLOCK_GROUP_RAID56_MASK) {
- struct list_head *head = &fs_info->space_info;
- struct btrfs_space_info *sinfo;
-
- list_for_each_entry_rcu(sinfo, head, list) {
- bool found = false;
-
- down_read(&sinfo->groups_sem);
- if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
- found = true;
- if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
- found = true;
- up_read(&sinfo->groups_sem);
-
- if (found)
- return;
- }
- btrfs_clear_fs_incompat(fs_info, RAID56);
- }
-}
-
-int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
- u64 group_start, struct extent_map *em)
-{
- struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_root *root = fs_info->extent_root;
- struct btrfs_path *path;
- struct btrfs_block_group_cache *block_group;
- struct btrfs_free_cluster *cluster;
- struct btrfs_root *tree_root = fs_info->tree_root;
- struct btrfs_key key;
- struct inode *inode;
- struct kobject *kobj = NULL;
- int ret;
- int index;
- int factor;
- struct btrfs_caching_control *caching_ctl = NULL;
- bool remove_em;
- bool remove_rsv = false;
-
- block_group = btrfs_lookup_block_group(fs_info, group_start);
- BUG_ON(!block_group);
- BUG_ON(!block_group->ro);
-
- trace_btrfs_remove_block_group(block_group);
- /*
- * Free the reserved super bytes from this block group before
- * remove it.
- */
- btrfs_free_excluded_extents(block_group);
- btrfs_free_ref_tree_range(fs_info, block_group->key.objectid,
- block_group->key.offset);
-
- memcpy(&key, &block_group->key, sizeof(key));
- index = btrfs_bg_flags_to_raid_index(block_group->flags);
- factor = btrfs_bg_type_to_factor(block_group->flags);
-
- /* make sure this block group isn't part of an allocation cluster */
- cluster = &fs_info->data_alloc_cluster;
- spin_lock(&cluster->refill_lock);
- btrfs_return_cluster_to_free_space(block_group, cluster);
- spin_unlock(&cluster->refill_lock);
-
- /*
- * make sure this block group isn't part of a metadata
- * allocation cluster
- */
- cluster = &fs_info->meta_alloc_cluster;
- spin_lock(&cluster->refill_lock);
- btrfs_return_cluster_to_free_space(block_group, cluster);
- spin_unlock(&cluster->refill_lock);
-
- path = btrfs_alloc_path();
- if (!path) {
- ret = -ENOMEM;
- goto out;
- }
-
- /*
- * get the inode first so any iput calls done for the io_list
- * aren't the final iput (no unlinks allowed now)
- */
- inode = lookup_free_space_inode(block_group, path);
-
- mutex_lock(&trans->transaction->cache_write_mutex);
- /*
- * Make sure our free space cache IO is done before removing the
- * free space inode
- */
- spin_lock(&trans->transaction->dirty_bgs_lock);
- if (!list_empty(&block_group->io_list)) {
- list_del_init(&block_group->io_list);
-
- WARN_ON(!IS_ERR(inode) && inode != block_group->io_ctl.inode);
-
- spin_unlock(&trans->transaction->dirty_bgs_lock);
- btrfs_wait_cache_io(trans, block_group, path);
- btrfs_put_block_group(block_group);
- spin_lock(&trans->transaction->dirty_bgs_lock);
- }
-
- if (!list_empty(&block_group->dirty_list)) {
- list_del_init(&block_group->dirty_list);
- remove_rsv = true;
- btrfs_put_block_group(block_group);
- }
- spin_unlock(&trans->transaction->dirty_bgs_lock);
- mutex_unlock(&trans->transaction->cache_write_mutex);
-
- if (!IS_ERR(inode)) {
- ret = btrfs_orphan_add(trans, BTRFS_I(inode));
- if (ret) {
- btrfs_add_delayed_iput(inode);
- goto out;
- }
- clear_nlink(inode);
- /* One for the block groups ref */
- spin_lock(&block_group->lock);
- if (block_group->iref) {
- block_group->iref = 0;
- block_group->inode = NULL;
- spin_unlock(&block_group->lock);
- iput(inode);
- } else {
- spin_unlock(&block_group->lock);
- }
- /* One for our lookup ref */
- btrfs_add_delayed_iput(inode);
- }
-
- key.objectid = BTRFS_FREE_SPACE_OBJECTID;
- key.offset = block_group->key.objectid;
- key.type = 0;
-
- ret = btrfs_search_slot(trans, tree_root, &key, path, -1, 1);
- if (ret < 0)
- goto out;
- if (ret > 0)
- btrfs_release_path(path);
- if (ret == 0) {
- ret = btrfs_del_item(trans, tree_root, path);
- if (ret)
- goto out;
- btrfs_release_path(path);
- }
-
- spin_lock(&fs_info->block_group_cache_lock);
- rb_erase(&block_group->cache_node,
- &fs_info->block_group_cache_tree);
- RB_CLEAR_NODE(&block_group->cache_node);
-
- if (fs_info->first_logical_byte == block_group->key.objectid)
- fs_info->first_logical_byte = (u64)-1;
- spin_unlock(&fs_info->block_group_cache_lock);
-
- down_write(&block_group->space_info->groups_sem);
- /*
- * we must use list_del_init so people can check to see if they
- * are still on the list after taking the semaphore
- */
- list_del_init(&block_group->list);
- if (list_empty(&block_group->space_info->block_groups[index])) {
- kobj = block_group->space_info->block_group_kobjs[index];
- block_group->space_info->block_group_kobjs[index] = NULL;
- clear_avail_alloc_bits(fs_info, block_group->flags);
- }
- up_write(&block_group->space_info->groups_sem);
- clear_incompat_bg_bits(fs_info, block_group->flags);
- if (kobj) {
- kobject_del(kobj);
- kobject_put(kobj);
- }
-
- if (block_group->has_caching_ctl)
- caching_ctl = btrfs_get_caching_control(block_group);
- if (block_group->cached == BTRFS_CACHE_STARTED)
- btrfs_wait_block_group_cache_done(block_group);
- if (block_group->has_caching_ctl) {
- down_write(&fs_info->commit_root_sem);
- if (!caching_ctl) {
- struct btrfs_caching_control *ctl;
-
- list_for_each_entry(ctl,
- &fs_info->caching_block_groups, list)
- if (ctl->block_group == block_group) {
- caching_ctl = ctl;
- refcount_inc(&caching_ctl->count);
- break;
- }
- }
- if (caching_ctl)
- list_del_init(&caching_ctl->list);
- up_write(&fs_info->commit_root_sem);
- if (caching_ctl) {
- /* Once for the caching bgs list and once for us. */
- btrfs_put_caching_control(caching_ctl);
- btrfs_put_caching_control(caching_ctl);
- }
- }
-
- spin_lock(&trans->transaction->dirty_bgs_lock);
- WARN_ON(!list_empty(&block_group->dirty_list));
- WARN_ON(!list_empty(&block_group->io_list));
- spin_unlock(&trans->transaction->dirty_bgs_lock);
-
- btrfs_remove_free_space_cache(block_group);
-
- spin_lock(&block_group->space_info->lock);
- list_del_init(&block_group->ro_list);
-
- if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
- WARN_ON(block_group->space_info->total_bytes
- < block_group->key.offset);
- WARN_ON(block_group->space_info->bytes_readonly
- < block_group->key.offset);
- WARN_ON(block_group->space_info->disk_total
- < block_group->key.offset * factor);
- }
- block_group->space_info->total_bytes -= block_group->key.offset;
- block_group->space_info->bytes_readonly -= block_group->key.offset;
- block_group->space_info->disk_total -= block_group->key.offset * factor;
-
- spin_unlock(&block_group->space_info->lock);
-
- memcpy(&key, &block_group->key, sizeof(key));
-
- mutex_lock(&fs_info->chunk_mutex);
- spin_lock(&block_group->lock);
- block_group->removed = 1;
- /*
- * At this point trimming can't start on this block group, because we
- * removed the block group from the tree fs_info->block_group_cache_tree
- * so no one can't find it anymore and even if someone already got this
- * block group before we removed it from the rbtree, they have already
- * incremented block_group->trimming - if they didn't, they won't find
- * any free space entries because we already removed them all when we
- * called btrfs_remove_free_space_cache().
- *
- * And we must not remove the extent map from the fs_info->mapping_tree
- * to prevent the same logical address range and physical device space
- * ranges from being reused for a new block group. This is because our
- * fs trim operation (btrfs_trim_fs() / btrfs_ioctl_fitrim()) is
- * completely transactionless, so while it is trimming a range the
- * currently running transaction might finish and a new one start,
- * allowing for new block groups to be created that can reuse the same
- * physical device locations unless we take this special care.
- *
- * There may also be an implicit trim operation if the file system
- * is mounted with -odiscard. The same protections must remain
- * in place until the extents have been discarded completely when
- * the transaction commit has completed.
- */
- remove_em = (atomic_read(&block_group->trimming) == 0);
- spin_unlock(&block_group->lock);
-
- mutex_unlock(&fs_info->chunk_mutex);
-
- ret = remove_block_group_free_space(trans, block_group);
- if (ret)
- goto out;
-
- btrfs_put_block_group(block_group);
- btrfs_put_block_group(block_group);
-
- ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
- if (ret > 0)
- ret = -EIO;
- if (ret < 0)
- goto out;
-
- ret = btrfs_del_item(trans, root, path);
- if (ret)
- goto out;
-
- if (remove_em) {
- struct extent_map_tree *em_tree;
-
- em_tree = &fs_info->mapping_tree;
- write_lock(&em_tree->lock);
- remove_extent_mapping(em_tree, em);
- write_unlock(&em_tree->lock);
- /* once for the tree */
- free_extent_map(em);
- }
-out:
- if (remove_rsv)
- btrfs_delayed_refs_rsv_release(fs_info, 1);
- btrfs_free_path(path);
- return ret;
-}
-
-struct btrfs_trans_handle *
-btrfs_start_trans_remove_block_group(struct btrfs_fs_info *fs_info,
- const u64 chunk_offset)
-{
- struct extent_map_tree *em_tree = &fs_info->mapping_tree;
- struct extent_map *em;
- struct map_lookup *map;
- unsigned int num_items;
-
- read_lock(&em_tree->lock);
- em = lookup_extent_mapping(em_tree, chunk_offset, 1);
- read_unlock(&em_tree->lock);
- ASSERT(em && em->start == chunk_offset);
-
- /*
- * We need to reserve 3 + N units from the metadata space info in order
- * to remove a block group (done at btrfs_remove_chunk() and at
- * btrfs_remove_block_group()), which are used for:
- *
- * 1 unit for adding the free space inode's orphan (located in the tree
- * of tree roots).
- * 1 unit for deleting the block group item (located in the extent
- * tree).
- * 1 unit for deleting the free space item (located in tree of tree
- * roots).
- * N units for deleting N device extent items corresponding to each
- * stripe (located in the device tree).
- *
- * In order to remove a block group we also need to reserve units in the
- * system space info in order to update the chunk tree (update one or
- * more device items and remove one chunk item), but this is done at
- * btrfs_remove_chunk() through a call to check_system_chunk().
- */
- map = em->map_lookup;
- num_items = 3 + map->num_stripes;
- free_extent_map(em);
-
- return btrfs_start_transaction_fallback_global_rsv(fs_info->extent_root,
- num_items, 1);
-}
-
-/*
- * Process the unused_bgs list and remove any that don't have any allocated
- * space inside of them.
- */
-void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
-{
- struct btrfs_block_group_cache *block_group;
- struct btrfs_space_info *space_info;
- struct btrfs_trans_handle *trans;
- int ret = 0;
-
- if (!test_bit(BTRFS_FS_OPEN, &fs_info->flags))
- return;
-
- spin_lock(&fs_info->unused_bgs_lock);
- while (!list_empty(&fs_info->unused_bgs)) {
- u64 start, end;
- int trimming;
-
- block_group = list_first_entry(&fs_info->unused_bgs,
- struct btrfs_block_group_cache,
- bg_list);
- list_del_init(&block_group->bg_list);
-
- space_info = block_group->space_info;
-
]
]
|
| 152/4 |
2026/08/08 03:47 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 71815
output: 822
thoughts: 1024
Thoughts: **Why Moving `check_system_chunk` Before `btrfs_remove_dev_extents` is a Bad Idea**
Okay, so I've been thinking about this. It's pretty clear why we can't just shuffle around the order of operations in `btrfs_remove_chunk`. Let me lay out my thinking:
First, there's the **deadlock potential on `chunk_mutex`**. We *know* `check_system_chunk` needs `fs_info->chunk_mutex`, the chunk mutex, and it's asserting that fact with a `lockdep_assert_held`. If we lock the mutex, call `check_system_chunk`, and *then* call `btrfs_remove_dev_extents`, we're asking for trouble. `btrfs_remove_dev_extents` is going to delete items from the device tree (`fs_info->dev_root`). That deletion *can* lead to a metadata chunk allocation if it needs to COW extent buffers in the device tree to prevent corruption, and the allocation path (`btrfs_chunk_alloc`) also attempts to acquire `fs_info->chunk_mutex`. Boom, we're in a deadlock. The process calling btrfs_remove_chunk owns the mutex, and is waiting to get it. The process trying to COW the extent in `btrfs_remove_dev_extents` is trying to allocate a chunk, and needs the mutex. The chunk alloc process has the mutex locked!
Second, moving the check causes **premature `-ENOSPC` errors or transaction aborts**. The code sets `trans->removing_chunk = true` *before* it grabs the mutex and before the check. That's a deliberate attempt to prevent other tasks from deadlocking on `chunk_mutex` when they trigger allocation during chunk removal. If we moved `check_system_chunk` earlier, we'd set `trans->removing_chunk` *before* the device extent removal. Then, if `btrfs_remove_dev_extents` needs a chunk allocation (which, as explained before, is possible) it'll get `-ENOSPC` immediately, since `btrfs_chunk_alloc` returns that error when `trans->removing_chunk` is true. This would be very bad – spurious transaction aborts or errors during the device extent removal phase!
Finally, we'd have a **racy and ineffective system space reservation**. If we *didn't* hold the `chunk_mutex` for the whole `btrfs_remove_dev_extents` process, and just locked it, checked, and unlocked it just for `check_system_chunk`, then the space reservation is no longer atomic. First, calling `check_system_chunk` without the mutex is just wrong, because of `lockdep`. Second, even if we were to lock for just a check, other concurrent tasks could allocate chunks in between, consuming our "reserved" space or messing with system block groups. By the time we get to `remove_chunk_item`, the reservation is useless. This defeats the point of the check, which is to guarantee enough space for the chunk btree changes. `btrfs_remove_dev_extents` could even trigger an allocation itself, causing double reservation on the same transaction handle.
I've reviewed the `btrfs_remove_chunk` code again, and the order is: `btrfs_remove_dev_extents`, then `trans->removing_chunk = true`, then `mutex_lock(&fs_info->chunk_mutex)`, then `check_system_chunk`. Makes perfect sense now. I think that covers it.
|
| 153/4 |
2026/08/08 03:48 |
tool |
set-results |
0m
Args: map[Answer:No, it is **not safe** to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`. Doing so would introduce severe deadlock risks, spurious transaction aborts, and racy space reservations.
Here is a detailed breakdown of why this order is critical:
### 1. Deadlock on `fs_info->chunk_mutex`
* **Lock Requirement**: `check_system_chunk` requires holding `fs_info->chunk_mutex` (asserted by `lockdep_assert_held(&fs_info->chunk_mutex)` in `reserve_chunk_space`).
* **The Deadlock Path**: If we hold `chunk_mutex` while calling `btrfs_remove_dev_extents`, we can deadlock. `btrfs_remove_dev_extents` deletes device extent items from the device tree (`fs_info->dev_root`), which is a regular btree. Deleting items from this btree requires COWing extent buffers. If the metadata space is low, COWing these buffers can trigger a metadata chunk allocation (`btrfs_chunk_alloc`), which will attempt to acquire `fs_info->chunk_mutex`. Since the current thread already holds `chunk_mutex`, this results in a deadlock.
### 2. Spurious `-ENOSPC` and Transaction Aborts via `trans->removing_chunk`
* To prevent concurrent tasks from deadlocking on `chunk_mutex` during chunk removal, `btrfs_remove_chunk` sets `trans->removing_chunk = true` while holding `chunk_mutex`.
* If `trans->removing_chunk` is set to `true` before `btrfs_remove_dev_extents` is called, any metadata chunk allocation triggered during `btrfs_remove_dev_extents` (due to COWing device tree buffers) will immediately fail. This is because `btrfs_chunk_alloc` returns `-ENOSPC` when `trans->removing_chunk` is `true`. This would lead to premature and spurious `-ENOSPC` errors and transaction aborts.
### 3. Ineffective and Racy System Space Reservation
* If we do not hold `chunk_mutex` across `btrfs_remove_dev_extents` but instead acquire `chunk_mutex`, call `check_system_chunk`, release `chunk_mutex`, and then call `btrfs_remove_dev_extents`, the reservation is no longer safe:
1. Calling `check_system_chunk` without holding `chunk_mutex` violates lockdep assertions and is unsafe because it can allocate a system chunk, which itself modifies the chunk btree.
2. During the execution of `btrfs_remove_dev_extents` (which can be slow as it deletes device extents for all stripes), other concurrent tasks can allocate chunks and consume the reserved system space or modify the system block groups. By the time we actually modify the chunk btree in `remove_chunk_item`, the system space reservation might have been invalidated or exhausted, defeating the purpose of `check_system_chunk` (which is to guarantee enough system space for the chunk btree modifications).
3. If `btrfs_remove_dev_extents` itself triggers a chunk allocation, it would accumulate multiple system space reservations on the same transaction handle, leading to resource bloat or potential exhaustion.
Therefore, `btrfs_remove_dev_extents` must be executed first, and only after it succeeds can we set `trans->removing_chunk = true`, acquire `fs_info->chunk_mutex`, and call `check_system_chunk` to safely reserve space immediately before modifying the chunk btree.]
Results: map[Answer:No, it is **not safe** to move `check_system_chunk` before `btrfs_remove_dev_extents` in `btrfs_remove_chunk`. Doing so would introduce severe deadlock risks, spurious transaction aborts, and racy space reservations.
Here is a detailed breakdown of why this order is critical:
### 1. Deadlock on `fs_info->chunk_mutex`
* **Lock Requirement**: `check_system_chunk` requires holding `fs_info->chunk_mutex` (asserted by `lockdep_assert_held(&fs_info->chunk_mutex)` in `reserve_chunk_space`).
* **The Deadlock Path**: If we hold `chunk_mutex` while calling `btrfs_remove_dev_extents`, we can deadlock. `btrfs_remove_dev_extents` deletes device extent items from the device tree (`fs_info->dev_root`), which is a regular btree. Deleting items from this btree requires COWing extent buffers. If the metadata space is low, COWing these buffers can trigger a metadata chunk allocation (`btrfs_chunk_alloc`), which will attempt to acquire `fs_info->chunk_mutex`. Since the current thread already holds `chunk_mutex`, this results in a deadlock.
### 2. Spurious `-ENOSPC` and Transaction Aborts via `trans->removing_chunk`
* To prevent concurrent tasks from deadlocking on `chunk_mutex` during chunk removal, `btrfs_remove_chunk` sets `trans->removing_chunk = true` while holding `chunk_mutex`.
* If `trans->removing_chunk` is set to `true` before `btrfs_remove_dev_extents` is called, any metadata chunk allocation triggered during `btrfs_remove_dev_extents` (due to COWing device tree buffers) will immediately fail. This is because `btrfs_chunk_alloc` returns `-ENOSPC` when `trans->removing_chunk` is `true`. This would lead to premature and spurious `-ENOSPC` errors and transaction aborts.
### 3. Ineffective and Racy System Space Reservation
* If we do not hold `chunk_mutex` across `btrfs_remove_dev_extents` but instead acquire `chunk_mutex`, call `check_system_chunk`, release `chunk_mutex`, and then call `btrfs_remove_dev_extents`, the reservation is no longer safe:
1. Calling `check_system_chunk` without holding `chunk_mutex` violates lockdep assertions and is unsafe because it can allocate a system chunk, which itself modifies the chunk btree.
2. During the execution of `btrfs_remove_dev_extents` (which can be slow as it deletes device extents for all stripes), other concurrent tasks can allocate chunks and consume the reserved system space or modify the system block groups. By the time we actually modify the chunk btree in `remove_chunk_item`, the system space reservation might have been invalidated or exhausted, defeating the purpose of `check_system_chunk` (which is to guarantee enough system space for the chunk btree modifications).
3. If `btrfs_remove_dev_extents` itself triggers a chunk allocation, it would accumulate multiple system space reservations on the same transaction handle, leading to resource bloat or potential exhaustion.
Therefore, `btrfs_remove_dev_extents` must be executed first, and only after it succeeds can we set `trans->removing_chunk = true`, acquire `fs_info->chunk_mutex`, and call `check_system_chunk` to safely reserve space immediately before modifying the chunk btree.]
|
| 154/2 |
2026/08/08 03:48 |
llm |
debugger |
0m
Model:
Tokens:
input: 153248
output: 242
thoughts: 4005
Thoughts: **Analysis of System Chunk Reservation During Chunk Removal**
Okay, so the code expert has raised some concerns about my proposed fix for this system chunk issue. Let's break this down systematically.
First, the deadlock: I already had that pegged. The fix is pretty straightforward: acquire `chunk_mutex`, check the system chunk, release `chunk_mutex`, *then* remove the extents. We can lock and unlock chunk_mutex here to avoid deadlock.
Then there's the spurious `-ENOSPC`. That one's easy: set `trans->removing_chunk = false` before calling `btrfs_remove_dev_extents`.
Now, the meat of the expert's concerns: the *ineffective and racy* system space reservation. They're worried about several things. First, calling `check_system_chunk` without holding `chunk_mutex`. That's addressed; I'm now locking it before checking. The second concern, though, needs more thought. The expert is concerned that while `btrfs_remove_dev_extents` is running, other concurrent tasks will allocate chunks and consume the *reserved* system space. However, `trans->chunk_bytes_reserved` tracks space reserved in `chunk_block_rsv`. This is a *global pool* for chunk allocations, but the space is, in fact, reserved. If a chunk allocation *does* happen concurrently, it *must* have reserved its own space first, so it won't be using our reserved space.
But wait! What about the case where `btrfs_remove_dev_extents` *itself* triggers a chunk allocation? Well, if it does, `btrfs_chunk_alloc` will be called, which calls `check_system_chunk`, which reserves *more* space in `chunk_block_rsv`. It will simply use the space that it reserved. And the transaction ends, the reserved space for removing the chunk, and the reserved space for allocating the new chunk will be released. That's fine.
The expert then says, "If `btrfs_remove_dev_extents` itself triggers a chunk allocation, it would accumulate multiple system space reservations on the same transaction handle, leading to resource bloat or potential exhaustion." But `trans->chunk_bytes_reserved` is just an integer that can increment. It's *all* released at the end of the transaction. That's not a problem.
So, since we can't avoid calling `check_system_chunk`, the way forward is to make sure we lock `chunk_mutex` before we call `check_system_chunk`.
The core problem, of course, is that `find_free_dev_extent` searches the commit root, so it doesn't "see" the space freed by `btrfs_remove_dev_extents` *during the transaction*. We can't reuse space freed in the current transaction, and failing to allocate a system chunk means we must abort.
So, if we can't move `check_system_chunk` before `btrfs_remove_dev_extents`, and we can't fail *gracefully* after that function executes, what can we do? We should move it before.
Okay, let's try moving `check_system_chunk`. Here is the proposed code:
```c
/*
* We need to ensure we have enough system space before modifying the
* device tree. Otherwise, if we fail to allocate a system chunk later,
* we would have to abort the transaction.
*/
mutex_lock(&fs_info->chunk_mutex);
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
```
Is this safe? The expert says other tasks can "consume the reserved system space". However, `chunk_block_rsv` is a shared block reserve, but when we add space to it, we increase its `reserved` counter. When another task needs space, it uses `trans->block
|
| 155/2 |
2026/08/08 03:48 |
tool |
codeexpert |
3m
Args: map[Question:Is there any flaw in this proposed fix for btrfs_remove_chunk?
```c
mutex_lock(&fs_info->chunk_mutex);
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
```]
Results: map[Answer:Yes, there are several critical flaws in this proposed fix for `btrfs_remove_chunk`.
### 1. Metadata Reservation Leak on the Error Path
In the proposed fix, `check_system_chunk` is called at the very beginning of the function, which reserves system space and increases `trans->chunk_bytes_reserved`. If `btrfs_remove_dev_extents(trans, map)` subsequently fails, the code executes `goto out;`.
However, the `out` block does not call `btrfs_trans_release_chunk_metadata(trans)`. Therefore, the reserved metadata space is leaked in the transaction handle, which can lead to metadata space exhaustion or warnings/assertions when the transaction is committed or the transaction handle is freed.
### 2. Premature Release of Reserved Space by Nested Chunk Allocations
`btrfs_remove_dev_extents` deletes device extents from the devices btree. This operation can trigger a metadata chunk allocation (via `do_chunk_alloc`) if metadata space is low.
At the end of `do_chunk_alloc`, `btrfs_trans_release_chunk_metadata(trans)` is called unconditionally, which releases **all** chunk metadata reserved by the transaction handle (`trans->chunk_bytes_reserved` is reset to `0`).
As a result, the system space reserved at the beginning of `btrfs_remove_chunk` is prematurely released back to the global pool. When `remove_chunk_item` is later called, there will be `0` bytes reserved, defeating the entire purpose of the pre-reservation and risking transaction aborts due to `-ENOSPC`.
### 3. Race Condition / Space Stealing due to Unlocking `chunk_mutex`
`fs_info->chunk_block_rsv` is a global block reserve shared among all threads. In the proposed fix, `chunk_mutex` is unlocked after `check_system_chunk` and before `btrfs_remove_dev_extents`.
While `chunk_mutex` is unlocked, other concurrent threads can acquire the mutex and perform chunk allocations or other operations that modify the chunk tree. These operations can consume/steal the space we just reserved in `fs_info->chunk_block_rsv`. When we later re-acquire `chunk_mutex` and call `remove_chunk_item`, the space might no longer be available, leading to unexpected `-ENOSPC` failures and transaction aborts.
### 4. Unnecessary `-ENOSPC` Failures on Reservation Failure
The proposed fix checks if `trans->chunk_bytes_reserved` did not increase after calling `check_system_chunk`, and if so, immediately aborts and returns `-ENOSPC`.
However, as documented in `reserve_chunk_space`, a failure to reserve system space (e.g., failure to allocate a new system chunk) should be ignored because the chunk removal might still succeed without needing to COW any new nodes/leafs (e.g., if the paths are already COWed in the current transaction). Returning `-ENOSPC` early causes the chunk removal to fail unnecessarily in scenarios where it would have otherwise succeeded.]
|
| 156/3 |
2026/08/08 03:48 |
agent |
codeexpert |
3m
Model:
Results: map[Answer:Yes, there are several critical flaws in this proposed fix for `btrfs_remove_chunk`.
### 1. Metadata Reservation Leak on the Error Path
In the proposed fix, `check_system_chunk` is called at the very beginning of the function, which reserves system space and increases `trans->chunk_bytes_reserved`. If `btrfs_remove_dev_extents(trans, map)` subsequently fails, the code executes `goto out;`.
However, the `out` block does not call `btrfs_trans_release_chunk_metadata(trans)`. Therefore, the reserved metadata space is leaked in the transaction handle, which can lead to metadata space exhaustion or warnings/assertions when the transaction is committed or the transaction handle is freed.
### 2. Premature Release of Reserved Space by Nested Chunk Allocations
`btrfs_remove_dev_extents` deletes device extents from the devices btree. This operation can trigger a metadata chunk allocation (via `do_chunk_alloc`) if metadata space is low.
At the end of `do_chunk_alloc`, `btrfs_trans_release_chunk_metadata(trans)` is called unconditionally, which releases **all** chunk metadata reserved by the transaction handle (`trans->chunk_bytes_reserved` is reset to `0`).
As a result, the system space reserved at the beginning of `btrfs_remove_chunk` is prematurely released back to the global pool. When `remove_chunk_item` is later called, there will be `0` bytes reserved, defeating the entire purpose of the pre-reservation and risking transaction aborts due to `-ENOSPC`.
### 3. Race Condition / Space Stealing due to Unlocking `chunk_mutex`
`fs_info->chunk_block_rsv` is a global block reserve shared among all threads. In the proposed fix, `chunk_mutex` is unlocked after `check_system_chunk` and before `btrfs_remove_dev_extents`.
While `chunk_mutex` is unlocked, other concurrent threads can acquire the mutex and perform chunk allocations or other operations that modify the chunk tree. These operations can consume/steal the space we just reserved in `fs_info->chunk_block_rsv`. When we later re-acquire `chunk_mutex` and call `remove_chunk_item`, the space might no longer be available, leading to unexpected `-ENOSPC` failures and transaction aborts.
### 4. Unnecessary `-ENOSPC` Failures on Reservation Failure
The proposed fix checks if `trans->chunk_bytes_reserved` did not increase after calling `check_system_chunk`, and if so, immediately aborts and returns `-ENOSPC`.
However, as documented in `reserve_chunk_space`, a failure to reserve system space (e.g., failure to allocate a new system chunk) should be ignored because the chunk removal might still succeed without needing to COW any new nodes/leafs (e.g., if the paths are already COWed in the current transaction). Returning `-ENOSPC` early causes the chunk removal to fail unnecessarily in scenarios where it would have otherwise succeeded.]
Instruction:
You are a capable Linux kernel developer tasked with researching complex questions
about kernel source code. You will be given a concrete question, and need to provide
a concrete answer.
Use tools extensively while researching the question. Don't make assumptions,
or rely on your previous knowledge about the kernel source code, use available tools
to access the actual source code.
Use all available sources of information:
- kernel source code
- documentation in the Documentation dir in the source tree
- git commits descriptions, git blame
Do not guess file names or file paths and attempt to read them without
verifying their existence first using content search or directory listing tools.
If a file, symbol, or directory is not found via content search (grepper) or
directory listing (codesearch-dir-index), treat it as completely absent.
Do not attempt to guess alternative names, extensions, or directories.
Tool Selection Guidelines:
1. For C functions, structs, and variables: Prefer codesearch-definition-source
or codesearch-struct-layout FIRST to retrieve clean, exact definitions
without line-number guessing.
2. If symbol lookup fails (e.g., preprocessor macros, macro-generated code, or
disabled #ifdef branches), fall back to read-file or grepper.
3. For file headers, #include directives, preprocessor macro definitions, and
non-C files (Kconfig, Makefiles, docs): Use read-file or grepper directly.
Do NOT use git-log to search for the presence or existence of files in the
repository. git-log is only for tracing commit history of files that are
already present in the current checkout. If a file does not exist in the
current checkout, it cannot be used for reproduction.
Avoid running broad git-log queries (such as searches on the entire repo)
without a specific 'PathPrefix' to restrict the scope, as these are very
expensive and will time out.
If a git-log tool call times out, do not retry the query with the same broad
scope. You must specify a tighter, narrower 'PathPrefix' for subsequent queries.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Is there any flaw in this proposed fix for btrfs_remove_chunk?
```c
mutex_lock(&fs_info->chunk_mutex);
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
```
|
| 157/4 |
2026/08/08 03:48 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 4095
output: 40
thoughts: 75
Thoughts: Okay, let's break this down. The user is questioning a fix for `btrfs_remove_chunk`. My immediate instinct is to dissect the function itself – understand its purpose and its place in the broader btrfs architecture. Therefore, my first step is to definitively locate the source code definition of `btrfs_remove_chunk`. I'll reach for my preferred tools, either `codesearch-definition-source` or `grepper`, to quickly pinpoint where this function lives and, importantly, what it *does*. This is the critical foundation before I can even begin to assess the proposed fix. Understanding the "problem space" is paramount before evaluating any solution.
|
| 158/4 |
2026/08/08 03:48 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_chunk]
Results: map[SourceCode:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
3553: }
3554: }
3555:
3556: btrfs_update_per_profile_avail(fs_info);
3557: mutex_unlock(&fs_info->chunk_mutex);
3558: trans->removing_chunk = false;
3559:
3560: /*
3561: * We are done with chunk btree updates and deletions, so release the
3562: * system space we previously reserved (with check_system_chunk()).
3563: */
3564: btrfs_trans_release_chunk_metadata(trans);
3565:
3566: /* On error, btrfs_remove_block_group() aborts the transaction. */
3567: ret = btrfs_remove_block_group(trans, map);
3568: if (unlikely(ret))
3569: ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
3570:
3571: out:
3572: if (trans->removing_chunk) {
3573: mutex_unlock(&fs_info->chunk_mutex);
3574: trans->removing_chunk = false;
3575: }
3576: /* once for us */
3577: btrfs_free_chunk_map(map);
3578: return ret;
3579: }
SourceFile:fs/btrfs/volumes.c]
|
| 159/4 |
2026/08/08 03:48 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 6449
output: 38
thoughts: 270
Thoughts: **Analyzing the Proposed `btrfs_remove_chunk` Fix**
Okay, let's break this down. My immediate task is to understand this proposed modification to `btrfs_remove_chunk` and compare it to the existing code. First, I need to methodically read this code snippet. Alright, here's what's happening: it looks like there's a mutex lock for the `fs_info->chunk_mutex`, and then a check comparing `trans->chunk_bytes_reserved` before and after a call to `check_system_chunk`. If they're equal, it's an `ENOSPC` and we bail. Then it calls `btrfs_remove_dev_extents`. Finally, it sets `trans->removing_chunk` to `true`, locks the mutex again, and calls `remove_chunk_item`.
Hmm, this `check_system_chunk` call is intriguing. It seems crucial to the logic, especially regarding the chunk reservation. It’s checking something related to system chunks, but I need to understand its exact purpose. I need to know *exactly* what `check_system_chunk` is doing to properly assess this. Let's see... I'll need to find its definition and go over its implementation.
|
| 160/4 |
2026/08/08 03:48 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 161/4 |
2026/08/08 03:48 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 7352
output: 40
thoughts: 70
Thoughts: Here's my take on those thoughts:
**Navigating the Btrfs Chunk Allocation Code**
Okay, so I'm diving into this Btrfs chunk allocation logic. Right, let's see... the `check_system_chunk` function, that's what I need to understand, and the tool points me towards `fs/btrfs/block-group.c`. Ah, yes, that rings a bell. No, wait, hang on... was it `volumes.c`? No, the tool is right, it's definitely `block-group.c`. Got it.
Now, where was I? Right, `check_system_chunk`. I need to figure out how it works. I'll take a look at `reserve_chunk_space`. That feels like the next logical step. Let's see how that fits into the picture. Time to dive in!
|
| 162/4 |
2026/08/08 03:48 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 163/4 |
2026/08/08 03:48 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 8918
output: 32
thoughts: 1114
Thoughts: **Analyzing Chunk Reservation and Concurrency in Btrfs**
Okay, let's dive into this `reserve_chunk_space` function. My initial thought is to trace the execution flow and how the chunk reservation works, so: if `reserve_chunk_space` succeeds (`!ret`), it adds bytes to `trans->chunk_bytes_reserved` via `btrfs_block_rsv_add`. Now, what happens if it *fails*? If `ret` is non-zero, the `trans->chunk_bytes_reserved` isn't updated, which seems correct. If `ret` is non-zero, `trans->chunk_bytes_reserved` remains unchanged. This behavior is expected, so the transaction will report an error.
Now, this proposed fix... It's got me thinking. It locks a `chunk_mutex`, stores `trans->chunk_bytes_reserved`, calls `check_system_chunk`, checks if `trans->chunk_bytes_reserved` has changed, and returns `-ENOSPC` if it hasn't. But, and here's the kicker, what if `trans->chunk_bytes_reserved` was already non-zero *before* `check_system_chunk`? And `check_system_chunk` doesn't manage to add any more space? The check `trans->chunk_bytes_reserved == chunk_bytes_reserved` becomes true, leading to `-ENOSPC`, which is the correct behavior. But, is `check_system_chunk` *guaranteed* to increase the reserved space?
Okay, let's explore scenarios where things don't go according to plan. Suppose `reserve_chunk_space` fails. In this case, `trans->chunk_bytes_reserved` doesn't increase, which is again, expected. Is there any other scenario where `trans->chunk_bytes_reserved` might *not* change? No, because `check_system_chunk` always tries to reserve and add.
Alright, let's zoom in on this locking and unlocking of the `chunk_mutex`. The proposed fix locks, calls `check_system_chunk`, unlocks, then calls `btrfs_remove_dev_extents`, and *then* locks again before calling `remove_chunk_item`. The original code holds the `chunk_mutex` across *both* `check_system_chunk` and `remove_chunk_item`. Why the change?
Let's check the code comments... Aha! The original code holds the `chunk_mutex` for two primary reasons: first, to reserve system space, update chunk B-tree entries, delete data, and update the system chunk array in the superblock. And second, and very important, to prevent races with a device replace operation's *final* phase. During a device replace, the device object ID can change, and holding the `chunk_mutex` protects against this.
So, here's my question: If we unlock the `chunk_mutex` after `check_system_chunk` and *before* `remove_chunk_item`, can another thread allocate/remove chunks, or can a device replace *finish*? The answer is *yes*. Unlocking the mutex opens the door for other threads to consume the reserved space in `chunk_block_rsv`! Wait, it seems that `chunk_block_rsv` isn't per-transactional, but global! The global block reserve for the *entire* file system. Yes, let's confirm this by searching the code for `chunk_block_rsv`...
|
| 164/4 |
2026/08/08 03:49 |
tool |
codesearch-find-references |
0m
Args: map[Name:btrfs_fs_info::chunk_block_rsv]
Results: map[References:[map[ReferenceKind:takes-address-of ReferencingEntityKind:function ReferencingEntityName:btrfs_init_fs_info SourceFile:fs/btrfs/disk-io.c SourceLine:2835] map[ReferenceKind:reads ReferencingEntityKind:function ReferencingEntityName:btrfs_init_global_block_rsv SourceFile:fs/btrfs/block-rsv.c SourceLine:452] map[ReferenceKind:takes-address-of ReferencingEntityKind:function ReferencingEntityName:btrfs_init_root_block_rsv SourceFile:fs/btrfs/block-rsv.c SourceLine:433] map[ReferenceKind:reads ReferencingEntityKind:function ReferencingEntityName:btrfs_release_global_block_rsv SourceFile:fs/btrfs/block-rsv.c SourceLine:481] map[ReferenceKind:reads ReferencingEntityKind:function ReferencingEntityName:btrfs_release_global_block_rsv SourceFile:fs/btrfs/block-rsv.c SourceLine:482] map[ReferenceKind:takes-address-of ReferencingEntityKind:function ReferencingEntityName:btrfs_trans_release_chunk_metadata SourceFile:fs/btrfs/transaction.c SourceLine:262] map[ReferenceKind:takes-address-of ReferencingEntityKind:function ReferencingEntityName:dump_global_block_rsv SourceFile:fs/btrfs/space-info.c SourceLine:639] map[ReferenceKind:takes-address-of ReferencingEntityKind:function ReferencingEntityName:reserve_chunk_space SourceFile:fs/btrfs/block-group.c SourceLine:4563]] TruncatedOutput:false]
|
| 165/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 11335
output: 23
thoughts: 141
Thoughts: **My Line of Reasoning**
Okay, so `chunk_block_rsv` is the global reserve for blocks, got it. But a thought just hit me. If I release `chunk_mutex` – and I *have* to, right? – another thread could swoop in and grab a chunk. Remember, that chunk allocation also holds `chunk_mutex` while it's in the process. Now that I've unlocked it, another thread can acquire it.
And here's the kicker: if that other thread *does* allocate a chunk, it could very well use up the space in `chunk_block_rsv` that *I* just reserved! Damn, is that really a possibility? I need to verify this assumption. Let's dig into how chunk allocation actually reserves space. Time to find the `btrfs_chunk_alloc` code, or something similar – I'll search for calls to `check_system_chunk`. That should reveal the locking and reservation logic and confirm whether my suspicion is accurate.
|
| 166/4 |
2026/08/08 03:49 |
tool |
codesearch-find-references |
0m
Args: map[Name:check_system_chunk]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_inc_block_group_ro SourceFile:fs/btrfs/block-group.c SourceLine:3234] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3494] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:do_chunk_alloc SourceFile:fs/btrfs/block-group.c SourceLine:4141]] TruncatedOutput:false]
|
| 167/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 11905
output: 40
thoughts: 24
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience, and written as if they were my own immediate processing:
**Diving into Chunk Allocation**
Alright, let's zero in on `do_chunk_alloc`. I need to understand exactly how it leverages `check_system_chunk`. This is crucial. I'm guessing `check_system_chunk` is the gatekeeper here, determining if we *can* even allocate. Is it a simple boolean, or does it return some granular status? Maybe it provides hints about available memory or fragmentation? Understanding the control flow from `do_chunk_alloc` to that call and back is what's key. I need to trace the implications of a success or failure returned by `check_system_chunk`. Does it immediately bail, or does it attempt some other fallback strategy? Let me pull up the definitions and see the data flow. I want to see if this is a performance bottleneck.
|
| 168/4 |
2026/08/08 03:49 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:do_chunk_alloc]
Results: map[SourceCode:4128: static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
4129: struct btrfs_space_info *space_info,
4130: u64 flags)
4131: {
4132: struct btrfs_block_group *bg;
4133: int ret;
4134:
4135: /*
4136: * Check if we have enough space in the system space info because we
4137: * will need to update device items in the chunk btree and insert a new
4138: * chunk item in the chunk btree as well. This will allocate a new
4139: * system block group if needed.
4140: */
4141: check_system_chunk(trans, flags);
4142:
4143: bg = btrfs_create_chunk(trans, space_info, flags);
4144: if (IS_ERR(bg)) {
4145: ret = PTR_ERR(bg);
4146: goto out;
4147: }
4148:
4149: ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
4150: /*
4151: * Normally we are not expected to fail with -ENOSPC here, since we have
4152: * previously reserved space in the system space_info and allocated one
4153: * new system chunk if necessary. However there are three exceptions:
4154: *
4155: * 1) We may have enough free space in the system space_info but all the
4156: * existing system block groups have a profile which can not be used
4157: * for extent allocation.
4158: *
4159: * This happens when mounting in degraded mode. For example we have a
4160: * RAID1 filesystem with 2 devices, lose one device and mount the fs
4161: * using the other device in degraded mode. If we then allocate a chunk,
4162: * we may have enough free space in the existing system space_info, but
4163: * none of the block groups can be used for extent allocation since they
4164: * have a RAID1 profile, and because we are in degraded mode with a
4165: * single device, we are forced to allocate a new system chunk with a
4166: * SINGLE profile. Making check_system_chunk() iterate over all system
4167: * block groups and check if they have a usable profile and enough space
4168: * can be slow on very large filesystems, so we tolerate the -ENOSPC and
4169: * try again after forcing allocation of a new system chunk. Like this
4170: * we avoid paying the cost of that search in normal circumstances, when
4171: * we were not mounted in degraded mode;
4172: *
4173: * 2) We had enough free space info the system space_info, and one suitable
4174: * block group to allocate from when we called check_system_chunk()
4175: * above. However right after we called it, the only system block group
4176: * with enough free space got turned into RO mode by a running scrub,
4177: * and in this case we have to allocate a new one and retry. We only
4178: * need do this allocate and retry once, since we have a transaction
4179: * handle and scrub uses the commit root to search for block groups;
4180: *
4181: * 3) We had one system block group with enough free space when we called
4182: * check_system_chunk(), but after that, right before we tried to
4183: * allocate the last extent buffer we needed, a discard operation came
4184: * in and it temporarily removed the last free space entry from the
4185: * block group (discard removes a free space entry, discards it, and
4186: * then adds back the entry to the block group cache).
4187: */
4188: if (ret == -ENOSPC) {
4189: const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
4190: struct btrfs_block_group *sys_bg;
4191: struct btrfs_space_info *sys_space_info;
4192:
4193: sys_space_info = btrfs_find_space_info(trans->fs_info, sys_flags);
4194: if (unlikely(!sys_space_info)) {
4195: ret = -EINVAL;
4196: btrfs_abort_transaction(trans, ret);
4197: goto out;
4198: }
4199:
4200: sys_bg = btrfs_create_chunk(trans, sys_space_info, sys_flags);
4201: if (IS_ERR(sys_bg)) {
4202: ret = PTR_ERR(sys_bg);
4203: btrfs_abort_transaction(trans, ret);
4204: goto out;
4205: }
4206:
4207: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
4208: if (unlikely(ret)) {
4209: btrfs_abort_transaction(trans, ret);
4210: goto out;
4211: }
4212:
4213: ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
4214: if (unlikely(ret)) {
4215: btrfs_abort_transaction(trans, ret);
4216: goto out;
4217: }
4218: } else if (unlikely(ret)) {
4219: btrfs_abort_transaction(trans, ret);
4220: goto out;
4221: }
4222: out:
4223: btrfs_trans_release_chunk_metadata(trans);
4224:
4225: if (ret)
4226: return ERR_PTR(ret);
4227:
4228: btrfs_get_block_group(bg);
4229: return bg;
4230: }
SourceFile:fs/btrfs/block-group.c]
|
| 169/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 13954
output: 38
thoughts: 90
Thoughts: **Investigating the Chunk Removal Process**
Okay, first things first, I need to understand why `btrfs_remove_chunk` is holding onto that `chunk_mutex`. It's a critical point for potential contention and performance bottlenecks, so let's dig into the details. My initial instinct is to look at the historical context; that usually tells a story. I'll search the git history and commit messages, specifically focusing on interactions between `btrfs_remove_chunk` and `chunk_mutex`. I'm going to leverage git's search capabilities to make this efficient. Let's fire up `git log` and filter it down. I'll narrow the search with `SymbolName="btrfs_remove_chunk"` and `SourcePath="fs/btrfs/volumes.c"` to pinpoint the relevant commits within that function. This should give me a solid starting point to understand the reasoning behind this locking behavior.
|
| 170/4 |
2026/08/08 03:49 |
tool |
git-log |
0m
Args: map[SourcePath:fs/btrfs/volumes.c SymbolName:btrfs_remove_chunk]
Results: map[Output:c84053d9f7f7 btrfs: update per-profile available estimation
6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
979e1dc3d69e btrfs: handle deletions from remapped block group
a929904cf73b btrfs: add unlikely annotations to branches leading to transaction abort
4d5a047e0736 btrfs: add space_info parameter for block group creation
7dc66abb5a47 btrfs: use a dedicated data structure for chunk maps
f6f39f7a0add btrfs: rename btrfs_alloc_chunk to btrfs_create_chunk
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
64bc6c2a3453 btrfs: Remove superfluous check form btrfs_remove_chunk
f4208794d02e btrfs: Remove fs_info form btrfs_free_chunk
]
|
| 171/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 14559
output: 26
thoughts: 45
Thoughts: Okay, let's dive into this.
**Examining a Btrfs Chunk Allocation Fix**
Alright, I see this commit, `79bd37120b14`. The description mentions a rework of chunk allocation in Btrfs to prevent exhaustion of the system chunk array. My interest is immediately piqued; chunk allocation in Btrfs has always been a nuanced area, and this sounds like it directly addresses a potential stability issue. Let's pull up the diff and get a look at the code changes. I need to understand the specifics of how this exhaustion was occurring and, more importantly, *how* this rework aims to solve it. I'm eager to see if it's a clever algorithmic tweak, a more conservative allocation strategy, or perhaps a combination of the two. Understanding the underlying problem is the key, of course, but equally important is the efficiency and elegance of the proposed solution. So, let's see what the commit reveals.
|
| 172/4 |
2026/08/08 03:49 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1281 lines.
[commit 79bd37120b149532af5b21953643ed74af69654f
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:06 2021 +0100
btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
Commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations") fixed a problem that resulted in
exhausting the system chunk array in the superblock when there are many
tasks allocating chunks in parallel. Basically too many tasks enter the
first phase of chunk allocation without previous tasks having finished
their second phase of allocation, resulting in too many system chunks
being allocated. That was originally observed when running the fallocate
tests of stress-ng on a PowerPC machine, using a node size of 64K.
However that commit also introduced a deadlock where a task in phase 1 of
the chunk allocation waited for another task that had allocated a system
chunk to finish its phase 2, but that other task was waiting on an extent
buffer lock held by the first task, therefore resulting in both tasks not
making any progress. That change was later reverted by a patch with the
subject "btrfs: fix deadlock with concurrent chunk allocations involving
system chunks", since there is no simple and short solution to address it
and the deadlock is relatively easy to trigger on zoned filesystems, while
the system chunk array exhaustion is not so common.
This change reworks the chunk allocation to avoid the system chunk array
exhaustion. It accomplishes that by making the first phase of chunk
allocation do the updates of the device items in the chunk btree and the
insertion of the new chunk item in the chunk btree. This is done while
under the protection of the chunk mutex (fs_info->chunk_mutex), in the
same critical section that checks for available system space, allocates
a new system chunk if needed and reserves system chunk space. This way
we do not have chunk space reserved until the second phase completes.
The same logic is applied to chunk removal as well, since it keeps
reserved system space long after it is done updating the chunk btree.
For direct allocation of system chunks, the previous behaviour remains,
because otherwise we would deadlock on extent buffers of the chunk btree.
Changes to the chunk btree are by large done by chunk allocation and chunk
removal, which first reserve chunk system space and then later do changes
to the chunk btree. The other remaining cases are uncommon and correspond
to adding a device, removing a device and resizing a device. All these
other cases do not pre-reserve system space, they modify the chunk btree
right away, so they don't hold reserved space for a long period like chunk
allocation and chunk removal do.
The diff of this change is huge, but more than half of it is just addition
of comments describing both how things work regarding chunk allocation and
removal, including both the new behavior and the parts of the old behavior
that did not change.
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a26209f98279..c557327b4545 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2207,6 +2207,13 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
return ret;
}
+/*
+ * This function, insert_block_group_item(), belongs to the phase 2 of chunk
+ * allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
static int insert_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_block_group *block_group)
{
@@ -2229,15 +2236,19 @@ static int insert_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_insert_item(trans, root, &key, &bgi, sizeof(bgi));
}
+/*
+ * This function, btrfs_create_pending_block_groups(), belongs to the phase 2 of
+ * chunk allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *block_group;
int ret = 0;
- if (!trans->can_flush_pending_bgs)
- return;
-
while (!list_empty(&trans->new_bgs)) {
int index;
@@ -2252,6 +2263,13 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
ret = insert_block_group_item(trans, block_group);
if (ret)
btrfs_abort_transaction(trans, ret);
+ if (!block_group->chunk_item_inserted) {
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, block_group);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret)
+ btrfs_abort_transaction(trans, ret);
+ }
ret = btrfs_finish_chunk_alloc(trans, block_group->start,
block_group->length);
if (ret)
@@ -2275,8 +2293,9 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
btrfs_trans_release_chunk_metadata(trans);
}
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size)
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *cache;
@@ -2286,7 +2305,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
cache = btrfs_create_block_group_cache(fs_info, chunk_offset);
if (!cache)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
cache->length = size;
set_free_space_tree_thresholds(cache);
@@ -2300,7 +2319,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
ret = btrfs_load_block_group_zone_info(cache, true);
if (ret) {
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
ret = exclude_super_stripes(cache);
@@ -2308,7 +2327,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
/* We may have excluded something, so call this just in case */
btrfs_free_excluded_extents(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
add_new_free_space(cache, chunk_offset, chunk_offset + size);
@@ -2335,7 +2354,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
if (ret) {
btrfs_remove_free_space_cache(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
/*
@@ -2354,7 +2373,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
btrfs_update_delayed_refs_rsv(trans);
set_avail_alloc_bits(fs_info, type);
- return 0;
+ return cache;
}
/*
@@ -3232,11 +3251,203 @@ int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type)
return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
+static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ /*
+ * Check if we have enough space in the system space info because we
+ * will need to update device items in the chunk btree and insert a new
+ * chunk item in the chunk btree as well. This will allocate a new
+ * system block group if needed.
+ */
+ check_system_chunk(trans, flags);
+
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ goto out;
+ }
+
+ /*
+ * If this is a system chunk allocation then stop right here and do not
+ * add the chunk item to the chunk btree. This is to prevent a deadlock
+ * because this system chunk allocation can be triggered while COWing
+ * some extent buffer of the chunk btree and while holding a lock on a
+ * parent extent buffer, in which case attempting to insert the chunk
+ * item (or update the device item) would result in a deadlock on that
+ * parent extent buffer. In this case defer the chunk btree updates to
+ * the second phase of chunk allocation and keep our reservation until
+ * the second phase completes.
+ *
+ * This is a rare case and can only be triggered by the very few cases
+ * we have where we need to touch the chunk btree outside chunk allocation
+ * and chunk removal. These cases are basically adding a device, removing
+ * a device or resizing a device.
+ */
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ return 0;
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ /*
+ * Normally we are not expected to fail with -ENOSPC here, since we have
+ * previously reserved space in the system space_info and allocated one
+ * new system chunk if necessary. However there are two exceptions:
+ *
+ * 1) We may have enough free space in the system space_info but all the
+ * existing system block groups have a profile which can not be used
+ * for extent allocation.
+ *
+ * This happens when mounting in degraded mode. For example we have a
+ * RAID1 filesystem with 2 devices, lose one device and mount the fs
+ * using the other device in degraded mode. If we then allocate a chunk,
+ * we may have enough free space in the existing system space_info, but
+ * none of the block groups can be used for extent allocation since they
+ * have a RAID1 profile, and because we are in degraded mode with a
+ * single device, we are forced to allocate a new system chunk with a
+ * SINGLE profile. Making check_system_chunk() iterate over all system
+ * block groups and check if they have a usable profile and enough space
+ * can be slow on very large filesystems, so we tolerate the -ENOSPC and
+ * try again after forcing allocation of a new system chunk. Like this
+ * we avoid paying the cost of that search in normal circumstances, when
+ * we were not mounted in degraded mode;
+ *
+ * 2) We had enough free space info the system space_info, and one suitable
+ * block group to allocate from when we called check_system_chunk()
+ * above. However right after we called it, the only system block group
+ * with enough free space got turned into RO mode by a running scrub,
+ * and in this case we have to allocate a new one and retry. We only
+ * need do this allocate and retry once, since we have a transaction
+ * handle and scrub uses the commit root to search for block groups.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+out:
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
+}
+
/*
- * If force is CHUNK_ALLOC_FORCE:
+ * Chunk allocation is done in 2 phases:
+ *
+ * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
+ * the chunk, the chunk mapping, create its block group and add the items
+ * that belong in the chunk btree to it - more specifically, we need to
+ * update device items in the chunk btree and add a new chunk item to it.
+ *
+ * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
+ * group item to the extent btree and the device extent items to the devices
+ * btree.
+ *
+ * This is done to prevent deadlocks. For example when COWing a node from the
+ * extent btree we are holding a write lock on the node's parent and if we
+ * trigger chunk allocation and attempted to insert the new block group item
+ * in the extent btree right way, we could deadlock because the path for the
+ * insertion can include that parent node. At first glance it seems impossible
+ * to trigger chunk allocation after starting a transaction since tasks should
+ * reserve enough transaction units (metadata space), however while that is true
+ * most of the time, chunk allocation may still be triggered for several reasons:
+ *
+ * 1) When reserving metadata, we check if there is enough free space in the
+ * metadata space_info and therefore don't trigger allocation of a new chunk.
+ * However later when the task actually tries to COW an extent buffer from
+ * the extent btree or from the device btree for example, it is forced to
+ * allocate a new block group (chunk) because the only one that had enough
+ * free space was just turned to RO mode by a running scrub for example (or
+ * device replace, block group reclaim thread, etc), so we can not use it
+ * for allocating an extent and end up being forced to allocate a new one;
+ *
+ * 2) Because we only check that the metadata space_info has enough free bytes,
+ * we end up not allocating a new metadata chunk in that case. However if
+ * the filesystem was mounted in degraded mode, none of the existing block
+ * groups might be suitable for extent allocation due to their incompatible
+ * profile (for e.g. mounting a 2 devices filesystem, where all block groups
+ * use a RAID1 profile, in degraded mode using a single device). In this case
+ * when the task attempts to COW some extent buffer of the extent btree for
+ * example, it will trigger allocation of a new metadata block group with a
+ * suitable profile (SINGLE profile in the example of the degraded mount of
+ * the RAID1 filesystem);
+ *
+ * 3) The task has reserved enough transaction units / metadata space, but when
+ * it attempts to COW an extent buffer from the extent or device btree for
+ * example, it does not find any free extent in any metadata block group,
+ * therefore forced to try to allocate a new metadata block group.
+ * This is because some other task allocated all available extents in the
+ * meanwhile - this typically happens with tasks that don't reserve space
+ * properly, either intentionally or as a bug. One example where this is
+ * done intentionally is fsync, as it does not reserve any transaction units
+ * and ends up allocating a variable number of metadata extents for log
+ * tree extent buffers.
+ *
+ * We also need this 2 phases setup when adding a device to a filesystem with
+ * a seed device - we must create new metadata and system chunks without adding
+ * any of the block group items to the chunk, extent and device btrees. If we
+ * did not do it this way, we would get ENOSPC when attempting to update those
+ * btrees, since all the chunks from the seed device are read-only.
+ *
+ * Phase 1 does the updates and insertions to the chunk btree because if we had
+ * it done in phase 2 and have a thundering herd of tasks allocating chunks in
+ * parallel, we risk having too many system chunks allocated by many tasks if
+ * many tasks reach phase 1 without the previous ones completing phase 2. In the
+ * extreme case this leads to exhaustion of the system chunk array in the
+ * superblock. This is easier to trigger if using a btree node/leaf size of 64K
+ * and with RAID filesystems (so we have more device items in the chunk btree).
+ * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
+ * the system chunk array due to concurrent allocations") provides more details.
+ *
+ * For allocation of system chunks, we defer the updates and insertions into the
+ * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
+ * if the chunk allocation is triggered while COWing an extent buffer of the
+ * chunk btree, we are holding a lock on the parent of that extent buffer and
+ * doing the chunk btree updates and insertions can require locking that parent.
+ * This is for the very few and rare cases where we update the chunk btree that
+ * are not chunk allocation or chunk removal: adding a device, removing a device
+ * or resizing a device.
+ *
+ * The reservation of system space, done through check_system_chunk(), as well
+ * as all the updates and insertions into the chunk btree must be done while
+ * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
+ * an extent buffer from the chunks btree we never trigger allocation of a new
+ * system chunk, which would result in a deadlock (trying to lock twice an
+ * extent buffer of the chunk btree, first time before triggering the chunk
+ * allocation and the second time during chunk allocation while attempting to
+ * update the chunks btree). The system chunk array is also updated while holding
+ * that mutex. The same logic applies to removing chunks - we must reserve system
+ * space, update the chunk btree and the system chunk array in the superblock
+ * while holding fs_info->chunk_mutex.
+ *
+ * This function, btrfs_chunk_alloc(), belongs to phase 1.
+ *
+ * If @force is CHUNK_ALLOC_FORCE:
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
+ * If @force is NOT CHUNK_ALLOC_FORCE:
* - return 0 if it doesn't need to allocate a new chunk,
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
@@ -3253,6 +3464,13 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
@@ -3316,13 +3534,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
force_metadata_allocation(fs_info);
}
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
+ ret = do_chunk_alloc(trans, flags);
trans->allocating_chunk = false;
spin_lock(&space_info->lock);
@@ -3341,22 +3553,6 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
space_info->chunk_alloc = 0;
spin_unlock(&space_info->lock);
mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
return ret;
}
@@ -3409,14 +3605,31 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *bg;
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
+ *
+ * Also, if our caller is allocating a system chunk, do not
+ * attempt to insert the chunk item in the chunk btree, as we
+ * could deadlock on an extent buffer since our caller may be
+ * COWing an extent buffer from the chunk btree.
*/
- ret = btrfs_alloc_chunk(trans, flags);
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ /*
+ * If we fail to add the chunk item here, we end up
+ * trying again at phase 2 of chunk allocation, at
+ * btrfs_create_pending_block_groups(). So ignore
+ * any error here.
+ */
+ btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ }
}
if (!ret) {
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 7b927425dc71..c72a71efcb18 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -97,6 +97,7 @@ struct btrfs_block_group {
unsigned int removed:1;
unsigned int to_copy:1;
unsigned int relocating_repair:1;
+ unsigned int chunk_item_inserted:1;
int disk_cache_state;
@@ -268,8 +269,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work);
void btrfs_reclaim_bgs(struct btrfs_fs_info *fs_info);
void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg);
int btrfs_read_block_groups(struct btrfs_fs_info *info);
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size);
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
bool do_chunk_alloc);
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 4bc3ca2cbd7d..c5c08c87e130 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -364,49 +364,6 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
return 0;
}
-static struct extent_buffer *alloc_tree_block_no_bg_flush(
- struct btrfs_trans_handle *trans,
- struct btrfs_root *root,
- u64 parent_start,
- const struct btrfs_disk_key *disk_key,
- int level,
- u64 hint,
- u64 empty_size,
- enum btrfs_lock_nesting nest)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct extent_buffer *ret;
-
- /*
- * If we are COWing a node/leaf from the extent, chunk, device or free
- * space trees, make sure that we do not finish block group creation of
- * pending block groups. We do this to avoid a deadlock.
- * COWing can result in allocation of a new chunk, and flushing pending
- * block groups (btrfs_create_pending_block_groups()) can be triggered
- * when finishing allocation of a new chunk. Creation of a pending block
- * group modifies the extent, chunk, device and free space trees,
- * therefore we could deadlock with ourselves since we are holding a
- * lock on an extent buffer that btrfs_create_pending_block_groups() may
- * try to COW later.
- * For similar reasons, we also need to delay flushing pending block
- * groups when splitting a leaf or node, from one of those trees, since
- * we are holding a write lock on it and its parent or when inserting a
- * new root node for one of those trees.
- */
- if (root == fs_info->extent_root ||
- root == fs_info->chunk_root ||
- root == fs_info->dev_root ||
- root == fs_info->free_space_root)
- trans->can_flush_pending_bgs = false;
-
- ret = btrfs_alloc_tree_block(trans, root, parent_start,
- root->root_key.objectid, disk_key, level,
- hint, empty_size, nest);
- trans->can_flush_pending_bgs = true;
-
- return ret;
-}
-
/*
* does the dirty work in cow of a single block. The parent block (if
* supplied) is updated to point to the new cow copy. The new buffer is marked
@@ -455,8 +412,9 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
if ((root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) && parent)
parent_start = parent->start;
- cow = alloc_tree_block_no_bg_flush(trans, root, parent_start, &disk_key,
- level, search_start, empty_size, nest);
+ cow = btrfs_alloc_tree_block(trans, root, parent_start,
+ root->root_key.objectid, &disk_key, level,
+ search_start, empty_size, nest);
if (IS_ERR(cow))
return PTR_ERR(cow);
@@ -2458,9 +2416,9 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
else
btrfs_node_key(lower, &lower_key, 0);
- c = alloc_tree_block_no_bg_flush(trans, root, 0, &lower_key, level,
- root->node->start, 0,
- BTRFS_NESTING_NEW_ROOT);
+ c = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &lower_key, level, root->node->start, 0,
+ BTRFS_NESTING_NEW_ROOT);
if (IS_ERR(c))
return PTR_ERR(c);
@@ -2589,8 +2547,9 @@ static noinline int split_node(struct btrfs_trans_handle *trans,
mid = (c_nritems + 1) / 2;
btrfs_node_key(c, &disk_key, mid);
- split = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, level,
- c->start, 0, BTRFS_NESTING_SPLIT);
+ split = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, level, c->start, 0,
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(split))
return PTR_ERR(split);
@@ -3381,10 +3340,10 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans,
* BTRFS_NESTING_SPLIT_THE_SPLITTENING if we need to, but for now just
* use BTRFS_NESTING_NEW_ROOT.
*/
- right = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, 0,
- l->start, 0, num_doubles ?
- BTRFS_NESTING_NEW_ROOT :
- BTRFS_NESTING_SPLIT);
+ right = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, 0, l->start, 0,
+ num_doubles ? BTRFS_NESTING_NEW_ROOT :
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(right))
return PTR_ERR(right);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 443c348bc6f3..14b9fdc8aaa9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -254,8 +254,11 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
}
/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
+ * To be called after doing the chunk btree updates right after allocating a new
+ * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
+ * chunk after all chunk btree updates and after finishing the second phase of
+ * chunk allocation (btrfs_create_pending_block_groups()) in case some block
+ * group had its chunk item insertion delayed to the second phase.
*/
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
@@ -264,8 +267,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
if (!trans->chunk_bytes_reserved)
return;
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
trans->chunk_bytes_reserved = 0;
@@ -696,7 +697,6 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,
h->fs_info = root->fs_info;
h->type = type;
- h->can_flush_pending_bgs = true;
INIT_LIST_HEAD(&h->new_bgs);
smp_mb();
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index a18d67796b54..ba45065f9451 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -132,7 +132,7 @@ struct btrfs_trans_handle {
short aborted;
bool adding_csums;
bool allocating_chunk;
- bool can_flush_pending_bgs;
+ bool removing_chunk;
bool reloc_reserved;
bool in_fsync;
struct btrfs_root *root;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 782e16795bc4..c6c14315b1c9 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1745,19 +1745,14 @@ static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_handle_fs_error(fs_info, ret, "Slot search failed");
goto out;
}
*dev_extent_len = btrfs_dev_extent_length(leaf, extent);
ret = btrfs_del_item(trans, root, path);
- if (ret) {
- btrfs_handle_fs_error(fs_info, ret,
- "Failed to remove dev extent item");
- } else {
+ if (ret == 0)
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
- }
out:
btrfs_free_path(path);
return ret;
@@ -2942,7 +2937,7 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
u32 cur;
struct btrfs_key key;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
array_size = btrfs_super_sys_array_size(super_copy);
ptr = super_copy->sys_chunk_array;
@@ -2972,7 +2967,6 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
cur += len;
}
}
- mutex_unlock(&fs_info->chunk_mutex);
return ret;
}
@@ -3012,6 +3006,29 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,
return em;
}
+static int remove_chunk_item(struct btrfs_trans_handle *trans,
+ struct map_lookup *map, u64 chunk_offset)
+{
+ int i;
+
+ /*
+ * Removing chunk items and updating the device items in the chunks btree
+ * requires holding the chunk_mutex.
+ * See the comment at btrfs_chunk_alloc() for the details.
+ */
+ lockdep_assert_held(&trans->fs_info->chunk_mutex);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ int ret;
+
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ if (ret)
+ return ret;
+ }
+
+ return btrfs_free_chunk(trans, chunk_offset);
+}
+
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3032,14 +3049,16 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(em);
}
map = em->map_lookup;
- mutex_lock(&fs_info->chunk_mutex);
- check_system_chunk(trans, map->type);
- mutex_unlock(&fs_info->chunk_mutex);
/*
- * Take the device list mutex to prevent races with the final phase of
- * a device replace operation that replaces the device object associated
- * with map stripes (dev-replace.c:btrfs_dev_replace_finishing()).
+ * First delete the device extent items from the devices btree.
+ * We take the device_list_mutex to avoid racing with the finishing phase
+ * of a device replace operation. See the comment below before acquiring
+ * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
+ * because that can result in a deadlock when deleting the device extent
+ * items from the devices btree - COWing an extent buffer from the btree
+ * may result in allocating a new metadata chunk, which would attempt to
+ * lock again fs_info->chunk_mutex.
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
@@ -3061,18 +3080,73 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
btrfs_clear_space_info_full(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
}
+ }
+ mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_update_device(trans, device);
+ /*
+ * We acquire fs_info->chunk_mutex for 2 reasons:
+ *
+ * 1) Just like with the first phase of the chunk allocation, we must
+ * reserve system space, do all chunk btree updates and deletions, and
+ * update the system chunk array in the superblock while holding this
+ * mutex. This is for similar reasons as explained on the comment at
+ * the top of btrfs_chunk_alloc();
+ *
+ * 2) Prevent races with the final phase of a device replace operation
+ * that replaces the device object associated with the map's stripes,
+ * because the device object's id can change at any time during that
+ * final phase of the device replace operation
+ * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
+ * replaced device and then see it with an ID of
+ * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
+ * the device item, which does not exists on the chunk btree.
+ * The finishing phase of device replace acquires both the
+ * device_list_mutex and the chunk_mutex, in that order, so we are
+ * safe by just acquiring the chunk_mutex.
+ */
+ trans->removing_chunk = true;
+ mutex_lock(&fs_info->chunk_mutex);
+
+ check_system_chunk(trans, map->type);
+
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ /*
+ * Normally we should not get -ENOSPC since we reserved space before
+ * through the call to check_system_chunk().
+ *
+ * Despite our system space_info having enough free space, we may not
+ * be able to allocate extents from its block groups, because all have
+ * an incompatible profile, which will force us to allocate a new system
+ * block group with the right profile, or right after we called
+ * check_system_space() above, a scrub turned the only system block group
+ * with enough free space into RO mode.
+ * This is explained with more detail at do_chunk_alloc().
+ *
+ * So if we get -ENOSPC, allocate a new system chunk and retry once.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
goto out;
}
- }
- mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_free_chunk(trans, chunk_offset);
- if (ret) {
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -3087,6 +3161,15 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
}
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+
+ /*
+ * We are done with chunk btree updates and deletions, so release the
+ * system space we previously reserved (with check_system_chunk()).
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+
ret = btrfs_remove_block_group(trans, chunk_offset, em);
if (ret) {
btrfs_abort_transaction(trans, ret);
@@ -3094,6 +3177,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
out:
+ if (trans->removing_chunk) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ }
/* once for us */
free_extent_map(em);
return ret;
@@ -4860,13 +4947,12 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
u32 array_size;
u8 *ptr;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
+
array_size = btrfs_super_sys_array_size(super_copy);
if (array_size + item_size + sizeof(disk_key)
- > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
- mutex_unlock(&fs_info->chunk_mutex);
+ > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
return -EFBIG;
- }
ptr = super_copy->sys_chunk_array + array_size;
btrfs_cpu_key_to_disk(&disk_key, key);
@@ -4875,7 +4961,6 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
memcpy(ptr, chunk, item_size);
item_size += sizeof(disk_key);
btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
- mutex_unlock(&fs_info->chunk_mutex);
return 0;
}
@@ -5225,13 +5310,14 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
}
}
-static int create_chunk(struct btrfs_trans_handle *trans,
+static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
{
struct btrfs_fs_info *info = trans->fs_info;
struct map_lookup *map = NULL;
struct extent_map_tree *em_tree;
+ struct btrfs_block_group *block_group;
struct extent_map *em;
u64 start = ctl->start;
u64 type = ctl->type;
@@ -5241,7 +5327,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
map = kmalloc(map_lookup_size(ctl->num_stripes), GFP_NOFS);
if (!map)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
map->num_stripes = ctl->num_stripes;
for (i = 0; i < ctl->ndevs; ++i) {
@@ -5263,7 +5349,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
em = alloc_extent_map();
if (!em) {
kfree(map);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
set_bit(EXTENT_FLAG_FS_MAPPING, &em->flags);
em->map_lookup = map;
@@ -5279,12 +5365,12 @@ static int create_chunk(struct btrfs_trans_handle *trans,
if (ret) {
write_unlock(&em_tree->lock);
free_extent_map(em);
- return ret;
+ return ERR_PTR(ret);
}
write_unlock(&em_tree->lock);
- ret = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
- if (ret)
+ block_group = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
+ if (IS_ERR(block_group))
goto error_del_extent;
for (i = 0; i < map->num_stripes; i++) {
@@ -5304,7 +5390,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
check_raid56_incompat_flag(info, type);
check_raid1c34_incompat_flag(info, type);
- return 0;
+ return block_group;
error_del_extent:
write_lock(&em_tree->lock);
@@ -5316,34 +5402,36 @@ static int create_chunk(struct btrfs_trans_handle *trans,
/* One for the tree reference */
free_extent_map(em);
- return ret;
+ return block_group;
}
-int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
+struct btrfs_block_group *btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ u64 type)
{
struct btrfs_fs_info *info = trans->fs_info;
struct btrfs_fs_devices *fs_devices = info->fs_devices;
struct btrfs_device_info *devices_info = NULL;
struct alloc_chunk_ctl ctl;
+ struct btrfs_block_group *block_group;
int ret;
lockdep_assert_held(&info->chunk_mutex);
if (!alloc_profile_is_valid(type, 0)) {
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (list_empty(&fs_devices->alloc_list)) {
if (btrfs_test_opt(info, ENOSPC_DEBUG))
btrfs_debug(info, "%s: no writable device", __func__);
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
}
if (!(type & BTRFS_BLOCK_GROUP_TYPE_MASK)) {
btrfs_err(info, "invalid chunk type 0x%llx requested", type);
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
ctl.start = find_next_chunk(info);
@@ -5353,46 +5441,43 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
devices_info = kcalloc(fs_devices->rw_devices, sizeof(*devices_info),
GFP_NOFS);
if (!devices_info)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = gather_device_info(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
ret = decide_stripe_size(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
- ret = create_chunk(trans, &ctl, devices_info);
+ block_group = create_chunk(trans, &ctl, devices_info);
out:
kfree(devices_info);
- return ret;
+ return block_group;
}
/*
- * Chunk allocation falls into two parts. The first part does work
]
]
|
| 173/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 28871
output: 43
thoughts: 445
Thoughts: **Analyzing the Locking Strategy in `btrfs_remove_chunk` and Proposed Fix**
Okay, let's dive into this. Initially, I was going to simply read the full commit description or the code itself to understand the locking nuances. But hold on, let's revisit that comment in `btrfs_remove_chunk`. Ah, yes: the device extent items are removed first from the devices btree. The `device_list_mutex` is taken to prevent racing with the final part of a device replacement, and we avoid taking `fs_info->chunk_mutex` at this point because of the potential for deadlock when deleting those device extent items; it could lead to metadata chunk allocation, which would then try to acquire `fs_info->chunk_mutex` again. Right, that's the key.
Now, let's see what the proposed fix suggests. It looks like the fix acquires `fs_info->chunk_mutex`, checks `chunk_bytes_reserved` and unlocks it, then calls `btrfs_remove_dev_extents`. Then `trans->removing_chunk` is set to `true`, and it re-acquires `fs_info->chunk_mutex`. The comment indicates that space has been reserved.
Interesting. That raises the question: what exactly does `btrfs_remove_dev_extents` do? It's essential to understand its locking behaviour to fully grasp the change. I need to track down the definition of `btrfs_remove_dev_extents`.
|
| 174/4 |
2026/08/08 03:49 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 175/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 30554
output: 20
thoughts: 3042
Thoughts: **Analyzing the `btrfs_remove_chunk` Logic and Proposed Fix**
Alright, let's break this down. I'm looking at `btrfs_remove_dev_extents` and how it interacts with chunk allocation. First, I see `btrfs_free_dev_extent` being called, deleting device extent items. This can trigger metadata chunk allocation because of potential COW operations on the devices btree. That allocation will call `btrfs_chunk_alloc`, and here's where it gets interesting.
I note that in the original code, `trans->removing_chunk` is set to `true` *after* `btrfs_remove_dev_extents` is called, which feels right because we want chunk allocation to be able to proceed within that function. The critical part is that chunk allocation fails if the `trans->removing_chunk` flag is already `true`. This protects the system from re-entering chunk allocation while it is already underway.
The proposed fix, however, seems to change the order. It locks `fs_info->chunk_mutex`, reserves some chunk space by using `check_system_chunk`, then *unlocks* `fs_info->chunk_mutex`, then calls `btrfs_remove_dev_extents`. This seems problematic at first. I'm trying to follow the execution flow.
`check_system_chunk` reserves system space by calling `reserve_chunk_space`. That will add bytes to `trans->chunk_bytes_reserved` and to `fs_info->chunk_block_rsv`. We then unlock the mutex. The function might need to call `do_chunk_alloc` which calls `check_system_chunk`.
Now, the main concern I see is that unlocking the mutex *after* the initial space reservation could let *another* thread step in and perform its *own* chunk allocation or removal, potentially using up the reserved space. But, is the other thread's state shared? My transaction handle `trans` is private to my thread, so that's okay. But what about `fs_info->chunk_block_rsv`?
The second thread will use that global `fs_info->chunk_block_rsv`. If the second thread does chunk allocation it *will* call `btrfs_trans_release_chunk_metadata`, so everything will work fine. I'm wondering if another thread might cause a conflict. That is, if my thread calls `check_system_chunk`, and reserves some bytes in `fs_info->chunk_block_rsv`, and I unlock the mutex, what if *another thread* calls `check_system_chunk` and tries to add even more bytes to `fs_info->chunk_block_rsv`?
Then I remember reason #2 from `btrfs_remove_chunk`'s comment regarding the mutex. It's about preventing races with the final phase of device replace operations. I now realize this is where the *real* problem could be. With the mutex unlocked, a device replacement could *complete* between our `check_system_chunk` and `remove_chunk_item` calls, potentially changing the device object IDs and causing a failure later in the process.
So, in the proposed fix, by unlocking the mutex, we may create a window where this race can happen. This means we're potentially breaking a critical invariant! Looking back at the original code, it calls `btrfs_remove_dev_extents` *without* holding the mutex, so the proposed fix is not making any change to it. This leaves me confident that unlocking the mutex before calling `btrfs_remove_dev_extents` is fine.
However, the bigger issue is in the proposed code where we are calling `check_system_chunk`, and reserving space *while holding the mutex*, and then *unlocking it*. This seems problematic if *other* threads are going to be using that same space! Another thread could consume space, and my thread would not be aware of it!
|
| 176/4 |
2026/08/08 03:49 |
tool |
grepper |
0m
Args: map[Expression:chunk_block_rsv]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4562- ret = btrfs_block_rsv_add(fs_info,
fs/btrfs/block-group.c:4563: &fs_info->chunk_block_rsv,
fs/btrfs/block-group.c-4564- bytes, BTRFS_RESERVE_NO_FLUSH);
--
fs/btrfs/block-rsv.c=415=void btrfs_init_root_block_rsv(struct btrfs_root *root)
--
fs/btrfs/block-rsv.c-432- case BTRFS_CHUNK_TREE_OBJECTID:
fs/btrfs/block-rsv.c:433: root->block_rsv = &fs_info->chunk_block_rsv;
fs/btrfs/block-rsv.c-434- break;
--
fs/btrfs/block-rsv.c=447=void btrfs_init_global_block_rsv(struct btrfs_fs_info *fs_info)
--
fs/btrfs/block-rsv.c-451- space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
fs/btrfs/block-rsv.c:452: fs_info->chunk_block_rsv.space_info = space_info;
fs/btrfs/block-rsv.c-453-
--
fs/btrfs/block-rsv.c=475=void btrfs_release_global_block_rsv(struct btrfs_fs_info *fs_info)
--
fs/btrfs/block-rsv.c-480- WARN_ON(fs_info->trans_block_rsv.reserved > 0);
fs/btrfs/block-rsv.c:481: WARN_ON(fs_info->chunk_block_rsv.size > 0);
fs/btrfs/block-rsv.c:482: WARN_ON(fs_info->chunk_block_rsv.reserved > 0);
fs/btrfs/block-rsv.c-483- WARN_ON(fs_info->remap_block_rsv.size > 0);
--
fs/btrfs/disk-io.c=2771=void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
--
fs/btrfs/disk-io.c-2834- btrfs_init_block_rsv(&fs_info->trans_block_rsv, BTRFS_BLOCK_RSV_TRANS);
fs/btrfs/disk-io.c:2835: btrfs_init_block_rsv(&fs_info->chunk_block_rsv, BTRFS_BLOCK_RSV_CHUNK);
fs/btrfs/disk-io.c-2836- btrfs_init_block_rsv(&fs_info->remap_block_rsv, BTRFS_BLOCK_RSV_REMAP);
--
fs/btrfs/fs.h=502=struct btrfs_fs_info {
--
fs/btrfs/fs.h-547- /* Block reservation for chunk tree */
fs/btrfs/fs.h:548: struct btrfs_block_rsv chunk_block_rsv;
fs/btrfs/fs.h-549- /* Block reservation for remap tree. */
--
fs/btrfs/space-info.c=635=static void dump_global_block_rsv(struct btrfs_fs_info *fs_info)
--
fs/btrfs/space-info.c-638- DUMP_BLOCK_RSV(fs_info, trans_block_rsv);
fs/btrfs/space-info.c:639: DUMP_BLOCK_RSV(fs_info, chunk_block_rsv);
fs/btrfs/space-info.c-640- DUMP_BLOCK_RSV(fs_info, remap_block_rsv);
--
fs/btrfs/transaction.c=255=void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
--
fs/btrfs/transaction.c-261-
fs/btrfs/transaction.c:262: btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
fs/btrfs/transaction.c-263- trans->chunk_bytes_reserved, NULL);
]
|
| 177/4 |
2026/08/08 03:49 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 35723
output: 37
thoughts: 5237
Thoughts: **A Deep Dive into Btrfs Chunk Removal Concurrency and Block Reservation**
Alright, let's break this down. My initial observation focuses on `fs/btrfs/block-rsv.c` and how the chunk tree (`chunk_root`) utilizes `fs_info->chunk_block_rsv`. This is straightforward; modifications to the chunk tree, like during chunk allocation or removal, draw from this reserve. The critical point is that we're dealing with a shared resource, and concurrency is my primary concern.
I quickly spot a potential race condition when calling `check_system_chunk` to reserve space and then unlocking `fs_info->chunk_mutex`. If another thread gets in there and allocates a chunk before the removal transaction can continue, it's very possible that the new thread modifies the chunk tree, eating up the reserved space in `fs_info->chunk_block_rsv`. This becomes problematic when we later lock the mutex again and call `remove_chunk_item`, because the reserved space may have been exhausted by the other thread!
Let's dig deeper: What operations actually modify the chunk tree, and do they all call `check_system_chunk` to correctly reserve space? I find that `do_chunk_alloc`, `btrfs_remove_chunk` and `btrfs_inc_block_group_ro` are the only functions explicitly calling `check_system_chunk`. Adding, removing, or resizing a device also modify the chunk tree, but the comments indicate they *don't* pre-reserve space. Those other operations modify the chunk tree directly, drawing from the block reserve. This means we're in danger of our reserved space getting stolen.
Now, a more insidious issue arises: consider what happens if `btrfs_remove_dev_extents` (which deletes device extents from the devices btree) requires *its* own chunk allocation. If it does, `do_chunk_alloc` is called. That in turn, calls `btrfs_trans_release_chunk_metadata`, which releases ALL of the reserved space, including the space we reserved at the start. So, our pre-reservation is defeated.
Let's carefully trace this: We start with space reserved, but after `btrfs_remove_dev_extents` triggers a chunk allocation, we have nothing reserved. Then when we re-lock `fs_info->chunk_mutex` and call `remove_chunk_item`, there's no space. This is a fatal flaw!
I also analyzed the proposed fix and there are flaws there, too. First, it needlessly returns `-ENOSPC` where a chunk removal might actually have succeeded anyway, based on how Btrfs operates. Second, the proposed fix removes the retry logic for `-ENOSPC` that the original code included, which means that the transaction is going to abort in a lot more situations.
Finally, I considered what happens if the chunk removal thread unlocks the mutex too early. Then a concurrent chunk allocation or removal could happen. I realized `btrfs_remove_dev_extents` does NOT hold the chunk_mutex and the other operations are serialized on the `device_list_mutex` so, concurrent operations are handled safely. However, this is a separate issue to that first flaw in the reservation mechanism.
|
| 178/4 |
2026/08/08 03:50 |
tool |
git-log |
0m
Args: map[CodeRegexp:check_system_chunk PathPrefix:fs/btrfs/volumes.c]
Results: map[Output:79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
451a2c130342 btrfs: Remove fs_info from check_system_chunk
2ff7e61e0d30 btrfs: take an fs_info directly when the root is not used otherwise
4617ea3a52cf Btrfs: fix necessary chunk tree space calculation when allocating a chunk
39c2d7faccc5 Btrfs: fix -ENOSPC on block group removal
]
|
| 179/4 |
2026/08/08 03:50 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 41846
output: 24
thoughts: 38
Thoughts: Here's my take on those thoughts, keeping in mind my expertise:
**Diving into that Btrfs `ENOSPC` Bug**
Alright, let's pull up that commit `39c2d7faccc5` ("Btrfs: fix -ENOSPC on block group removal"). I'm curious what's going on with this one. Seems like a block group removal issue leading to an `ENOSPC` error – always a fun debugging adventure in the world of filesystems. Let's get the code up and see what they were dealing with. I bet it has something to do with metadata updates or maybe some race condition related to space accounting, given my experience with Btrfs's particular complexities. Time to dissect those changes and see if I can understand the root cause and the fix.
|
| 180/4 |
2026/08/08 03:50 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 39c2d7faccc5ca5a1be682b01c0db5fafa8adeda
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed May 20 14:01:55 2015 +0100
Btrfs: fix -ENOSPC on block group removal
Unlike when attempting to allocate a new block group, where we check
that we have enough space in the system space_info to update the device
items and insert a new chunk item in the chunk tree, we were not checking
if the system space_info had enough space for updating the device items
and deleting the chunk item in the chunk tree. This often lead to -ENOSPC
error when attempting to allocate blocks for the chunk tree (during btree
node/leaf COW operations) while updating the device items or deleting the
chunk item, which resulted in the current transaction being aborted and
turning the filesystem into read-only mode.
While running fstests generic/038, which stresses allocation of block
groups and removal of unused block groups, with a large scratch device
(750Gb) this happened often, despite more than enough unallocated space,
and resulted in the following trace:
[68663.586604] WARNING: CPU: 3 PID: 1521 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x114 [btrfs]()
[68663.600407] BTRFS: Transaction aborted (error -28)
(...)
[68663.730829] Call Trace:
[68663.732585] [<ffffffff8142fa46>] dump_stack+0x4f/0x7b
[68663.734334] [<ffffffff8108b6a2>] ? console_unlock+0x361/0x3ad
[68663.739980] [<ffffffff81045ea5>] warn_slowpath_common+0xa1/0xbb
[68663.757153] [<ffffffffa036ca6d>] ? __btrfs_abort_transaction+0x52/0x114 [btrfs]
[68663.760925] [<ffffffff81045f05>] warn_slowpath_fmt+0x46/0x48
[68663.762854] [<ffffffffa03b159d>] ? btrfs_update_device+0x15a/0x16c [btrfs]
[68663.764073] [<ffffffffa036ca6d>] __btrfs_abort_transaction+0x52/0x114 [btrfs]
[68663.765130] [<ffffffffa03b3638>] btrfs_remove_chunk+0x597/0x5ee [btrfs]
[68663.765998] [<ffffffffa0384663>] ? btrfs_delete_unused_bgs+0x245/0x296 [btrfs]
[68663.767068] [<ffffffffa0384676>] btrfs_delete_unused_bgs+0x258/0x296 [btrfs]
[68663.768227] [<ffffffff8143527f>] ? _raw_spin_unlock_irq+0x2d/0x4c
[68663.769081] [<ffffffffa038b109>] cleaner_kthread+0x13d/0x16c [btrfs]
[68663.799485] [<ffffffffa038afcc>] ? btrfs_alloc_root+0x28/0x28 [btrfs]
[68663.809208] [<ffffffff8105f367>] kthread+0xef/0xf7
[68663.828795] [<ffffffff810e603f>] ? time_hardirqs_on+0x15/0x28
[68663.844942] [<ffffffff8105f278>] ? __kthread_parkme+0xad/0xad
[68663.846486] [<ffffffff81435a88>] ret_from_fork+0x58/0x90
[68663.847760] [<ffffffff8105f278>] ? __kthread_parkme+0xad/0xad
[68663.849503] ---[ end trace 798477c6d6dbaad6 ]---
[68663.850525] BTRFS: error (device sdc) in btrfs_remove_chunk:2652: errno=-28 No space left
So fix this by verifying that enough space exists in system space_info,
and reserving the space in the chunk block reserve, before attempting to
delete the block group and allocate a new system chunk if we don't have
enough space to perform the necessary updates and delete in the chunk
tree. Like for the block group creation case, we don't error our if we
fail to allocate a new system chunk, since we might end up not needing
it (no node/leaf splits happen during the COW operations and/or we end
up not needing to COW any btree nodes or leafs because they were already
COWed in the current transaction and their writeback didn't start yet).
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Chris Mason <clm@fb.com>
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 8ee5645ef9e1..92e908394403 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3516,6 +3516,10 @@ int btrfs_delayed_refs_qgroup_accounting(struct btrfs_trans_handle *trans,
int __get_raid_index(u64 flags);
int btrfs_start_write_no_snapshoting(struct btrfs_root *root);
void btrfs_end_write_no_snapshoting(struct btrfs_root *root);
+void check_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const u64 type,
+ const bool is_allocation);
/* ctree.c */
int btrfs_bin_search(struct extent_buffer *eb, struct btrfs_key *key,
int level, int *slot);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 4e08e47ace30..e78ab29f8f1b 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4092,7 +4092,7 @@ static int should_alloc_chunk(struct btrfs_root *root,
return 1;
}
-static u64 get_system_chunk_thresh(struct btrfs_root *root, u64 type)
+static u64 get_profile_num_devs(struct btrfs_root *root, u64 type)
{
u64 num_dev;
@@ -4106,17 +4106,24 @@ static u64 get_system_chunk_thresh(struct btrfs_root *root, u64 type)
else
num_dev = 1; /* DUP or single */
- /* metadata for updaing devices and chunk tree */
- return btrfs_calc_trans_metadata_size(root, num_dev + 1);
+ return num_dev;
}
-static void check_system_chunk(struct btrfs_trans_handle *trans,
- struct btrfs_root *root, u64 type)
+/*
+ * If @is_allocation is true, reserve space in the system space info necessary
+ * for allocating a chunk, otherwise if it's false, reserve space necessary for
+ * removing a chunk.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 type,
+ const bool is_allocation)
{
struct btrfs_space_info *info;
u64 left;
u64 thresh;
int ret = 0;
+ u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -4131,7 +4138,15 @@ static void check_system_chunk(struct btrfs_trans_handle *trans,
info->bytes_may_use;
spin_unlock(&info->lock);
- thresh = get_system_chunk_thresh(root, type);
+ num_devs = get_profile_num_devs(root, type);
+
+ /* num_devs device items to update and 1 chunk item to add or remove */
+ if (is_allocation)
+ thresh = btrfs_calc_trans_metadata_size(root, num_devs + 1);
+ else
+ thresh = btrfs_calc_trans_metadata_size(root, num_devs) +
+ btrfs_calc_trunc_metadata_size(root, 1);
+
if (left < thresh && btrfs_test_opt(root, ENOSPC_DEBUG)) {
btrfs_info(root->fs_info, "left=%llu, need=%llu, flags=%llu",
left, thresh, type);
@@ -4243,7 +4258,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans,
* Check if we have enough space in SYSTEM chunk because we may need
* to update devices.
*/
- check_system_chunk(trans, extent_root, flags);
+ check_system_chunk(trans, extent_root, flags, true);
ret = btrfs_alloc_chunk(trans, extent_root, flags);
trans->allocating_chunk = false;
@@ -8905,7 +8920,7 @@ int btrfs_set_block_group_ro(struct btrfs_root *root,
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = update_block_group_flags(root, cache->flags);
lock_chunks(root->fs_info->chunk_root);
- check_system_chunk(trans, root, alloc_flags);
+ check_system_chunk(trans, root, alloc_flags, true);
unlock_chunks(root->fs_info->chunk_root);
}
mutex_unlock(&root->fs_info->ro_block_group_mutex);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 534be440dd56..d7668756b9d0 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2625,6 +2625,9 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans,
return -EINVAL;
}
map = (struct map_lookup *)em->bdev;
+ lock_chunks(root->fs_info->chunk_root);
+ check_system_chunk(trans, extent_root, map->type, false);
+ unlock_chunks(root->fs_info->chunk_root);
for (i = 0; i < map->num_stripes; i++) {
struct btrfs_device *device = map->stripes[i].dev;
]
|
| 181/4 |
2026/08/08 03:50 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 45343
output: 17
thoughts: 4498
Thoughts: **Analysis of the Proposed Chunk Removal Fix**
Okay, here's the breakdown of my thinking on this proposed fix for chunk removal. This looks like a solid starting point, but let's dive into the details. My initial thought process involves understanding the reasoning behind changing the order of operations, specifically moving `check_system_chunk` earlier. The core idea seems to be preemptively checking for system space before deleting device extents, aiming to avoid transaction aborts.
Here's the problem: The original `btrfs_remove_chunk` function can lead to inconsistent state if `btrfs_remove_dev_extents` (which *removes* extents from the device B-tree) succeeds, but either `check_system_chunk` fails to reserve space (because it cannot create a new system chunk) or `remove_chunk_item` fails (e.g., due to an out of space condition when writing to the metadata B-tree). In such a scenario, the transaction will be aborted, the entire transaction is rolled back, and the device extents are restored. This results in a read-only filesystem.
The proposed fix aims to avoid transaction aborts and the subsequent read-only state. By moving `check_system_chunk` earlier, it intends to check if enough system space is available *before* the deletion of device extents and *before* setting `trans->removing_chunk` to `true`. This way, if the reservation fails, the function can return `-ENOSPC` early, without modifying the chunk tree and triggering the inconsistent state.
However, I've identified several *critical* flaws with this approach:
### Flaw 1: Metadata Space Leak on Error Path
The proposed fix introduces a metadata space leak. `check_system_chunk` is called at the beginning and reserves space for a new system chunk. However, if `btrfs_remove_dev_extents` fails (due to I/O error or other reasons), the code skips releasing the reservation and directly jumps to `out:`. This results in `trans->chunk_bytes_reserved` holding a reserved amount of bytes that is *never* released in the error case. This leads to a metadata leak, which can lead to metadata space exhaustion and unexpected errors later on.
### Flaw 2: Reserved Space Released Prematurely by Nested Chunk Allocations
The issue lies within the execution of `btrfs_remove_dev_extents`. This function can, and often does, *trigger* metadata chunk allocations itself (e.g., when it needs to write new metadata to the device B-tree, it may trigger an allocation). These nested chunk allocations end up calling `btrfs_trans_release_chunk_metadata`, which *unconditionally* releases all reserved metadata including the system space. This will lead to an incorrect state where the reservation performed early on is invalidated during `btrfs_remove_dev_extents`. When we later call `remove_chunk_item`, there's no space reservation, and we could easily hit `-ENOSPC`, leading to a transaction abort *anyway* because the reservation was defeated.
### Flaw 3: Race Condition / Space Stealing due to Unlocking `chunk_mutex`
The proposed fix unnecessarily unlocks `chunk_mutex` after `check_system_chunk` and *before* calling `btrfs_remove_dev_extents`. This creates a race condition. While `chunk_mutex` is unlocked, other threads may acquire it, and consume (or "steal") space from the chunk block reserve (`fs_info->chunk_block_rsv`). This reserved space could be consumed or used for other purposes by other threads. When the original thread reacquires `chunk_mutex` and proceeds to call `remove_chunk_item`, the reserved space might be gone, resulting in `-ENOSPC` and transaction aborts, despite the initial reservation.
### Flaw 4: Unnecessary `-ENOSPC` Failures on Reservation Failure
The proposed fix returns `-ENOSPC` immediately if `check_system_chunk` fails to reserve space. However, `check_system_chunk` might fail to reserve the new system chunk, but the chunk removal can still succeed and it may not actually need to allocate any additional system chunks because the paths it is going to touch are already COWed. The comment in `reserve_chunk_space` says: "... *Ignore failure to create system chunk. We might end up not needing it, as we might not need to COW all nodes/leafs from the paths we visit in the chunk tree (they were already COWed or created in the current transaction for example).*..." So, the proposed fix can lead to unnecessary `-ENOSPC` errors and failures in cases where the chunk removal would have otherwise succeeded.
The original code'
|
| 182/4 |
2026/08/08 03:50 |
tool |
grepper |
0m
Args: map[Expression:removing_chunk]
Results: map[Output:fs/btrfs/transaction.h=139=struct btrfs_trans_handle {
--
fs/btrfs/transaction.h-159- bool allocating_chunk;
fs/btrfs/transaction.h:160: bool removing_chunk;
fs/btrfs/transaction.h-161- bool reloc_reserved;
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3490- */
fs/btrfs/volumes.c:3491: trans->removing_chunk = true;
fs/btrfs/volumes.c-3492- mutex_lock(&fs_info->chunk_mutex);
--
fs/btrfs/volumes.c-3557- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3558: trans->removing_chunk = false;
fs/btrfs/volumes.c-3559-
--
fs/btrfs/volumes.c-3571-out:
fs/btrfs/volumes.c:3572: if (trans->removing_chunk) {
fs/btrfs/volumes.c-3573- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3574: trans->removing_chunk = false;
fs/btrfs/volumes.c-3575- }
]
|
| 183/4 |
2026/08/08 03:50 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 51293
output: 29
thoughts: 42
Thoughts: Okay, here's my interpretation of those thoughts, framed as my own:
**Investigating Chunk Removal**
Alright, I need to get to the bottom of this `removing_chunk` issue. First, I'll restrict my search to `block-group.c` or any related files, since that's where I'd expect to find the relevant code. *Pause*, nothing jumping out. Okay, let's broaden the search a bit. I'll scan the *entire* repository for `removing_chunk`. This is likely to give me the whole context, including call sites and potential interactions with other parts of the system. I'm after anything, from a simple function name, to a comment explaining the purpose of a particular section of the code, or a log statement.
|
| 184/4 |
2026/08/08 03:50 |
tool |
grepper |
0m
Args: map[Expression:removing_chunk PathPrefix:fs/btrfs/]
Results: map[Output:fs/btrfs/transaction.h=139=struct btrfs_trans_handle {
--
fs/btrfs/transaction.h-159- bool allocating_chunk;
fs/btrfs/transaction.h:160: bool removing_chunk;
fs/btrfs/transaction.h-161- bool reloc_reserved;
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3490- */
fs/btrfs/volumes.c:3491: trans->removing_chunk = true;
fs/btrfs/volumes.c-3492- mutex_lock(&fs_info->chunk_mutex);
--
fs/btrfs/volumes.c-3557- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3558: trans->removing_chunk = false;
fs/btrfs/volumes.c-3559-
--
fs/btrfs/volumes.c-3571-out:
fs/btrfs/volumes.c:3572: if (trans->removing_chunk) {
fs/btrfs/volumes.c-3573- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3574: trans->removing_chunk = false;
fs/btrfs/volumes.c-3575- }
]
|
| 185/4 |
2026/08/08 03:50 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 51940
output: 42
thoughts: 105
Thoughts: **Investigating Btrfs Chunk Allocation and Removal**
Okay, so I'm thinking I need to delve into how Btrfs handles chunk allocation and, specifically, removal. My initial thought is to search for instances of `removing_chunk` within the Btrfs filesystem code, specifically looking at `fs/btrfs/block-group.c`, but it's possible that this is handled elsewhere, so let's also search broadly.
Now, a related function that's probably key to all this is `btrfs_chunk_alloc`. I want to see how that function actually works. Is it in `fs/btrfs/block-group.c`? Or maybe it's in `fs/btrfs/volumes.c`? Where exactly is `btrfs_chunk_alloc` defined? It's essential to pinpoint its location to understand how it interfaces with chunk management. I need to find its definition before I can move forward.
|
| 186/4 |
2026/08/08 03:50 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:btrfs_chunk_alloc]
Results: map[SourceCode:4232: /*
4233: * Chunk allocation is done in 2 phases:
4234: *
4235: * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
4236: * the chunk, the chunk mapping, create its block group and add the items
4237: * that belong in the chunk btree to it - more specifically, we need to
4238: * update device items in the chunk btree and add a new chunk item to it.
4239: *
4240: * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
4241: * group item to the extent btree and the device extent items to the devices
4242: * btree.
4243: *
4244: * This is done to prevent deadlocks. For example when COWing a node from the
4245: * extent btree we are holding a write lock on the node's parent and if we
4246: * trigger chunk allocation and attempted to insert the new block group item
4247: * in the extent btree right way, we could deadlock because the path for the
4248: * insertion can include that parent node. At first glance it seems impossible
4249: * to trigger chunk allocation after starting a transaction since tasks should
4250: * reserve enough transaction units (metadata space), however while that is true
4251: * most of the time, chunk allocation may still be triggered for several reasons:
4252: *
4253: * 1) When reserving metadata, we check if there is enough free space in the
4254: * metadata space_info and therefore don't trigger allocation of a new chunk.
4255: * However later when the task actually tries to COW an extent buffer from
4256: * the extent btree or from the device btree for example, it is forced to
4257: * allocate a new block group (chunk) because the only one that had enough
4258: * free space was just turned to RO mode by a running scrub for example (or
4259: * device replace, block group reclaim thread, etc), so we can not use it
4260: * for allocating an extent and end up being forced to allocate a new one;
4261: *
4262: * 2) Because we only check that the metadata space_info has enough free bytes,
4263: * we end up not allocating a new metadata chunk in that case. However if
4264: * the filesystem was mounted in degraded mode, none of the existing block
4265: * groups might be suitable for extent allocation due to their incompatible
4266: * profile (for e.g. mounting a 2 devices filesystem, where all block groups
4267: * use a RAID1 profile, in degraded mode using a single device). In this case
4268: * when the task attempts to COW some extent buffer of the extent btree for
4269: * example, it will trigger allocation of a new metadata block group with a
4270: * suitable profile (SINGLE profile in the example of the degraded mount of
4271: * the RAID1 filesystem);
4272: *
4273: * 3) The task has reserved enough transaction units / metadata space, but when
4274: * it attempts to COW an extent buffer from the extent or device btree for
4275: * example, it does not find any free extent in any metadata block group,
4276: * therefore forced to try to allocate a new metadata block group.
4277: * This is because some other task allocated all available extents in the
4278: * meanwhile - this typically happens with tasks that don't reserve space
4279: * properly, either intentionally or as a bug. One example where this is
4280: * done intentionally is fsync, as it does not reserve any transaction units
4281: * and ends up allocating a variable number of metadata extents for log
4282: * tree extent buffers;
4283: *
4284: * 4) The task has reserved enough transaction units / metadata space, but right
4285: * before it tries to allocate the last extent buffer it needs, a discard
4286: * operation comes in and, temporarily, removes the last free space entry from
4287: * the only metadata block group that had free space (discard starts by
4288: * removing a free space entry from a block group, then does the discard
4289: * operation and, once it's done, it adds back the free space entry to the
4290: * block group).
4291: *
4292: * We also need this 2 phases setup when adding a device to a filesystem with
4293: * a seed device - we must create new metadata and system chunks without adding
4294: * any of the block group items to the chunk, extent and device btrees. If we
4295: * did not do it this way, we would get ENOSPC when attempting to update those
4296: * btrees, since all the chunks from the seed device are read-only.
4297: *
4298: * Phase 1 does the updates and insertions to the chunk btree because if we had
4299: * it done in phase 2 and have a thundering herd of tasks allocating chunks in
4300: * parallel, we risk having too many system chunks allocated by many tasks if
4301: * many tasks reach phase 1 without the previous ones completing phase 2. In the
4302: * extreme case this leads to exhaustion of the system chunk array in the
4303: * superblock. This is easier to trigger if using a btree node/leaf size of 64K
4304: * and with RAID filesystems (so we have more device items in the chunk btree).
4305: * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
4306: * the system chunk array due to concurrent allocations") provides more details.
4307: *
4308: * Allocation of system chunks does not happen through this function. A task that
4309: * needs to update the chunk btree (the only btree that uses system chunks), must
4310: * preallocate chunk space by calling either check_system_chunk() or
4311: * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
4312: * metadata chunk or when removing a chunk, while the later is used before doing
4313: * a modification to the chunk btree - use cases for the later are adding,
4314: * removing and resizing a device as well as relocation of a system chunk.
4315: * See the comment below for more details.
4316: *
4317: * The reservation of system space, done through check_system_chunk(), as well
4318: * as all the updates and insertions into the chunk btree must be done while
4319: * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
4320: * an extent buffer from the chunks btree we never trigger allocation of a new
4321: * system chunk, which would result in a deadlock (trying to lock twice an
4322: * extent buffer of the chunk btree, first time before triggering the chunk
4323: * allocation and the second time during chunk allocation while attempting to
4324: * update the chunks btree). The system chunk array is also updated while holding
4325: * that mutex. The same logic applies to removing chunks - we must reserve system
4326: * space, update the chunk btree and the system chunk array in the superblock
4327: * while holding fs_info->chunk_mutex.
4328: *
4329: * This function, btrfs_chunk_alloc(), belongs to phase 1.
4330: *
4331: * @space_info: specify which space_info the new chunk should belong to.
4332: *
4333: * If @force is CHUNK_ALLOC_FORCE:
4334: * - return 1 if it successfully allocates a chunk,
4335: * - return errors including -ENOSPC otherwise.
4336: * If @force is NOT CHUNK_ALLOC_FORCE:
4337: * - return 0 if it doesn't need to allocate a new chunk,
4338: * - return 1 if it successfully allocates a chunk,
4339: * - return errors including -ENOSPC otherwise.
4340: */
4341: int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
4342: struct btrfs_space_info *space_info, u64 flags,
4343: enum btrfs_chunk_alloc_enum force)
4344: {
4345: struct btrfs_fs_info *fs_info = trans->fs_info;
4346: struct btrfs_block_group *ret_bg;
4347: bool wait_for_alloc = false;
4348: bool should_alloc = false;
4349: bool from_extent_allocation = false;
4350: int ret = 0;
4351:
4352: if (force == CHUNK_ALLOC_FORCE_FOR_EXTENT) {
4353: from_extent_allocation = true;
4354: force = CHUNK_ALLOC_FORCE;
4355: }
4356:
4357: /* Don't re-enter if we're already allocating a chunk */
4358: if (trans->allocating_chunk)
4359: return -ENOSPC;
4360: /*
4361: * Allocation of system chunks can not happen through this path, as we
4362: * could end up in a deadlock if we are allocating a data or metadata
4363: * chunk and there is another task modifying the chunk btree.
4364: *
4365: * This is because while we are holding the chunk mutex, we will attempt
4366: * to add the new chunk item to the chunk btree or update an existing
4367: * device item in the chunk btree, while the other task that is modifying
4368: * the chunk btree is attempting to COW an extent buffer while holding a
4369: * lock on it and on its parent - if the COW operation triggers a system
4370: * chunk allocation, then we can deadlock because we are holding the
4371: * chunk mutex and we may need to access that extent buffer or its parent
4372: * in order to add the chunk item or update a device item.
4373: *
4374: * Tasks that want to modify the chunk tree should reserve system space
4375: * before updating the chunk btree, by calling either
4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
4377: * It's possible that after a task reserves the space, it still ends up
4378: * here - this happens in the cases described above at do_chunk_alloc().
4379: * The task will have to either retry or fail.
4380: */
4381: if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
4382: return -ENOSPC;
4383:
4384: do {
4385: spin_lock(&space_info->lock);
4386: if (force < space_info->force_alloc)
4387: force = space_info->force_alloc;
4388: should_alloc = should_alloc_chunk(fs_info, space_info, force);
4389: if (space_info->full) {
4390: /* No more free physical space */
4391: spin_unlock(&space_info->lock);
4392: if (should_alloc)
4393: ret = -ENOSPC;
4394: else
4395: ret = 0;
4396: return ret;
4397: } else if (!should_alloc) {
4398: spin_unlock(&space_info->lock);
4399: return 0;
4400: } else if (space_info->chunk_alloc) {
4401: /*
4402: * Someone is already allocating, so we need to block
4403: * until this someone is finished and then loop to
4404: * recheck if we should continue with our allocation
4405: * attempt.
4406: */
4407: spin_unlock(&space_info->lock);
4408: wait_for_alloc = true;
4409: force = CHUNK_ALLOC_NO_FORCE;
4410: mutex_lock(&fs_info->chunk_mutex);
4411: mutex_unlock(&fs_info->chunk_mutex);
4412: } else {
4413: /* Proceed with allocation */
4414: space_info->chunk_alloc = true;
4415: spin_unlock(&space_info->lock);
4416: wait_for_alloc = false;
4417: }
4418:
4419: cond_resched();
4420: } while (wait_for_alloc);
4421:
4422: mutex_lock(&fs_info->chunk_mutex);
4423: trans->allocating_chunk = true;
4424:
4425: /*
4426: * If we have mixed data/metadata chunks we want to make sure we keep
4427: * allocating mixed chunks instead of individual chunks.
4428: */
4429: if (btrfs_mixed_space_info(space_info))
4430: flags |= (BTRFS_BLOCK_GROUP_DATA | BTRFS_BLOCK_GROUP_METADATA);
4431:
4432: /*
4433: * if we're doing a data chunk, go ahead and make sure that
4434: * we keep a reasonable number of metadata chunks allocated in the
4435: * FS as well.
4436: */
4437: if (flags & BTRFS_BLOCK_GROUP_DATA && fs_info->metadata_ratio) {
4438: fs_info->data_chunk_allocations++;
4439: if (!(fs_info->data_chunk_allocations %
4440: fs_info->metadata_ratio))
4441: force_metadata_allocation(fs_info);
4442: }
4443:
4444: ret_bg = do_chunk_alloc(trans, space_info, flags);
4445: trans->allocating_chunk = false;
4446:
4447: if (IS_ERR(ret_bg)) {
4448: ret = PTR_ERR(ret_bg);
4449: } else if (from_extent_allocation && (flags & BTRFS_BLOCK_GROUP_DATA)) {
4450: /*
4451: * New block group is likely to be used soon. Try to activate
4452: * it now. Failure is OK for now.
4453: */
4454: btrfs_zone_activate(ret_bg);
4455: }
4456:
4457: if (!ret)
4458: btrfs_put_block_group(ret_bg);
4459:
4460: spin_lock(&space_info->lock);
4461: if (ret < 0) {
4462: if (ret == -ENOSPC)
4463: space_info->full = true;
4464: else
4465: goto out;
4466: } else {
4467: ret = 1;
4468: space_info->max_extent_size = 0;
4469: }
4470:
4471: space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
4472: out:
4473: space_info->chunk_alloc = false;
4474: spin_unlock(&space_info->lock);
4475: mutex_unlock(&fs_info->chunk_mutex);
4476:
4477: return ret;
4478: }
SourceFile:fs/btrfs/block-group.c]
|
| 187/4 |
2026/08/08 03:50 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 56727
output: 34
thoughts: 390
Thoughts: **Investigating the `btrfs_chunk_alloc` Logic**
Okay, I need to revisit `btrfs_chunk_alloc`. My initial concern is whether the code currently checks `trans->removing_chunk`. Let's confirm by examining lines 4357-4360:
```c
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
```
Right, hold on... does `btrfs_chunk_alloc` actually check for `trans->removing_chunk`? No! Looking at the current implementation, there's absolutely no check for `trans->removing_chunk`. Now, why is that?
Okay, let's trace the history. The commit `79bd37120b14` is the key:
```diff
- /* Don't re-enter if we're already allocating a chunk */
- if (trans->allocating_chunk)
- return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
```
Wait, but I just confirmed that the *current* code in `btrfs_chunk_alloc` that I just pulled up via `codesearch-definition-source` *doesn't* have that `trans->removing_chunk` check! This is definitely worth looking into.
Let's see if I can find `removing_chunk` within `fs/btrfs/block-group.c` using `grepper`. I need to figure out how this logic was changed and if this is a regression.
|
| 188/4 |
2026/08/08 03:51 |
tool |
grepper |
0m
Error:
Args: map[Expression:removing_chunk PathPrefix:fs/btrfs/block-group.c]
Results:
|
| 189/4 |
2026/08/08 03:51 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 57593
output: 32
thoughts: 75
Thoughts: **Investigating the Unused Variable**
Okay, so I'm diving into this code, and it looks like `trans->removing_chunk` isn't actually being used anywhere in `block-group.c` anymore. That's a bit odd. I need to figure out why. Did someone remove it intentionally? Or was it renamed or refactored into something else? I need to track down its history. The best way to do that is to consult the version control system. I'll fire up `git log` and search the repository history for any commits that touch that particular variable name, `removing_chunk`. Let's see what the log reveals.
|
| 190/4 |
2026/08/08 03:51 |
tool |
git-log |
0m
Args: map[CodeRegexp:removing_chunk PathPrefix:fs/btrfs/]
Results: map[Output:2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
]
|
| 191/4 |
2026/08/08 03:51 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 57906
output: 25
thoughts: 43
Thoughts: Okay, let's dive into commit `2bb2e00ed978`, "btrfs: fix deadlock between chunk allocation and chunk btree modifications." I need to see exactly what's being addressed here, so I can understand the implications on the btrfs filesystem's concurrency model. Let's pull up the details and get a good look at this one.
|
| 192/4 |
2026/08/08 03:51 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed Oct 13 10:12:49 2021 +0100
btrfs: fix deadlock between chunk allocation and chunk btree modifications
When a task is doing some modification to the chunk btree and it is not in
the context of a chunk allocation or a chunk removal, it can deadlock with
another task that is currently allocating a new data or metadata chunk.
These contexts are the following:
* When relocating a system chunk, when we need to COW the extent buffers
that belong to the chunk btree;
* When adding a new device (ioctl), where we need to add a new device item
to the chunk btree;
* When removing a device (ioctl), where we need to remove a device item
from the chunk btree;
* When resizing a device (ioctl), where we need to update a device item in
the chunk btree and may need to relocate a system chunk that lies beyond
the new device size when shrinking a device.
The problem happens due to a sequence of steps like the following:
1) Task A starts a data or metadata chunk allocation and it locks the
chunk mutex;
2) Task B is relocating a system chunk, and when it needs to COW an extent
buffer of the chunk btree, it has locked both that extent buffer as
well as its parent extent buffer;
3) Since there is not enough available system space, either because none
of the existing system block groups have enough free space or because
the only one with enough free space is in RO mode due to the relocation,
task B triggers a new system chunk allocation. It blocks when trying to
acquire the chunk mutex, currently held by task A;
4) Task A enters btrfs_chunk_alloc_add_chunk_item(), in order to insert
the new chunk item into the chunk btree and update the existing device
items there. But in order to do that, it has to lock the extent buffer
that task B locked at step 2, or its parent extent buffer, but task B
is waiting on the chunk mutex, which is currently locked by task A,
therefore resulting in a deadlock.
One example report when the deadlock happens with system chunk relocation:
INFO: task kworker/u9:5:546 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u9:5 state:D stack:25936 pid: 546 ppid: 2 flags:0x00004000
Workqueue: events_unbound btrfs_async_reclaim_metadata_space
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
rwsem_down_read_slowpath+0x4ee/0x9d0 kernel/locking/rwsem.c:993
__down_read_common kernel/locking/rwsem.c:1214 [inline]
__down_read kernel/locking/rwsem.c:1223 [inline]
down_read_nested+0xe6/0x440 kernel/locking/rwsem.c:1590
__btrfs_tree_read_lock+0x31/0x350 fs/btrfs/locking.c:47
btrfs_tree_read_lock fs/btrfs/locking.c:54 [inline]
btrfs_read_lock_root_node+0x8a/0x320 fs/btrfs/locking.c:191
btrfs_search_slot_get_root fs/btrfs/ctree.c:1623 [inline]
btrfs_search_slot+0x13b4/0x2140 fs/btrfs/ctree.c:1728
btrfs_update_device+0x11f/0x500 fs/btrfs/volumes.c:2794
btrfs_chunk_alloc_add_chunk_item+0x34d/0xea0 fs/btrfs/volumes.c:5504
do_chunk_alloc fs/btrfs/block-group.c:3408 [inline]
btrfs_chunk_alloc+0x84d/0xf50 fs/btrfs/block-group.c:3653
flush_space+0x54e/0xd80 fs/btrfs/space-info.c:670
btrfs_async_reclaim_metadata_space+0x396/0xa90 fs/btrfs/space-info.c:953
process_one_work+0x9df/0x16d0 kernel/workqueue.c:2297
worker_thread+0x90/0xed0 kernel/workqueue.c:2444
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
INFO: task syz-executor:9107 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor state:D stack:23200 pid: 9107 ppid: 7792 flags:0x00004004
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
__mutex_lock_common kernel/locking/mutex.c:669 [inline]
__mutex_lock+0xc96/0x1680 kernel/locking/mutex.c:729
btrfs_chunk_alloc+0x31a/0xf50 fs/btrfs/block-group.c:3631
find_free_extent_update_loop fs/btrfs/extent-tree.c:3986 [inline]
find_free_extent+0x25cb/0x3a30 fs/btrfs/extent-tree.c:4335
btrfs_reserve_extent+0x1f1/0x500 fs/btrfs/extent-tree.c:4415
btrfs_alloc_tree_block+0x203/0x1120 fs/btrfs/extent-tree.c:4813
__btrfs_cow_block+0x412/0x1620 fs/btrfs/ctree.c:415
btrfs_cow_block+0x2f6/0x8c0 fs/btrfs/ctree.c:570
btrfs_search_slot+0x1094/0x2140 fs/btrfs/ctree.c:1768
relocate_tree_block fs/btrfs/relocation.c:2694 [inline]
relocate_tree_blocks+0xf73/0x1770 fs/btrfs/relocation.c:2757
relocate_block_group+0x47e/0xc70 fs/btrfs/relocation.c:3673
btrfs_relocate_block_group+0x48a/0xc60 fs/btrfs/relocation.c:4070
btrfs_relocate_chunk+0x96/0x280 fs/btrfs/volumes.c:3181
__btrfs_balance fs/btrfs/volumes.c:3911 [inline]
btrfs_balance+0x1f03/0x3cd0 fs/btrfs/volumes.c:4301
btrfs_ioctl_balance+0x61e/0x800 fs/btrfs/ioctl.c:4137
btrfs_ioctl+0x39ea/0x7b70 fs/btrfs/ioctl.c:4949
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:874 [inline]
__se_sys_ioctl fs/ioctl.c:860 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:860
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
So fix this by making sure that whenever we try to modify the chunk btree
and we are neither in a chunk allocation context nor in a chunk remove
context, we reserve system space before modifying the chunk btree.
Reported-by: Hao Sun <sunhao.th@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/CACkBjsax51i4mu6C0C3vJqQN3NR_iVuucoeG3U1HXjrgzn5FFQ@mail.gmail.com/
Fixes: 79bd37120b1495 ("btrfs: rework chunk allocation to avoid exhaustion of the system chunk array")
CC: stable@vger.kernel.org # 5.14+
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index de9aeb3733cf..f971d043469c 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3425,25 +3425,6 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
goto out;
}
- /*
- * If this is a system chunk allocation then stop right here and do not
- * add the chunk item to the chunk btree. This is to prevent a deadlock
- * because this system chunk allocation can be triggered while COWing
- * some extent buffer of the chunk btree and while holding a lock on a
- * parent extent buffer, in which case attempting to insert the chunk
- * item (or update the device item) would result in a deadlock on that
- * parent extent buffer. In this case defer the chunk btree updates to
- * the second phase of chunk allocation and keep our reservation until
- * the second phase completes.
- *
- * This is a rare case and can only be triggered by the very few cases
- * we have where we need to touch the chunk btree outside chunk allocation
- * and chunk removal. These cases are basically adding a device, removing
- * a device or resizing a device.
- */
- if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
- return 0;
-
ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
/*
* Normally we are not expected to fail with -ENOSPC here, since we have
@@ -3576,14 +3557,14 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
* This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
* the system chunk array due to concurrent allocations") provides more details.
*
- * For allocation of system chunks, we defer the updates and insertions into the
- * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
- * if the chunk allocation is triggered while COWing an extent buffer of the
- * chunk btree, we are holding a lock on the parent of that extent buffer and
- * doing the chunk btree updates and insertions can require locking that parent.
- * This is for the very few and rare cases where we update the chunk btree that
- * are not chunk allocation or chunk removal: adding a device, removing a device
- * or resizing a device.
+ * Allocation of system chunks does not happen through this function. A task that
+ * needs to update the chunk btree (the only btree that uses system chunks), must
+ * preallocate chunk space by calling either check_system_chunk() or
+ * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
+ * metadata chunk or when removing a chunk, while the later is used before doing
+ * a modification to the chunk btree - use cases for the later are adding,
+ * removing and resizing a device as well as relocation of a system chunk.
+ * See the comment below for more details.
*
* The reservation of system space, done through check_system_chunk(), as well
* as all the updates and insertions into the chunk btree must be done while
@@ -3620,11 +3601,27 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
if (trans->allocating_chunk)
return -ENOSPC;
/*
- * If we are removing a chunk, don't re-enter or we would deadlock.
- * System space reservation and system chunk allocation is done by the
- * chunk remove operation (btrfs_remove_chunk()).
+ * Allocation of system chunks can not happen through this path, as we
+ * could end up in a deadlock if we are allocating a data or metadata
+ * chunk and there is another task modifying the chunk btree.
+ *
+ * This is because while we are holding the chunk mutex, we will attempt
+ * to add the new chunk item to the chunk btree or update an existing
+ * device item in the chunk btree, while the other task that is modifying
+ * the chunk btree is attempting to COW an extent buffer while holding a
+ * lock on it and on its parent - if the COW operation triggers a system
+ * chunk allocation, then we can deadlock because we are holding the
+ * chunk mutex and we may need to access that extent buffer or its parent
+ * in order to add the chunk item or update a device item.
+ *
+ * Tasks that want to modify the chunk tree should reserve system space
+ * before updating the chunk btree, by calling either
+ * btrfs_reserve_chunk_metadata() or check_system_chunk().
+ * It's possible that after a task reserves the space, it still ends up
+ * here - this happens in the cases described above at do_chunk_alloc().
+ * The task will have to either retry or fail.
*/
- if (trans->removing_chunk)
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
@@ -3723,17 +3720,14 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-/*
- * Reserve space in the system space for allocating or removing a chunk
- */
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+static void reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
- u64 thresh;
int ret = 0;
- u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -3746,19 +3740,13 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
- num_devs = get_profile_num_devs(fs_info, type);
-
- /* num_devs device items to update and 1 chunk item to add or remove */
- thresh = btrfs_calc_metadata_size(fs_info, num_devs) +
- btrfs_calc_insert_metadata_size(fs_info, 1);
-
- if (left < thresh && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
- left, thresh, type);
+ left, bytes, type);
btrfs_dump_space_info(fs_info, info, 0, 0);
}
- if (left < thresh) {
+ if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
@@ -3767,21 +3755,20 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
- *
- * Also, if our caller is allocating a system chunk, do not
- * attempt to insert the chunk item in the chunk btree, as we
- * could deadlock on an extent buffer since our caller may be
- * COWing an extent buffer from the chunk btree.
*/
bg = btrfs_create_chunk(trans, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
- } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ } else {
/*
* If we fail to add the chunk item here, we end up
* trying again at phase 2 of chunk allocation, at
* btrfs_create_pending_block_groups(). So ignore
- * any error here.
+ * any error here. An ENOSPC here could happen, due to
+ * the cases described at do_chunk_alloc() - the system
+ * block group we just created was just turned into RO
+ * mode by a scrub for example, or a running discard
+ * temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
@@ -3790,12 +3777,61 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (!ret) {
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
- thresh, BTRFS_RESERVE_NO_FLUSH);
+ bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
- trans->chunk_bytes_reserved += thresh;
+ trans->chunk_bytes_reserved += bytes;
}
}
+/*
+ * Reserve space in the system space for allocating or removing a chunk.
+ * The caller must be holding fs_info->chunk_mutex.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ const u64 num_devs = get_profile_num_devs(fs_info, type);
+ u64 bytes;
+
+ /* num_devs device items to update and 1 chunk item to add or remove. */
+ bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
+ btrfs_calc_insert_metadata_size(fs_info, 1);
+
+ reserve_chunk_space(trans, bytes, type);
+}
+
+/*
+ * Reserve space in the system space, if needed, for doing a modification to the
+ * chunk btree.
+ *
+ * @trans: A transaction handle.
+ * @is_item_insertion: Indicate if the modification is for inserting a new item
+ * in the chunk btree or if it's for the deletion or update
+ * of an existing item.
+ *
+ * This is used in a context where we need to update the chunk btree outside
+ * block group allocation and removal, to avoid a deadlock with a concurrent
+ * task that is allocating a metadata or data block group and therefore needs to
+ * update the chunk btree while holding the chunk mutex. After the update to the
+ * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
+ *
+ */
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ u64 bytes;
+
+ if (is_item_insertion)
+ bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
+ else
+ bytes = btrfs_calc_metadata_size(fs_info, 1);
+
+ mutex_lock(&fs_info->chunk_mutex);
+ reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
+ mutex_unlock(&fs_info->chunk_mutex);
+}
+
void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
{
struct btrfs_block_group *block_group;
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 07f977d3816c..5878b7ce3b78 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -293,6 +293,8 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index fed823596248..33a0ee7ac590 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -2692,8 +2692,12 @@ static int relocate_tree_block(struct btrfs_trans_handle *trans,
list_add_tail(&node->list, &rc->backref_cache.changed);
} else {
path->lowest_level = node->level;
+ if (root == root->fs_info->chunk_root)
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, key, path, 0, 1);
btrfs_release_path(path);
+ if (root == root->fs_info->chunk_root)
+ btrfs_trans_release_chunk_metadata(trans);
if (ret > 0)
ret = 0;
}
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index debba6f04858..9eab8a741166 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1847,8 +1847,10 @@ static int btrfs_add_dev_item(struct btrfs_trans_handle *trans,
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, true);
ret = btrfs_insert_empty_item(trans, trans->fs_info->chunk_root, path,
&key, sizeof(*dev_item));
+ btrfs_trans_release_chunk_metadata(trans);
if (ret)
goto out;
@@ -1921,7 +1923,9 @@ static int btrfs_rm_dev_item(struct btrfs_device *device)
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret) {
if (ret > 0)
ret = -ENOENT;
@@ -2513,7 +2517,9 @@ static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
key.type = BTRFS_DEV_ITEM_KEY;
while (1) {
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0)
goto error;
@@ -2862,6 +2868,7 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
struct btrfs_super_block *super_copy = fs_info->super_copy;
u64 old_total;
u64 diff;
+ int ret;
if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state))
return -EACCES;
@@ -2890,7 +2897,11 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
&trans->transaction->dev_update_list);
mutex_unlock(&fs_info->chunk_mutex);
- return btrfs_update_device(trans, device);
+ btrfs_reserve_chunk_metadata(trans, false);
+ ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
}
static int btrfs_free_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
@@ -4925,8 +4936,10 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
round_down(old_total - diff, fs_info->sectorsize));
mutex_unlock(&fs_info->chunk_mutex);
+ btrfs_reserve_chunk_metadata(trans, false);
/* Now btrfs_update_device() will change the on-disk size. */
ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0) {
btrfs_abort_transaction(trans, ret);
btrfs_end_transaction(trans);
]
|
| 193/4 |
2026/08/08 03:51 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 65578
output: 37
thoughts: 1072
Thoughts: **Analyzing the Proposed Btrfs Chunk Removal Fix**
Okay, this is a fascinating area, and the proposed solution looks... complex. Let's dig into this proposed fix for the `-ENOSPC` issue in `btrfs_remove_chunk`, specifically during `remove_chunk_item` or `btrfs_remove_dev_extents`. I can see the intent, but my gut tells me this approach introduces several significant problems.
First, the core idea is to move `check_system_chunk` *before* `btrfs_remove_dev_extents`. This seems reasonable on the surface – pre-emptively ensure space is available. However, my immediate concern is a **metadata reservation leak**. If `check_system_chunk` reserves space, but `btrfs_remove_dev_extents` fails later, we're exiting without releasing that reserved space. The `out` block doesn't include the necessary `btrfs_trans_release_chunk_metadata(trans)` call. That's a serious flaw.
Then, there's a problem of **premature release by nested chunk allocations**. `btrfs_remove_dev_extents` can trigger *its own* chunk allocations. Critically, these nested allocations call `btrfs_trans_release_chunk_metadata(trans)` unconditionally within their completion. Because this unconditionally releases *all* chunk metadata reserved by the transaction handle, the space initially reserved at the beginning of `btrfs_remove_chunk` is prematurely released back to the global pool! This defeats the entire purpose of the pre-reservation and sets us up for `-ENOSPC` errors later in `remove_chunk_item`.
The third issue is a **race condition** and potential for "space stealing". The `chunk_mutex` is unlocked *after* `check_system_chunk` and before `btrfs_remove_dev_extents`. This creates a window where other threads can acquire the mutex, allocate chunks, modify the chunk tree, and potentially consume the very space we just reserved in `fs_info->chunk_block_rsv`. When we re-acquire the `chunk_mutex` and get to `remove_chunk_item`, the space might be gone, leading to unexpected `-ENOSPC` failures.
Finally, the proposed fix includes an **unnecessary `-ENOSPC` failure**. The code checks if `trans->chunk_bytes_reserved` didn't increase after `check_system_chunk` and immediately aborts. However, the documentation for `reserve_chunk_space` (specifically) states that a failure to *reserve* space should be ignored. Chunk removal *might* still succeed without needing to COW any new nodes, particularly if the necessary paths are already COWed in the current transaction. This early exit causes unnecessary failures in cases where the chunk removal would otherwise have worked.
Okay, let me re-examine the code snippet, it's worth reviewing the code again.
```c
mutex_lock(&fs_info->chunk_mutex);
u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
```
Let me double-check a field in `btrfs_trans_handle` since the code includes `trans->removing_chunk = true`. Let's confirm it still exists. I'll search the code.
```c
fs/btrfs/transaction.h:160: bool removing_chunk;
```
Aha, good. `grepper` found it. Let's make sure by reading `fs/btrfs/transaction.h`.
|
| 194/4 |
2026/08/08 03:51 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/transaction.h FirstLine:150 LineCount:20]
Results: map[Contents: 150: struct btrfs_pending_snapshot *pending_snapshot;
151: refcount_t use_count;
152: unsigned int type;
153: /*
154: * Error code of transaction abort, set outside of locks and must use
155: * the READ_ONCE/WRITE_ONCE access
156: */
157: short aborted;
158: bool adding_csums;
159: bool allocating_chunk;
160: bool removing_chunk;
161: bool reloc_reserved;
162: bool in_fsync;
163: struct btrfs_fs_info *fs_info;
164: struct list_head new_bgs;
165: struct btrfs_block_rsv delayed_rsv;
166: /* Extent buffers with writeback inhibited by this handle. */
167: struct xarray writeback_inhibited_ebs;
168: };
169:
]
|
| 195/4 |
2026/08/08 03:51 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 67935
output: 38
thoughts: 84
Thoughts: Okay, here's my summary, taking the first-person perspective:
**Investigating the Persistence of `removing_chunk`**
Right, so I was double-checking something and I've just confirmed it: `removing_chunk` *is* still present within the `struct btrfs_trans_handle` structure. Now, that's got me thinking... why is it still hanging around? There must be a reason.
Let's go back and revisit the `btrfs_remove_chunk` function, specifically within `fs/btrfs/volumes.c`. I need to refresh my memory, and maybe spot something I missed before. Let's focus around line 3491 and the surrounding code. Time to dig in and re-evaluate the context.
|
| 196/4 |
2026/08/08 03:51 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3480 LineCount:30]
Results: map[Contents:3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
]
|
| 197/4 |
2026/08/08 03:51 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 68798
output: 696
thoughts: 2658
Thoughts: **Critical Flaws in the Proposed `btrfs_remove_chunk` Modification**
As an expert in the field, I've thoroughly examined the proposed changes to `btrfs_remove_chunk` and identified several severe flaws that compromise the filesystem's correctness. The original approach, while complex, was designed to avoid these very issues, and the suggested modifications reintroduce them, leading to potential data corruption and filesystem instability.
Here's a detailed breakdown of the critical flaws:
1. **Flaw 1: Metadata Reservation Leak on Error Path**
The proposed code introduces a metadata reservation leak when an error occurs during the chunk removal process. Let's trace the execution:
* `btrfs_remove_chunk` begins.
* `check_system_chunk` is called to reserve metadata space. If this call succeeds, `trans->chunk_bytes_reserved` will be updated with the reserved amount.
* If `check_system_chunk` fails *after* calling `btrfs_reserve_chunk_metadata`, it will return an error, and the code jumps to `goto out`.
* Crucially, the `check_system_chunk` failure is treated as a fatal error, preventing any further progress.
* The `out:` label, however, contains `btrfs_trans_release_chunk_metadata(trans);`, which *releases* the chunk metadata reservation, even though the intended process was never completed.
* If `check_system_chunk` fails to reserve space due to the lack of available metadata space, a system chunk may not be allocated, but the block reserve would still be released, causing a race condition in the filesystem.
This means any failure within this function, including a failed reservation, results in releasing the reserved space before the chunk removal process can be completed, potentially leading to metadata inconsistencies or even the inability to perform the necessary COW operations later because of a lack of free space.
2. **Flaw 2: Premature Release of Reserved Space by Nested Chunk Allocations**
The proposed change introduces a critical flaw related to nested chunk allocations and the premature release of reserved metadata space. Consider this scenario:
* `btrfs_remove_chunk` reserves metadata space (using `check_system_chunk` -> `btrfs_reserve_chunk_metadata`).
* `mutex_lock(&fs_info->chunk_mutex)` is acquired.
* The code proceeds to remove chunk items, which, in turn, can involve modifications to the chunk tree and potentially trigger COW operations.
* These COW operations, in certain situations (e.g., deleting a device extent), *can* trigger a nested chunk allocation via `btrfs_free_dev_extent`, which in turn, might call `btrfs_chunk_alloc` and ultimately calls `do_chunk_alloc`.
* Within `do_chunk_alloc`, `btrfs_trans_release_chunk_metadata(trans)` is called as part of its error handling path (or completion of allocation).
* This releases the *initially* reserved metadata space of `btrfs_remove_chunk`.
* `btrfs_remove_chunk` then attempts to complete its operations, including `remove_chunk_item`, *without* a valid metadata reservation.
* The later `remove_chunk_item` call will potentially fail because there may not be enough free space for the device btree modification.
This nested allocation pattern and premature release completely break the reservation logic.
3. **Flaw 3: Race Condition / Space Stealing due to Unlocking `chunk_mutex`**
The suggested fix contains a race condition that allows other threads to potentially "steal" the reserved metadata space, leading to space exhaustion and corruption. The following sequence occurs:
* `btrfs_remove_chunk` calls `check_system_chunk`, which reserves space in `fs_info->chunk_block_rsv`.
* `btrfs_remove_chunk` locks `chunk_mutex`.
* After the reservation, the code *unlocks* `chunk_mutex`.
* Other threads can now potentially modify the chunk tree while `btrfs_remove_chunk` is still running.
* A competing thread can allocate metadata from the shared `fs_info->chunk_block_rsv`.
* `btrfs_remove_chunk` continues, but its reserved space might be partially or completely exhausted by the competing thread, leading to potential out-of-space errors or corruptions.
This race window is extremely dangerous and can lead to space exhaustion errors and corruptions.
|
| 198/4 |
2026/08/08 03:51 |
tool |
set-results |
0m
Args: map[Answer:Yes, there are several critical flaws in this proposed fix for `btrfs_remove_chunk`.
### 1. Metadata Reservation Leak on the Error Path
In the proposed fix, `check_system_chunk` is called at the very beginning of the function, which reserves system space and increases `trans->chunk_bytes_reserved`. If `btrfs_remove_dev_extents(trans, map)` subsequently fails, the code executes `goto out;`.
However, the `out` block does not call `btrfs_trans_release_chunk_metadata(trans)`. Therefore, the reserved metadata space is leaked in the transaction handle, which can lead to metadata space exhaustion or warnings/assertions when the transaction is committed or the transaction handle is freed.
### 2. Premature Release of Reserved Space by Nested Chunk Allocations
`btrfs_remove_dev_extents` deletes device extents from the devices btree. This operation can trigger a metadata chunk allocation (via `do_chunk_alloc`) if metadata space is low.
At the end of `do_chunk_alloc`, `btrfs_trans_release_chunk_metadata(trans)` is called unconditionally, which releases **all** chunk metadata reserved by the transaction handle (`trans->chunk_bytes_reserved` is reset to `0`).
As a result, the system space reserved at the beginning of `btrfs_remove_chunk` is prematurely released back to the global pool. When `remove_chunk_item` is later called, there will be `0` bytes reserved, defeating the entire purpose of the pre-reservation and risking transaction aborts due to `-ENOSPC`.
### 3. Race Condition / Space Stealing due to Unlocking `chunk_mutex`
`fs_info->chunk_block_rsv` is a global block reserve shared among all threads. In the proposed fix, `chunk_mutex` is unlocked after `check_system_chunk` and before `btrfs_remove_dev_extents`.
While `chunk_mutex` is unlocked, other concurrent threads can acquire the mutex and perform chunk allocations or other operations that modify the chunk tree. These operations can consume/steal the space we just reserved in `fs_info->chunk_block_rsv`. When we later re-acquire `chunk_mutex` and call `remove_chunk_item`, the space might no longer be available, leading to unexpected `-ENOSPC` failures and transaction aborts.
### 4. Unnecessary `-ENOSPC` Failures on Reservation Failure
The proposed fix checks if `trans->chunk_bytes_reserved` did not increase after calling `check_system_chunk`, and if so, immediately aborts and returns `-ENOSPC`.
However, as documented in `reserve_chunk_space`, a failure to reserve system space (e.g., failure to allocate a new system chunk) should be ignored because the chunk removal might still succeed without needing to COW any new nodes/leafs (e.g., if the paths are already COWed in the current transaction). Returning `-ENOSPC` early causes the chunk removal to fail unnecessarily in scenarios where it would have otherwise succeeded.]
Results: map[Answer:Yes, there are several critical flaws in this proposed fix for `btrfs_remove_chunk`.
### 1. Metadata Reservation Leak on the Error Path
In the proposed fix, `check_system_chunk` is called at the very beginning of the function, which reserves system space and increases `trans->chunk_bytes_reserved`. If `btrfs_remove_dev_extents(trans, map)` subsequently fails, the code executes `goto out;`.
However, the `out` block does not call `btrfs_trans_release_chunk_metadata(trans)`. Therefore, the reserved metadata space is leaked in the transaction handle, which can lead to metadata space exhaustion or warnings/assertions when the transaction is committed or the transaction handle is freed.
### 2. Premature Release of Reserved Space by Nested Chunk Allocations
`btrfs_remove_dev_extents` deletes device extents from the devices btree. This operation can trigger a metadata chunk allocation (via `do_chunk_alloc`) if metadata space is low.
At the end of `do_chunk_alloc`, `btrfs_trans_release_chunk_metadata(trans)` is called unconditionally, which releases **all** chunk metadata reserved by the transaction handle (`trans->chunk_bytes_reserved` is reset to `0`).
As a result, the system space reserved at the beginning of `btrfs_remove_chunk` is prematurely released back to the global pool. When `remove_chunk_item` is later called, there will be `0` bytes reserved, defeating the entire purpose of the pre-reservation and risking transaction aborts due to `-ENOSPC`.
### 3. Race Condition / Space Stealing due to Unlocking `chunk_mutex`
`fs_info->chunk_block_rsv` is a global block reserve shared among all threads. In the proposed fix, `chunk_mutex` is unlocked after `check_system_chunk` and before `btrfs_remove_dev_extents`.
While `chunk_mutex` is unlocked, other concurrent threads can acquire the mutex and perform chunk allocations or other operations that modify the chunk tree. These operations can consume/steal the space we just reserved in `fs_info->chunk_block_rsv`. When we later re-acquire `chunk_mutex` and call `remove_chunk_item`, the space might no longer be available, leading to unexpected `-ENOSPC` failures and transaction aborts.
### 4. Unnecessary `-ENOSPC` Failures on Reservation Failure
The proposed fix checks if `trans->chunk_bytes_reserved` did not increase after calling `check_system_chunk`, and if so, immediately aborts and returns `-ENOSPC`.
However, as documented in `reserve_chunk_space`, a failure to reserve system space (e.g., failure to allocate a new system chunk) should be ignored because the chunk removal might still succeed without needing to COW any new nodes/leafs (e.g., if the paths are already COWed in the current transaction). Returning `-ENOSPC` early causes the chunk removal to fail unnecessarily in scenarios where it would have otherwise succeeded.]
|
| 199/2 |
2026/08/08 03:51 |
llm |
debugger |
0m
Model:
Tokens:
input: 159213
output: 42
thoughts: 5002
Thoughts: **A Deep Dive into Btrfs Chunk Removal**
Okay, let's break this down. That expert brought up some really important points that I need to seriously consider. Initially, I thought that reserving space before `btrfs_remove_dev_extents` was the key. Now, it is clear that is a path to a flawed solution.
First, I had to be sure that I wasn't falling prey to a reservation leak if the removal of extents failed. The initial logic seemed sound – aborting the transaction and letting `btrfs_end_transaction` clean up. However, the expert pointed out an important piece: `btrfs_trans_release_chunk_metadata` might not be getting called. That, in itself, is not a huge problem.
But the real problem is that `do_chunk_alloc` calls `btrfs_trans_release_chunk_metadata` unconditionally. I see that if `btrfs_remove_dev_extents` triggers a chunk allocation, we're releasing *all* reserved chunk metadata, even the space we're trying to protect for the metadata item itself. That is a FATAL flaw. This completely invalidates any attempt to reserve space before that function.
The expert dismissed my initial concerns about the race condition related to unlocking `chunk_mutex`. After reviewing the block reservation mechanics, I'm now certain they are correct; the reservation is accounting-based, so space stealing isn't the concern. That is a relief.
The experts' comment regarding unnecessary `-ENOSPC` failures is something I'd already considered. I still believe that a graceful failure is a better approach than abruptly aborting the transaction.
My current proposed solution cannot work because the release of the space reservation after a nested allocation. That means that I *cannot* reserve space before `btrfs_remove_dev_extents`. Given this, and the fact that I cannot change the order of the functions, how do I actually fix this bug?
I need to understand why `btrfs_remove_chunk` aborts the transaction on `-ENOSPC` from `remove_chunk_item`. It is because the extents have already been removed from the device tree and so we cannot re-add the data.
I initially thought about re-adding the device extents if `remove_chunk_item` failed. That seems fraught with more complexity and potential `-ENOSPC` issues. It may also fail.
Instead, I had a flash of insight. What if I can prevent the `ENOSPC`? Can I change `check_system_chunk` to *not* ignore the error? Then, it would only fail if we *actually* need the space to create the chunk? But if `remove_chunk_item` fails with `-ENOSPC`, we *know* we need the space.
The issue arises in the subsequent chunk allocation, it tries to allocate a system chunk and the `find_free_dev_extent` function will search the commit root. That will see the BTRFS_DEV_EXTENT_KEY for the chunk we are removing, and so the allocation will fail.
I need to figure out how to "see" the freed space for the device extents. I've tried to figure out how to tell `find_free_dev_extent` to ignore a certain extent, but there's no way to pass the required data to that function.
The crucial turning point is recognizing the possibility of allocating the system chunk *before* calling `btrfs_remove_dev_extents`. If the device is full, then we *must* allocate the system chunk, as we need that space to remove the chunk item.
If we *cannot* allocate a system chunk, then it is impossible to remove the chunk.
Here is a plan:
First, Check for System Space: I can check if we have enough system space. If not, I'll attempt to allocate a system chunk before changing the device tree.
The `btrfs_can_reserve_system_space` function is the solution! The call will ensure there is enough space to reserve. It is far more robust than attempting an allocation later! If the system chunk allocation succeeds, then the current `remove_chunk` will be able to succeed.
Let's test this in code! I think it will look like this:
```c
/*
* Check if we have enough system space to remove the chunk.
* If we don't, and we can't allocate a new system chunk, we fail gracefully
* before modifying the device tree.
*/
mutex_lock(&fs_info->chunk_mutex);
if (!btrfs_can_reserve_system_space(trans, map->type)) {
mutex_
|
| 200/2 |
2026/08/08 03:52 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:btrfs_chunk_alloc]
Results: map[SourceCode:4232: /*
4233: * Chunk allocation is done in 2 phases:
4234: *
4235: * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
4236: * the chunk, the chunk mapping, create its block group and add the items
4237: * that belong in the chunk btree to it - more specifically, we need to
4238: * update device items in the chunk btree and add a new chunk item to it.
4239: *
4240: * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
4241: * group item to the extent btree and the device extent items to the devices
4242: * btree.
4243: *
4244: * This is done to prevent deadlocks. For example when COWing a node from the
4245: * extent btree we are holding a write lock on the node's parent and if we
4246: * trigger chunk allocation and attempted to insert the new block group item
4247: * in the extent btree right way, we could deadlock because the path for the
4248: * insertion can include that parent node. At first glance it seems impossible
4249: * to trigger chunk allocation after starting a transaction since tasks should
4250: * reserve enough transaction units (metadata space), however while that is true
4251: * most of the time, chunk allocation may still be triggered for several reasons:
4252: *
4253: * 1) When reserving metadata, we check if there is enough free space in the
4254: * metadata space_info and therefore don't trigger allocation of a new chunk.
4255: * However later when the task actually tries to COW an extent buffer from
4256: * the extent btree or from the device btree for example, it is forced to
4257: * allocate a new block group (chunk) because the only one that had enough
4258: * free space was just turned to RO mode by a running scrub for example (or
4259: * device replace, block group reclaim thread, etc), so we can not use it
4260: * for allocating an extent and end up being forced to allocate a new one;
4261: *
4262: * 2) Because we only check that the metadata space_info has enough free bytes,
4263: * we end up not allocating a new metadata chunk in that case. However if
4264: * the filesystem was mounted in degraded mode, none of the existing block
4265: * groups might be suitable for extent allocation due to their incompatible
4266: * profile (for e.g. mounting a 2 devices filesystem, where all block groups
4267: * use a RAID1 profile, in degraded mode using a single device). In this case
4268: * when the task attempts to COW some extent buffer of the extent btree for
4269: * example, it will trigger allocation of a new metadata block group with a
4270: * suitable profile (SINGLE profile in the example of the degraded mount of
4271: * the RAID1 filesystem);
4272: *
4273: * 3) The task has reserved enough transaction units / metadata space, but when
4274: * it attempts to COW an extent buffer from the extent or device btree for
4275: * example, it does not find any free extent in any metadata block group,
4276: * therefore forced to try to allocate a new metadata block group.
4277: * This is because some other task allocated all available extents in the
4278: * meanwhile - this typically happens with tasks that don't reserve space
4279: * properly, either intentionally or as a bug. One example where this is
4280: * done intentionally is fsync, as it does not reserve any transaction units
4281: * and ends up allocating a variable number of metadata extents for log
4282: * tree extent buffers;
4283: *
4284: * 4) The task has reserved enough transaction units / metadata space, but right
4285: * before it tries to allocate the last extent buffer it needs, a discard
4286: * operation comes in and, temporarily, removes the last free space entry from
4287: * the only metadata block group that had free space (discard starts by
4288: * removing a free space entry from a block group, then does the discard
4289: * operation and, once it's done, it adds back the free space entry to the
4290: * block group).
4291: *
4292: * We also need this 2 phases setup when adding a device to a filesystem with
4293: * a seed device - we must create new metadata and system chunks without adding
4294: * any of the block group items to the chunk, extent and device btrees. If we
4295: * did not do it this way, we would get ENOSPC when attempting to update those
4296: * btrees, since all the chunks from the seed device are read-only.
4297: *
4298: * Phase 1 does the updates and insertions to the chunk btree because if we had
4299: * it done in phase 2 and have a thundering herd of tasks allocating chunks in
4300: * parallel, we risk having too many system chunks allocated by many tasks if
4301: * many tasks reach phase 1 without the previous ones completing phase 2. In the
4302: * extreme case this leads to exhaustion of the system chunk array in the
4303: * superblock. This is easier to trigger if using a btree node/leaf size of 64K
4304: * and with RAID filesystems (so we have more device items in the chunk btree).
4305: * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
4306: * the system chunk array due to concurrent allocations") provides more details.
4307: *
4308: * Allocation of system chunks does not happen through this function. A task that
4309: * needs to update the chunk btree (the only btree that uses system chunks), must
4310: * preallocate chunk space by calling either check_system_chunk() or
4311: * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
4312: * metadata chunk or when removing a chunk, while the later is used before doing
4313: * a modification to the chunk btree - use cases for the later are adding,
4314: * removing and resizing a device as well as relocation of a system chunk.
4315: * See the comment below for more details.
4316: *
4317: * The reservation of system space, done through check_system_chunk(), as well
4318: * as all the updates and insertions into the chunk btree must be done while
4319: * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
4320: * an extent buffer from the chunks btree we never trigger allocation of a new
4321: * system chunk, which would result in a deadlock (trying to lock twice an
4322: * extent buffer of the chunk btree, first time before triggering the chunk
4323: * allocation and the second time during chunk allocation while attempting to
4324: * update the chunks btree). The system chunk array is also updated while holding
4325: * that mutex. The same logic applies to removing chunks - we must reserve system
4326: * space, update the chunk btree and the system chunk array in the superblock
4327: * while holding fs_info->chunk_mutex.
4328: *
4329: * This function, btrfs_chunk_alloc(), belongs to phase 1.
4330: *
4331: * @space_info: specify which space_info the new chunk should belong to.
4332: *
4333: * If @force is CHUNK_ALLOC_FORCE:
4334: * - return 1 if it successfully allocates a chunk,
4335: * - return errors including -ENOSPC otherwise.
4336: * If @force is NOT CHUNK_ALLOC_FORCE:
4337: * - return 0 if it doesn't need to allocate a new chunk,
4338: * - return 1 if it successfully allocates a chunk,
4339: * - return errors including -ENOSPC otherwise.
4340: */
4341: int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
4342: struct btrfs_space_info *space_info, u64 flags,
4343: enum btrfs_chunk_alloc_enum force)
4344: {
4345: struct btrfs_fs_info *fs_info = trans->fs_info;
4346: struct btrfs_block_group *ret_bg;
4347: bool wait_for_alloc = false;
4348: bool should_alloc = false;
4349: bool from_extent_allocation = false;
4350: int ret = 0;
4351:
4352: if (force == CHUNK_ALLOC_FORCE_FOR_EXTENT) {
4353: from_extent_allocation = true;
4354: force = CHUNK_ALLOC_FORCE;
4355: }
4356:
4357: /* Don't re-enter if we're already allocating a chunk */
4358: if (trans->allocating_chunk)
4359: return -ENOSPC;
4360: /*
4361: * Allocation of system chunks can not happen through this path, as we
4362: * could end up in a deadlock if we are allocating a data or metadata
4363: * chunk and there is another task modifying the chunk btree.
4364: *
4365: * This is because while we are holding the chunk mutex, we will attempt
4366: * to add the new chunk item to the chunk btree or update an existing
4367: * device item in the chunk btree, while the other task that is modifying
4368: * the chunk btree is attempting to COW an extent buffer while holding a
4369: * lock on it and on its parent - if the COW operation triggers a system
4370: * chunk allocation, then we can deadlock because we are holding the
4371: * chunk mutex and we may need to access that extent buffer or its parent
4372: * in order to add the chunk item or update a device item.
4373: *
4374: * Tasks that want to modify the chunk tree should reserve system space
4375: * before updating the chunk btree, by calling either
4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
4377: * It's possible that after a task reserves the space, it still ends up
4378: * here - this happens in the cases described above at do_chunk_alloc().
4379: * The task will have to either retry or fail.
4380: */
4381: if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
4382: return -ENOSPC;
4383:
4384: do {
4385: spin_lock(&space_info->lock);
4386: if (force < space_info->force_alloc)
4387: force = space_info->force_alloc;
4388: should_alloc = should_alloc_chunk(fs_info, space_info, force);
4389: if (space_info->full) {
4390: /* No more free physical space */
4391: spin_unlock(&space_info->lock);
4392: if (should_alloc)
4393: ret = -ENOSPC;
4394: else
4395: ret = 0;
4396: return ret;
4397: } else if (!should_alloc) {
4398: spin_unlock(&space_info->lock);
4399: return 0;
4400: } else if (space_info->chunk_alloc) {
4401: /*
4402: * Someone is already allocating, so we need to block
4403: * until this someone is finished and then loop to
4404: * recheck if we should continue with our allocation
4405: * attempt.
4406: */
4407: spin_unlock(&space_info->lock);
4408: wait_for_alloc = true;
4409: force = CHUNK_ALLOC_NO_FORCE;
4410: mutex_lock(&fs_info->chunk_mutex);
4411: mutex_unlock(&fs_info->chunk_mutex);
4412: } else {
4413: /* Proceed with allocation */
4414: space_info->chunk_alloc = true;
4415: spin_unlock(&space_info->lock);
4416: wait_for_alloc = false;
4417: }
4418:
4419: cond_resched();
4420: } while (wait_for_alloc);
4421:
4422: mutex_lock(&fs_info->chunk_mutex);
4423: trans->allocating_chunk = true;
4424:
4425: /*
4426: * If we have mixed data/metadata chunks we want to make sure we keep
4427: * allocating mixed chunks instead of individual chunks.
4428: */
4429: if (btrfs_mixed_space_info(space_info))
4430: flags |= (BTRFS_BLOCK_GROUP_DATA | BTRFS_BLOCK_GROUP_METADATA);
4431:
4432: /*
4433: * if we're doing a data chunk, go ahead and make sure that
4434: * we keep a reasonable number of metadata chunks allocated in the
4435: * FS as well.
4436: */
4437: if (flags & BTRFS_BLOCK_GROUP_DATA && fs_info->metadata_ratio) {
4438: fs_info->data_chunk_allocations++;
4439: if (!(fs_info->data_chunk_allocations %
4440: fs_info->metadata_ratio))
4441: force_metadata_allocation(fs_info);
4442: }
4443:
4444: ret_bg = do_chunk_alloc(trans, space_info, flags);
4445: trans->allocating_chunk = false;
4446:
4447: if (IS_ERR(ret_bg)) {
4448: ret = PTR_ERR(ret_bg);
4449: } else if (from_extent_allocation && (flags & BTRFS_BLOCK_GROUP_DATA)) {
4450: /*
4451: * New block group is likely to be used soon. Try to activate
4452: * it now. Failure is OK for now.
4453: */
4454: btrfs_zone_activate(ret_bg);
4455: }
4456:
4457: if (!ret)
4458: btrfs_put_block_group(ret_bg);
4459:
4460: spin_lock(&space_info->lock);
4461: if (ret < 0) {
4462: if (ret == -ENOSPC)
4463: space_info->full = true;
4464: else
4465: goto out;
4466: } else {
4467: ret = 1;
4468: space_info->max_extent_size = 0;
4469: }
4470:
4471: space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
4472: out:
4473: space_info->chunk_alloc = false;
4474: spin_unlock(&space_info->lock);
4475: mutex_unlock(&fs_info->chunk_mutex);
4476:
4477: return ret;
4478: }
SourceFile:fs/btrfs/block-group.c]
|
| 201/2 |
2026/08/08 03:52 |
llm |
debugger |
0m
Model:
Tokens:
input: 169715
output: 17
thoughts: 311
Thoughts: **Investigating Btrfs Chunk Allocation**
Okay, so it seems `btrfs_alloc_chunk` has dropped the `trans->removing_chunk` check. That's... interesting. I distinctly remember that check being there, specifically in the commit `79bd37120b149532af5b21953643ed74af69654f`. The code explicitly returned `-ENOSPC` if we were in the process of removing a chunk.
Now, looking at the current `btrfs_chunk_alloc` implementation, it's clear the check is gone. We have a check for `trans->allocating_chunk`, and a system chunk allocation check that returns `-ENOSPC` if the `BTRFS_BLOCK_GROUP_SYSTEM` flag is set. But no `trans->removing_chunk`. Hmm...
The implication is obvious: that flag, `trans->removing_chunk`, was removed entirely from `struct btrfs_trans_handle`. I need to confirm that that it's no longer there. Time to check the structure definition and see if it's been refactored or removed entirely.
|
| 202/2 |
2026/08/08 03:52 |
tool |
grepper |
0m
Args: map[Expression:removing_chunk]
Results: map[Output:fs/btrfs/transaction.h=139=struct btrfs_trans_handle {
--
fs/btrfs/transaction.h-159- bool allocating_chunk;
fs/btrfs/transaction.h:160: bool removing_chunk;
fs/btrfs/transaction.h-161- bool reloc_reserved;
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3490- */
fs/btrfs/volumes.c:3491: trans->removing_chunk = true;
fs/btrfs/volumes.c-3492- mutex_lock(&fs_info->chunk_mutex);
--
fs/btrfs/volumes.c-3557- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3558: trans->removing_chunk = false;
fs/btrfs/volumes.c-3559-
--
fs/btrfs/volumes.c-3571-out:
fs/btrfs/volumes.c:3572: if (trans->removing_chunk) {
fs/btrfs/volumes.c-3573- mutex_unlock(&fs_info->chunk_mutex);
fs/btrfs/volumes.c:3574: trans->removing_chunk = false;
fs/btrfs/volumes.c-3575- }
]
|
| 203/2 |
2026/08/08 03:52 |
llm |
debugger-compressor |
0m
Model:
Reply: ### 1. Approaches Tried and Results
* **Analysis of the Crash Path**:
* The crash is a transaction abort (`-ENOSPC`) at `fs/btrfs/volumes.c:3526` inside `btrfs_remove_chunk` during a balance/relocation operation.
* `btrfs_remove_chunk` calls `btrfs_remove_dev_extents` to delete device extents from the device tree, then calls `check_system_chunk` to reserve system space, and finally `remove_chunk_item` to delete the chunk item from the chunk tree.
* If `remove_chunk_item` fails with `-ENOSPC` (due to needing to COW a chunk tree node when system space is exhausted), it attempts to allocate a new system chunk via `btrfs_create_chunk`.
* `btrfs_create_chunk` calls `find_free_dev_extent`, which searches the *commit root* (`path->search_commit_root = true`). Consequently, the device space just freed by `btrfs_remove_dev_extents` in the current transaction is invisible to the allocator.
* If the device is otherwise fully allocated, `btrfs_create_chunk` fails with `-ENOSPC`, triggering a transaction abort.
* **Dead-end: Swapping the order of `remove_chunk_item` and `btrfs_remove_dev_extents`**:
* *Reasoning*: If we delete the chunk item first and it fails with `-ENOSPC`, we could return the error gracefully without aborting because the device tree hasn't been modified yet.
* *Result*: Invalid. `remove_chunk_item` calls `btrfs_update_device`, which writes the current `device->bytes_used` to the chunk tree. If we call `remove_chunk_item` before `btrfs_remove_dev_extents`, it writes the *old* (pre-decrement) `bytes_used` value. The transaction commit phase (`btrfs_commit_device_sizes`) only updates in-memory fields (`commit_bytes_used`) and does not rewrite the chunk tree. Thus, `btrfs_remove_dev_extents` must run before `remove_chunk_item`.
* **Dead-end: Changing `check_system_chunk` or `reserve_chunk_space` to return an error**:
* *Reasoning*: Propagate the allocation failure from `btrfs_create_chunk` inside `reserve_chunk_space`.
* *Result*: Unfavorable. `check_system_chunk` is called by `do_chunk_alloc` where ignoring the failure is intentional (the system space might not actually be needed if the path nodes are already COWed). Forcing an error here would fail operations that could otherwise succeed.
---
### 2. Current Hypotheses and Active Lines of Investigation
* **Hypothesis**: We can detect system space reservation failure *before* modifying the device tree by calling `check_system_chunk` early in `btrfs_remove_chunk`.
* **Mechanism**:
* `check_system_chunk` attempts to reserve space in `chunk_block_rsv` and increments `trans->chunk_bytes_reserved` on success. If it fails to allocate a system chunk, it does not increment this counter.
* By caching `trans->chunk_bytes_reserved` before calling `check_system_chunk` and comparing it afterward, we can detect reservation failure.
* If reservation fails, we can return `-ENOSPC` gracefully *before* calling `btrfs_remove_dev_extents`, avoiding transaction abort. The empty block group will simply remain and can be retried by the cleaner thread later.
---
### 3. Critical File Paths and Code Snippets
* **File Path**: `fs/btrfs/volumes.c`
* **File Path**: `fs/btrfs/block-group.c`
#### Proposed Fix in `fs/btrfs/volumes.c`:
```c
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_chunk_map *map;
u64 chunk_bytes_reserved;
int ret;
map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
if (IS_ERR(map)) {
DEBUG_WARN("errr %ld reading chunk map at offset %llu",
PTR_ERR(map), chunk_offset);
return PTR_ERR(map);
}
/*
* Reserve system space before removing the device extents.
* If we fail to reserve space, we can fail gracefully.
* If we remove the device extents first and then fail to reserve space,
* we would have to abort the transaction.
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
chunk_bytes_reserved = trans->chunk_bytes_reserved;
check_system_chunk(trans, map->type);
if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = -ENOSPC;
goto out;
}
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
* [existing comments...]
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
/* We already reserved space, so we don't need to call check_system_chunk here */
ret = remove_chunk_item(trans, map, chunk_offset);
/*
* Normally we should not get -ENOSPC since we reserved space before
* through the call to check_system_chunk().
* [existing retry logic...]
*/
```
---
### 4. Potential Reasoning Loops and Repetitive Tool Calls
* **Note**: Avoid searching further back in git history for the introduction of `search_commit_root = 1` / `search_commit_root = true` in `find_free_dev_extent`. This is a dead-end loop; the commit root search is a fundamental design requirement in Btrfs to prevent double-allocation of device extents before a transaction commits. The focus must remain on early reservation verification within `btrfs_remove_chunk`.
Tokens:
input: 169832
output: 1568
thoughts: 1568
|
| 204/2 |
2026/08/08 03:53 |
llm |
debugger |
0m
Model:
Tokens:
input: 41215
output: 57
thoughts: 18
Thoughts: Okay, here's my summary, in the first person, as if these are my thoughts, keeping in mind I'm an expert in my field:
**Spotting a Key Detail**
Alright, that's a relief! I was worried that `trans->removing_chunk` might have been a typo or an outdated reference, but there it is – it actually exists. Good. Now, the next crucial step in understanding this process is to trace its usage. I need to figure out where this `trans->removing_chunk` is actually *being* used within the code. That will tell me everything about its context and significance. That’s where I need to start now.
|
| 205/2 |
2026/08/08 03:53 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/volumes.c IncludeSnippetLines:0 Name:btrfs_trans_handle::removing_chunk SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:writes ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3491] map[ReferenceKind:writes ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3558] map[ReferenceKind:reads ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3572] map[ReferenceKind:writes ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3574]] TruncatedOutput:false]
|
| 206/2 |
2026/08/08 03:53 |
llm |
debugger |
0m
Model:
Tokens:
input: 41669
output: 36
thoughts: 243
Thoughts: **Unraveling the Mystery of `trans->removing_chunk`**
Okay, hold on a second...this thing, `trans->removing_chunk`... it's *only* used in `btrfs_remove_chunk`? That seems...odd. Let's double-check `btrfs_chunk_alloc` one more time, just to be absolutely sure. Nope, not there. Okay, so is there *anywhere* else in the code that actually touches `trans->removing_chunk`, aside from that one function? Nope, not even a peep.
Wait a minute... if it's *only* written to and read from within `btrfs_remove_chunk`, what in the world is its purpose? Then it hits me! It's right there in the `out` label... it's a conditional unlock for `chunk_mutex`.
```c
out:
if (trans->removing_chunk) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
}
```
Oh! So, `trans->removing_chunk` is *just* a state variable, used for error handling within `btrfs_remove_chunk` itself! It's a flag to make sure `chunk_mutex` gets unlocked in the event of an error. It has absolutely *nothing* to do with preventing chunk allocations. I see now!
Wow, it looks like the previous understanding of this was completely incorrect. The speculation about `trans->removing_chunk` causing those spurious `-ENOSPC` errors in `btrfs_chunk_alloc` was completely unfounded. Well, at least now I know how this flag is actually used. Now, let's take a look at the history of the commit and see what its initial intentions were.
|
| 207/2 |
2026/08/08 03:53 |
tool |
git-log |
0m
Args: map[Count:10 PathPrefix:fs/btrfs/transaction.h SymbolName:]
Results: map[Output:513528a286e0 btrfs: simplify how first hit is passed to __btrfs_abort_transaction()
e6c249adb721 btrfs: validate negative error number passed to btrfs_abort_transaction()
0938971abc4f btrfs: move transaction abort message to __btrfs_abort_transaction()
f9a48549a15a btrfs: inhibit extent buffer writeback to prevent COW amplification
c5667f9c8eb9 btrfs: headers cleanup to remove unnecessary local includes
b815a78e17b9 btrfs: move abort_should_print_stack() to transaction.h
2144e1f23f98 btrfs: correct typos in multiple comments across various files
cd9253c23aed btrfs: fix race between direct IO write and fsync when using same fd
c154a8446bb7 btrfs: switch btrfs_pending_snapshot::dir to btrfs_inode
ded980eb3fad btrfs: add and use helper to commit the current transaction
]
|
| 208/2 |
2026/08/08 03:53 |
llm |
debugger |
0m
Model:
Tokens:
input: 42630
output: 37
thoughts: 15
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Investigating a Code Removal with Git**
Alright, I need to understand when and why a specific chunk of code was removed. My first instinct, given the context, is to jump straight into the git history. A simple `grep` search is the most direct and efficient approach here. Specifically, I'll target the string "removing_chunk" as my search term. That should quickly pinpoint any commits where that phrase, which likely relates to my target code block's removal, was mentioned. This could be in commit messages, diffs, or even within the code itself (though less likely). It's a quick way to narrow down the relevant commits without having to painstakingly sift through the entire history. From there, I can analyze the surrounding context of those commits to fully understand the rationale and the specific changes made. It's a good first step to reconstruct the removal's history.
|
| 209/2 |
2026/08/08 03:53 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:removing_chunk Count:10 PathPrefix: SymbolName:]
Results:
|
| 210/2 |
2026/08/08 04:03 |
llm |
debugger |
0m
Model:
Tokens:
input: 42919
output: 46
thoughts: 16
Thoughts: Here's a summary of my thought process:
**Diving into the Btrfs Volume Code**
Okay, so I need to understand something related to Btrfs volumes, and my gut feeling is that the relevant code is likely within the `fs/btrfs/` directory, specifically in a file dealing with volume management. Given the file names I know, `volumes.c` seems like a pretty good bet. I'm going to quickly use `grep` to search within that file. This should help me pinpoint the specific areas I need to focus on to further my investigation, without having to exhaustively read the entire codebase. Grepping for what I am investigating is a good place to start.
|
| 211/2 |
2026/08/08 04:03 |
tool |
git-log |
0m
Args: map[CodeRegexp:removing_chunk Count:10 PathPrefix:fs/btrfs/volumes.c SymbolName:]
Results: map[Output:79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
]
|
| 212/2 |
2026/08/08 04:03 |
llm |
debugger |
0m
Model:
Tokens:
input: 43169
output: 26
thoughts: 25
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**Decoding the Identifier**
Right, `79bd37120b14`. That's the key. Immediately, my mind jumps to a specific piece of data, likely a commit hash, a database transaction ID, or perhaps a unique identifier in some custom system. It's a hex value, so I know it's probably encoding something specific. Now, the real work begins... I need to reconstruct the context. My immediate focus is on retrieving the activity that's tied to this ID, what happened at that point and time. Let's dig in. I need to understand this identifier's function.
|
| 213/2 |
2026/08/08 04:04 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1281 lines.
[commit 79bd37120b149532af5b21953643ed74af69654f
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:06 2021 +0100
btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
Commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations") fixed a problem that resulted in
exhausting the system chunk array in the superblock when there are many
tasks allocating chunks in parallel. Basically too many tasks enter the
first phase of chunk allocation without previous tasks having finished
their second phase of allocation, resulting in too many system chunks
being allocated. That was originally observed when running the fallocate
tests of stress-ng on a PowerPC machine, using a node size of 64K.
However that commit also introduced a deadlock where a task in phase 1 of
the chunk allocation waited for another task that had allocated a system
chunk to finish its phase 2, but that other task was waiting on an extent
buffer lock held by the first task, therefore resulting in both tasks not
making any progress. That change was later reverted by a patch with the
subject "btrfs: fix deadlock with concurrent chunk allocations involving
system chunks", since there is no simple and short solution to address it
and the deadlock is relatively easy to trigger on zoned filesystems, while
the system chunk array exhaustion is not so common.
This change reworks the chunk allocation to avoid the system chunk array
exhaustion. It accomplishes that by making the first phase of chunk
allocation do the updates of the device items in the chunk btree and the
insertion of the new chunk item in the chunk btree. This is done while
under the protection of the chunk mutex (fs_info->chunk_mutex), in the
same critical section that checks for available system space, allocates
a new system chunk if needed and reserves system chunk space. This way
we do not have chunk space reserved until the second phase completes.
The same logic is applied to chunk removal as well, since it keeps
reserved system space long after it is done updating the chunk btree.
For direct allocation of system chunks, the previous behaviour remains,
because otherwise we would deadlock on extent buffers of the chunk btree.
Changes to the chunk btree are by large done by chunk allocation and chunk
removal, which first reserve chunk system space and then later do changes
to the chunk btree. The other remaining cases are uncommon and correspond
to adding a device, removing a device and resizing a device. All these
other cases do not pre-reserve system space, they modify the chunk btree
right away, so they don't hold reserved space for a long period like chunk
allocation and chunk removal do.
The diff of this change is huge, but more than half of it is just addition
of comments describing both how things work regarding chunk allocation and
removal, including both the new behavior and the parts of the old behavior
that did not change.
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a26209f98279..c557327b4545 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2207,6 +2207,13 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
return ret;
}
+/*
+ * This function, insert_block_group_item(), belongs to the phase 2 of chunk
+ * allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
static int insert_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_block_group *block_group)
{
@@ -2229,15 +2236,19 @@ static int insert_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_insert_item(trans, root, &key, &bgi, sizeof(bgi));
}
+/*
+ * This function, btrfs_create_pending_block_groups(), belongs to the phase 2 of
+ * chunk allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *block_group;
int ret = 0;
- if (!trans->can_flush_pending_bgs)
- return;
-
while (!list_empty(&trans->new_bgs)) {
int index;
@@ -2252,6 +2263,13 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
ret = insert_block_group_item(trans, block_group);
if (ret)
btrfs_abort_transaction(trans, ret);
+ if (!block_group->chunk_item_inserted) {
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, block_group);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret)
+ btrfs_abort_transaction(trans, ret);
+ }
ret = btrfs_finish_chunk_alloc(trans, block_group->start,
block_group->length);
if (ret)
@@ -2275,8 +2293,9 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
btrfs_trans_release_chunk_metadata(trans);
}
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size)
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *cache;
@@ -2286,7 +2305,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
cache = btrfs_create_block_group_cache(fs_info, chunk_offset);
if (!cache)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
cache->length = size;
set_free_space_tree_thresholds(cache);
@@ -2300,7 +2319,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
ret = btrfs_load_block_group_zone_info(cache, true);
if (ret) {
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
ret = exclude_super_stripes(cache);
@@ -2308,7 +2327,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
/* We may have excluded something, so call this just in case */
btrfs_free_excluded_extents(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
add_new_free_space(cache, chunk_offset, chunk_offset + size);
@@ -2335,7 +2354,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
if (ret) {
btrfs_remove_free_space_cache(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
/*
@@ -2354,7 +2373,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
btrfs_update_delayed_refs_rsv(trans);
set_avail_alloc_bits(fs_info, type);
- return 0;
+ return cache;
}
/*
@@ -3232,11 +3251,203 @@ int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type)
return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
+static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ /*
+ * Check if we have enough space in the system space info because we
+ * will need to update device items in the chunk btree and insert a new
+ * chunk item in the chunk btree as well. This will allocate a new
+ * system block group if needed.
+ */
+ check_system_chunk(trans, flags);
+
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ goto out;
+ }
+
+ /*
+ * If this is a system chunk allocation then stop right here and do not
+ * add the chunk item to the chunk btree. This is to prevent a deadlock
+ * because this system chunk allocation can be triggered while COWing
+ * some extent buffer of the chunk btree and while holding a lock on a
+ * parent extent buffer, in which case attempting to insert the chunk
+ * item (or update the device item) would result in a deadlock on that
+ * parent extent buffer. In this case defer the chunk btree updates to
+ * the second phase of chunk allocation and keep our reservation until
+ * the second phase completes.
+ *
+ * This is a rare case and can only be triggered by the very few cases
+ * we have where we need to touch the chunk btree outside chunk allocation
+ * and chunk removal. These cases are basically adding a device, removing
+ * a device or resizing a device.
+ */
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ return 0;
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ /*
+ * Normally we are not expected to fail with -ENOSPC here, since we have
+ * previously reserved space in the system space_info and allocated one
+ * new system chunk if necessary. However there are two exceptions:
+ *
+ * 1) We may have enough free space in the system space_info but all the
+ * existing system block groups have a profile which can not be used
+ * for extent allocation.
+ *
+ * This happens when mounting in degraded mode. For example we have a
+ * RAID1 filesystem with 2 devices, lose one device and mount the fs
+ * using the other device in degraded mode. If we then allocate a chunk,
+ * we may have enough free space in the existing system space_info, but
+ * none of the block groups can be used for extent allocation since they
+ * have a RAID1 profile, and because we are in degraded mode with a
+ * single device, we are forced to allocate a new system chunk with a
+ * SINGLE profile. Making check_system_chunk() iterate over all system
+ * block groups and check if they have a usable profile and enough space
+ * can be slow on very large filesystems, so we tolerate the -ENOSPC and
+ * try again after forcing allocation of a new system chunk. Like this
+ * we avoid paying the cost of that search in normal circumstances, when
+ * we were not mounted in degraded mode;
+ *
+ * 2) We had enough free space info the system space_info, and one suitable
+ * block group to allocate from when we called check_system_chunk()
+ * above. However right after we called it, the only system block group
+ * with enough free space got turned into RO mode by a running scrub,
+ * and in this case we have to allocate a new one and retry. We only
+ * need do this allocate and retry once, since we have a transaction
+ * handle and scrub uses the commit root to search for block groups.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+out:
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
+}
+
/*
- * If force is CHUNK_ALLOC_FORCE:
+ * Chunk allocation is done in 2 phases:
+ *
+ * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
+ * the chunk, the chunk mapping, create its block group and add the items
+ * that belong in the chunk btree to it - more specifically, we need to
+ * update device items in the chunk btree and add a new chunk item to it.
+ *
+ * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
+ * group item to the extent btree and the device extent items to the devices
+ * btree.
+ *
+ * This is done to prevent deadlocks. For example when COWing a node from the
+ * extent btree we are holding a write lock on the node's parent and if we
+ * trigger chunk allocation and attempted to insert the new block group item
+ * in the extent btree right way, we could deadlock because the path for the
+ * insertion can include that parent node. At first glance it seems impossible
+ * to trigger chunk allocation after starting a transaction since tasks should
+ * reserve enough transaction units (metadata space), however while that is true
+ * most of the time, chunk allocation may still be triggered for several reasons:
+ *
+ * 1) When reserving metadata, we check if there is enough free space in the
+ * metadata space_info and therefore don't trigger allocation of a new chunk.
+ * However later when the task actually tries to COW an extent buffer from
+ * the extent btree or from the device btree for example, it is forced to
+ * allocate a new block group (chunk) because the only one that had enough
+ * free space was just turned to RO mode by a running scrub for example (or
+ * device replace, block group reclaim thread, etc), so we can not use it
+ * for allocating an extent and end up being forced to allocate a new one;
+ *
+ * 2) Because we only check that the metadata space_info has enough free bytes,
+ * we end up not allocating a new metadata chunk in that case. However if
+ * the filesystem was mounted in degraded mode, none of the existing block
+ * groups might be suitable for extent allocation due to their incompatible
+ * profile (for e.g. mounting a 2 devices filesystem, where all block groups
+ * use a RAID1 profile, in degraded mode using a single device). In this case
+ * when the task attempts to COW some extent buffer of the extent btree for
+ * example, it will trigger allocation of a new metadata block group with a
+ * suitable profile (SINGLE profile in the example of the degraded mount of
+ * the RAID1 filesystem);
+ *
+ * 3) The task has reserved enough transaction units / metadata space, but when
+ * it attempts to COW an extent buffer from the extent or device btree for
+ * example, it does not find any free extent in any metadata block group,
+ * therefore forced to try to allocate a new metadata block group.
+ * This is because some other task allocated all available extents in the
+ * meanwhile - this typically happens with tasks that don't reserve space
+ * properly, either intentionally or as a bug. One example where this is
+ * done intentionally is fsync, as it does not reserve any transaction units
+ * and ends up allocating a variable number of metadata extents for log
+ * tree extent buffers.
+ *
+ * We also need this 2 phases setup when adding a device to a filesystem with
+ * a seed device - we must create new metadata and system chunks without adding
+ * any of the block group items to the chunk, extent and device btrees. If we
+ * did not do it this way, we would get ENOSPC when attempting to update those
+ * btrees, since all the chunks from the seed device are read-only.
+ *
+ * Phase 1 does the updates and insertions to the chunk btree because if we had
+ * it done in phase 2 and have a thundering herd of tasks allocating chunks in
+ * parallel, we risk having too many system chunks allocated by many tasks if
+ * many tasks reach phase 1 without the previous ones completing phase 2. In the
+ * extreme case this leads to exhaustion of the system chunk array in the
+ * superblock. This is easier to trigger if using a btree node/leaf size of 64K
+ * and with RAID filesystems (so we have more device items in the chunk btree).
+ * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
+ * the system chunk array due to concurrent allocations") provides more details.
+ *
+ * For allocation of system chunks, we defer the updates and insertions into the
+ * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
+ * if the chunk allocation is triggered while COWing an extent buffer of the
+ * chunk btree, we are holding a lock on the parent of that extent buffer and
+ * doing the chunk btree updates and insertions can require locking that parent.
+ * This is for the very few and rare cases where we update the chunk btree that
+ * are not chunk allocation or chunk removal: adding a device, removing a device
+ * or resizing a device.
+ *
+ * The reservation of system space, done through check_system_chunk(), as well
+ * as all the updates and insertions into the chunk btree must be done while
+ * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
+ * an extent buffer from the chunks btree we never trigger allocation of a new
+ * system chunk, which would result in a deadlock (trying to lock twice an
+ * extent buffer of the chunk btree, first time before triggering the chunk
+ * allocation and the second time during chunk allocation while attempting to
+ * update the chunks btree). The system chunk array is also updated while holding
+ * that mutex. The same logic applies to removing chunks - we must reserve system
+ * space, update the chunk btree and the system chunk array in the superblock
+ * while holding fs_info->chunk_mutex.
+ *
+ * This function, btrfs_chunk_alloc(), belongs to phase 1.
+ *
+ * If @force is CHUNK_ALLOC_FORCE:
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
+ * If @force is NOT CHUNK_ALLOC_FORCE:
* - return 0 if it doesn't need to allocate a new chunk,
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
@@ -3253,6 +3464,13 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
@@ -3316,13 +3534,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
force_metadata_allocation(fs_info);
}
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
+ ret = do_chunk_alloc(trans, flags);
trans->allocating_chunk = false;
spin_lock(&space_info->lock);
@@ -3341,22 +3553,6 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
space_info->chunk_alloc = 0;
spin_unlock(&space_info->lock);
mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
return ret;
}
@@ -3409,14 +3605,31 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *bg;
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
+ *
+ * Also, if our caller is allocating a system chunk, do not
+ * attempt to insert the chunk item in the chunk btree, as we
+ * could deadlock on an extent buffer since our caller may be
+ * COWing an extent buffer from the chunk btree.
*/
- ret = btrfs_alloc_chunk(trans, flags);
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ /*
+ * If we fail to add the chunk item here, we end up
+ * trying again at phase 2 of chunk allocation, at
+ * btrfs_create_pending_block_groups(). So ignore
+ * any error here.
+ */
+ btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ }
}
if (!ret) {
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 7b927425dc71..c72a71efcb18 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -97,6 +97,7 @@ struct btrfs_block_group {
unsigned int removed:1;
unsigned int to_copy:1;
unsigned int relocating_repair:1;
+ unsigned int chunk_item_inserted:1;
int disk_cache_state;
@@ -268,8 +269,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work);
void btrfs_reclaim_bgs(struct btrfs_fs_info *fs_info);
void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg);
int btrfs_read_block_groups(struct btrfs_fs_info *info);
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size);
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
bool do_chunk_alloc);
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 4bc3ca2cbd7d..c5c08c87e130 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -364,49 +364,6 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
return 0;
}
-static struct extent_buffer *alloc_tree_block_no_bg_flush(
- struct btrfs_trans_handle *trans,
- struct btrfs_root *root,
- u64 parent_start,
- const struct btrfs_disk_key *disk_key,
- int level,
- u64 hint,
- u64 empty_size,
- enum btrfs_lock_nesting nest)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct extent_buffer *ret;
-
- /*
- * If we are COWing a node/leaf from the extent, chunk, device or free
- * space trees, make sure that we do not finish block group creation of
- * pending block groups. We do this to avoid a deadlock.
- * COWing can result in allocation of a new chunk, and flushing pending
- * block groups (btrfs_create_pending_block_groups()) can be triggered
- * when finishing allocation of a new chunk. Creation of a pending block
- * group modifies the extent, chunk, device and free space trees,
- * therefore we could deadlock with ourselves since we are holding a
- * lock on an extent buffer that btrfs_create_pending_block_groups() may
- * try to COW later.
- * For similar reasons, we also need to delay flushing pending block
- * groups when splitting a leaf or node, from one of those trees, since
- * we are holding a write lock on it and its parent or when inserting a
- * new root node for one of those trees.
- */
- if (root == fs_info->extent_root ||
- root == fs_info->chunk_root ||
- root == fs_info->dev_root ||
- root == fs_info->free_space_root)
- trans->can_flush_pending_bgs = false;
-
- ret = btrfs_alloc_tree_block(trans, root, parent_start,
- root->root_key.objectid, disk_key, level,
- hint, empty_size, nest);
- trans->can_flush_pending_bgs = true;
-
- return ret;
-}
-
/*
* does the dirty work in cow of a single block. The parent block (if
* supplied) is updated to point to the new cow copy. The new buffer is marked
@@ -455,8 +412,9 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
if ((root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) && parent)
parent_start = parent->start;
- cow = alloc_tree_block_no_bg_flush(trans, root, parent_start, &disk_key,
- level, search_start, empty_size, nest);
+ cow = btrfs_alloc_tree_block(trans, root, parent_start,
+ root->root_key.objectid, &disk_key, level,
+ search_start, empty_size, nest);
if (IS_ERR(cow))
return PTR_ERR(cow);
@@ -2458,9 +2416,9 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
else
btrfs_node_key(lower, &lower_key, 0);
- c = alloc_tree_block_no_bg_flush(trans, root, 0, &lower_key, level,
- root->node->start, 0,
- BTRFS_NESTING_NEW_ROOT);
+ c = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &lower_key, level, root->node->start, 0,
+ BTRFS_NESTING_NEW_ROOT);
if (IS_ERR(c))
return PTR_ERR(c);
@@ -2589,8 +2547,9 @@ static noinline int split_node(struct btrfs_trans_handle *trans,
mid = (c_nritems + 1) / 2;
btrfs_node_key(c, &disk_key, mid);
- split = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, level,
- c->start, 0, BTRFS_NESTING_SPLIT);
+ split = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, level, c->start, 0,
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(split))
return PTR_ERR(split);
@@ -3381,10 +3340,10 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans,
* BTRFS_NESTING_SPLIT_THE_SPLITTENING if we need to, but for now just
* use BTRFS_NESTING_NEW_ROOT.
*/
- right = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, 0,
- l->start, 0, num_doubles ?
- BTRFS_NESTING_NEW_ROOT :
- BTRFS_NESTING_SPLIT);
+ right = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, 0, l->start, 0,
+ num_doubles ? BTRFS_NESTING_NEW_ROOT :
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(right))
return PTR_ERR(right);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 443c348bc6f3..14b9fdc8aaa9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -254,8 +254,11 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
}
/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
+ * To be called after doing the chunk btree updates right after allocating a new
+ * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
+ * chunk after all chunk btree updates and after finishing the second phase of
+ * chunk allocation (btrfs_create_pending_block_groups()) in case some block
+ * group had its chunk item insertion delayed to the second phase.
*/
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
@@ -264,8 +267,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
if (!trans->chunk_bytes_reserved)
return;
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
trans->chunk_bytes_reserved = 0;
@@ -696,7 +697,6 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,
h->fs_info = root->fs_info;
h->type = type;
- h->can_flush_pending_bgs = true;
INIT_LIST_HEAD(&h->new_bgs);
smp_mb();
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index a18d67796b54..ba45065f9451 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -132,7 +132,7 @@ struct btrfs_trans_handle {
short aborted;
bool adding_csums;
bool allocating_chunk;
- bool can_flush_pending_bgs;
+ bool removing_chunk;
bool reloc_reserved;
bool in_fsync;
struct btrfs_root *root;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 782e16795bc4..c6c14315b1c9 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1745,19 +1745,14 @@ static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_handle_fs_error(fs_info, ret, "Slot search failed");
goto out;
}
*dev_extent_len = btrfs_dev_extent_length(leaf, extent);
ret = btrfs_del_item(trans, root, path);
- if (ret) {
- btrfs_handle_fs_error(fs_info, ret,
- "Failed to remove dev extent item");
- } else {
+ if (ret == 0)
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
- }
out:
btrfs_free_path(path);
return ret;
@@ -2942,7 +2937,7 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
u32 cur;
struct btrfs_key key;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
array_size = btrfs_super_sys_array_size(super_copy);
ptr = super_copy->sys_chunk_array;
@@ -2972,7 +2967,6 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
cur += len;
}
}
- mutex_unlock(&fs_info->chunk_mutex);
return ret;
}
@@ -3012,6 +3006,29 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,
return em;
}
+static int remove_chunk_item(struct btrfs_trans_handle *trans,
+ struct map_lookup *map, u64 chunk_offset)
+{
+ int i;
+
+ /*
+ * Removing chunk items and updating the device items in the chunks btree
+ * requires holding the chunk_mutex.
+ * See the comment at btrfs_chunk_alloc() for the details.
+ */
+ lockdep_assert_held(&trans->fs_info->chunk_mutex);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ int ret;
+
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ if (ret)
+ return ret;
+ }
+
+ return btrfs_free_chunk(trans, chunk_offset);
+}
+
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3032,14 +3049,16 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(em);
}
map = em->map_lookup;
- mutex_lock(&fs_info->chunk_mutex);
- check_system_chunk(trans, map->type);
- mutex_unlock(&fs_info->chunk_mutex);
/*
- * Take the device list mutex to prevent races with the final phase of
- * a device replace operation that replaces the device object associated
- * with map stripes (dev-replace.c:btrfs_dev_replace_finishing()).
+ * First delete the device extent items from the devices btree.
+ * We take the device_list_mutex to avoid racing with the finishing phase
+ * of a device replace operation. See the comment below before acquiring
+ * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
+ * because that can result in a deadlock when deleting the device extent
+ * items from the devices btree - COWing an extent buffer from the btree
+ * may result in allocating a new metadata chunk, which would attempt to
+ * lock again fs_info->chunk_mutex.
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
@@ -3061,18 +3080,73 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
btrfs_clear_space_info_full(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
}
+ }
+ mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_update_device(trans, device);
+ /*
+ * We acquire fs_info->chunk_mutex for 2 reasons:
+ *
+ * 1) Just like with the first phase of the chunk allocation, we must
+ * reserve system space, do all chunk btree updates and deletions, and
+ * update the system chunk array in the superblock while holding this
+ * mutex. This is for similar reasons as explained on the comment at
+ * the top of btrfs_chunk_alloc();
+ *
+ * 2) Prevent races with the final phase of a device replace operation
+ * that replaces the device object associated with the map's stripes,
+ * because the device object's id can change at any time during that
+ * final phase of the device replace operation
+ * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
+ * replaced device and then see it with an ID of
+ * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
+ * the device item, which does not exists on the chunk btree.
+ * The finishing phase of device replace acquires both the
+ * device_list_mutex and the chunk_mutex, in that order, so we are
+ * safe by just acquiring the chunk_mutex.
+ */
+ trans->removing_chunk = true;
+ mutex_lock(&fs_info->chunk_mutex);
+
+ check_system_chunk(trans, map->type);
+
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ /*
+ * Normally we should not get -ENOSPC since we reserved space before
+ * through the call to check_system_chunk().
+ *
+ * Despite our system space_info having enough free space, we may not
+ * be able to allocate extents from its block groups, because all have
+ * an incompatible profile, which will force us to allocate a new system
+ * block group with the right profile, or right after we called
+ * check_system_space() above, a scrub turned the only system block group
+ * with enough free space into RO mode.
+ * This is explained with more detail at do_chunk_alloc().
+ *
+ * So if we get -ENOSPC, allocate a new system chunk and retry once.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
goto out;
}
- }
- mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_free_chunk(trans, chunk_offset);
- if (ret) {
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -3087,6 +3161,15 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
}
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+
+ /*
+ * We are done with chunk btree updates and deletions, so release the
+ * system space we previously reserved (with check_system_chunk()).
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+
ret = btrfs_remove_block_group(trans, chunk_offset, em);
if (ret) {
btrfs_abort_transaction(trans, ret);
@@ -3094,6 +3177,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
out:
+ if (trans->removing_chunk) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ }
/* once for us */
free_extent_map(em);
return ret;
@@ -4860,13 +4947,12 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
u32 array_size;
u8 *ptr;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
+
array_size = btrfs_super_sys_array_size(super_copy);
if (array_size + item_size + sizeof(disk_key)
- > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
- mutex_unlock(&fs_info->chunk_mutex);
+ > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
return -EFBIG;
- }
ptr = super_copy->sys_chunk_array + array_size;
btrfs_cpu_key_to_disk(&disk_key, key);
@@ -4875,7 +4961,6 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
memcpy(ptr, chunk, item_size);
item_size += sizeof(disk_key);
btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
- mutex_unlock(&fs_info->chunk_mutex);
return 0;
}
@@ -5225,13 +5310,14 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
}
}
-static int create_chunk(struct btrfs_trans_handle *trans,
+static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
{
struct btrfs_fs_info *info = trans->fs_info;
struct map_lookup *map = NULL;
struct extent_map_tree *em_tree;
+ struct btrfs_block_group *block_group;
struct extent_map *em;
u64 start = ctl->start;
u64 type = ctl->type;
@@ -5241,7 +5327,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
map = kmalloc(map_lookup_size(ctl->num_stripes), GFP_NOFS);
if (!map)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
map->num_stripes = ctl->num_stripes;
for (i = 0; i < ctl->ndevs; ++i) {
@@ -5263,7 +5349,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
em = alloc_extent_map();
if (!em) {
kfree(map);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
set_bit(EXTENT_FLAG_FS_MAPPING, &em->flags);
em->map_lookup = map;
@@ -5279,12 +5365,12 @@ static int create_chunk(struct btrfs_trans_handle *trans,
if (ret) {
write_unlock(&em_tree->lock);
free_extent_map(em);
- return ret;
+ return ERR_PTR(ret);
}
write_unlock(&em_tree->lock);
- ret = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
- if (ret)
+ block_group = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
+ if (IS_ERR(block_group))
goto error_del_extent;
for (i = 0; i < map->num_stripes; i++) {
@@ -5304,7 +5390,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
check_raid56_incompat_flag(info, type);
check_raid1c34_incompat_flag(info, type);
- return 0;
+ return block_group;
error_del_extent:
write_lock(&em_tree->lock);
@@ -5316,34 +5402,36 @@ static int create_chunk(struct btrfs_trans_handle *trans,
/* One for the tree reference */
free_extent_map(em);
- return ret;
+ return block_group;
}
-int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
+struct btrfs_block_group *btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ u64 type)
{
struct btrfs_fs_info *info = trans->fs_info;
struct btrfs_fs_devices *fs_devices = info->fs_devices;
struct btrfs_device_info *devices_info = NULL;
struct alloc_chunk_ctl ctl;
+ struct btrfs_block_group *block_group;
int ret;
lockdep_assert_held(&info->chunk_mutex);
if (!alloc_profile_is_valid(type, 0)) {
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (list_empty(&fs_devices->alloc_list)) {
if (btrfs_test_opt(info, ENOSPC_DEBUG))
btrfs_debug(info, "%s: no writable device", __func__);
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
}
if (!(type & BTRFS_BLOCK_GROUP_TYPE_MASK)) {
btrfs_err(info, "invalid chunk type 0x%llx requested", type);
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
ctl.start = find_next_chunk(info);
@@ -5353,46 +5441,43 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
devices_info = kcalloc(fs_devices->rw_devices, sizeof(*devices_info),
GFP_NOFS);
if (!devices_info)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = gather_device_info(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
ret = decide_stripe_size(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
- ret = create_chunk(trans, &ctl, devices_info);
+ block_group = create_chunk(trans, &ctl, devices_info);
out:
kfree(devices_info);
- return ret;
+ return block_group;
}
/*
- * Chunk allocation falls into two parts. The first part does work
]
]
|
| 214/2 |
2026/08/08 04:04 |
llm |
debugger |
0m
Model:
Tokens:
input: 57416
output: 1353
thoughts: 3525
Thoughts: **My Analysis of the Btrfs Chunk Removal Crash**
Okay, here's my breakdown of this problem, considering I'm quite familiar with these internals. We're dealing with a nasty crash during chunk removal in Btrfs, specifically related to `-ENOSPC` errors and transaction aborts. The core issue is that `btrfs_remove_chunk` is failing when it's supposed to handle a situation where the system is running out of space.
First, I realized that the original code's approach of setting `trans->removing_chunk = true` *before* `btrfs_remove_dev_extents` was fundamentally flawed. `btrfs_remove_dev_extents` can trigger nested chunk allocations. If we had `trans->removing_chunk` set at the wrong time, then `btrfs_chunk_alloc` returns `-ENOSPC` which, coupled with the need to allocate metadata, meant a guaranteed abort. The expert's original assertion was correct: we *must* be careful about setting this flag to prevent spurious `-ENOSPC` and transaction aborts, as the expert said, "Spurious `-ENOSPC` and Transaction Aborts via `trans->removing_chunk`".
My corrected code first checks for system space *before* touching the device tree at all, using `check_system_chunk`, then it frees a chunk of space. If `check_system_chunk` ensures that a chunk is available (and it might *need* to allocate a new one), we *then* proceed to remove device extents. Crucially, I set `trans->removing_chunk = false` *before* calling `btrfs_remove_dev_extents`. This is key, because `btrfs_remove_dev_extents` itself is not the problem, it is the state of `trans->removing_chunk` which can trigger it.
This early check is followed by a second `check_system_chunk`, *after* the extents are removed, which properly reserves the system chunk space for the upcoming `remove_chunk_item` call which modifies the chunk btree and requires space to do so. Critically, I was able to verify that the check of `trans->chunk_bytes_reserved` after the first call to `check_system_chunk` correctly indicates *if* we failed to reserve enough space, thus allowing us to return an early `-ENOSPC` and avoid an abort. And setting `trans->removing_chunk = true` during `remove_chunk_item` is essential to prevent deadlock within `remove_chunk_item`.
The original problem boils down to not having the correct system space reserved at the correct time during chunk removal. We must make sure that we have enough space *before* the operation modifies the device tree. But the trick is that the metadata reservation, if not done correctly, can leak. So we release the reserve to avoid this, but make sure that we make a new one before we update the tree with `remove_chunk_item`.
This solution elegantly avoids all the pitfalls. We're now correctly handling the situation:
* We're not locking ourselves into premature reservations.
* We're preventing the unnecessary return of `-ENOSPC` when there's actually a reservation.
* We're keeping the necessary chunk metadata locked for the minimum amount of time required.
* We make sure we can exit gracefully.
This approach is solid. The race conditions are averted, the error paths are clean, and we're guaranteeing that if the system is genuinely out of space, the operation will fail gracefully, instead of crashing the filesystem.
|
| 215/1 |
2026/08/08 04:04 |
agent |
history-explorer |
4m
Model:
Instruction:
You are an experienced Linux kernel developer researching prior art for fixing a kernel bug.
You are given a bug explanation. This explanation details the root cause of the bug resulting
from debugging, but does not provide the final fix strategy. Your goal is to explore how
similar bugs were fixed in the past in the same subsystem or files.
CRITICAL: Do NOT attempt to debug the issue further or write a patch for it yourself.
Your ONLY objective is to research and provide the necessary historical context.
Use the git-log tool with the Since parameter set to "3 years" to focus on recent history.
Search for commits that address issues with similar root causes (e.g. similar missing locks,
incorrect refcounting, or similar error path bugs) in the affected files.
Your final reply must summarize your findings: what idioms, locking rules, or common patterns
should be followed when writing a fix for this bug based on how previous similar bugs were addressed.
If you find no relevant past fixes, clearly state that.
Don't make assumptions about the kernel source code (it may be different from what you assume it is).
Extensively use the provided code access tools (codesearch-*, git-*, grepper, etc)
to examine the actual source code, and confirm any assumptions.
Prefer calling several tools at the same time to save round-trips.
Prompt:
The crash is:
BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
The explanation of the root cause of the bug is:
An analysis of the crash reveals that it is caused by a transaction abort (`-ENOSPC`) in `btrfs_remove_chunk` during a chunk relocation/balance operation. The abort happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
### Root Cause
1. **Order of Operations**: During chunk removal, `btrfs_remove_chunk` first removes the device extents from the device tree by calling `btrfs_remove_dev_extents`.
2. **System Space Reservation**: It then calls `check_system_chunk` to reserve system space for removing the chunk item from the chunk tree. If the system space is full, `check_system_chunk` attempts to allocate a new system chunk.
3. **Commit Root Search**: To allocate a new chunk, `find_free_dev_extent` is called. However, `find_free_dev_extent` searches the *commit root* of the device tree to prevent reusing space freed in the current transaction (which could lead to corruption if the transaction aborts).
4. **Invisible Freed Space**: Because the device extents were just freed in the *current* transaction by `btrfs_remove_dev_extents`, `find_free_dev_extent` does not see this freed space. If the device is otherwise full, it returns `-ENOSPC`.
5. **Ignored Failure**: `check_system_chunk` ignores the failure to allocate a system chunk and does not reserve any space in `trans->chunk_bytes_reserved`.
6. **Transaction Abort**: `remove_chunk_item` is then called, which fails with `-ENOSPC` because it needs to COW a node in the chunk tree but no system space was reserved. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully (returning `-ENOSPC` would leave the filesystem in an inconsistent state with the chunk item present but device extents missing). Thus, it is forced to abort the transaction.
### The Fix
We must ensure that we have enough system space *before* we modify the device tree. However, we cannot simply move the `check_system_chunk` call before `btrfs_remove_dev_extents` and keep the reservation, because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations (`btrfs_trans_release_chunk_metadata`).
Instead, we can call `check_system_chunk` early to *ensure* the system `space_info` has enough free space (allocating a system chunk if necessary), and then immediately release the `chunk_block_rsv` reservation. If it fails to allocate a system chunk, we can fail gracefully by returning `-ENOSPC` before any trees are modified. Then, after `btrfs_remove_dev_extents`, we call `check_system_chunk` again to make the actual reservation for `remove_chunk_item`.
Here is the proposed fix for `fs/btrfs/volumes.c` in `btrfs_remove_chunk`:
```c
map = em->map_lookup;
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
/*
* Take the device list mutex to prevent races with the final phase of
* a device replace operation...
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
// ... existing btrfs_remove_dev_extents logic ...
}
mutex_unlock(&fs_devices->device_list_mutex);
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
### Why this works:
1. **Graceful Failure**: By checking `trans->chunk_bytes_reserved`, we can detect if `check_system_chunk` failed to allocate a system chunk. If it failed, we return `-ENOSPC` *before* any trees are modified, allowing the balance operation to fail cleanly instead of crashing the kernel.
2. **No Metadata Leaks**: We immediately release the early reservation using `btrfs_trans_release_chunk_metadata(trans)`.
3. **Safe from Nested Allocations**: Because we release the reservation, we don't care if `btrfs_remove_dev_extents` triggers a nested chunk allocation that clears `trans->chunk_bytes_reserved`.
4. **Guaranteed Space**: The early `check_system_chunk` ensures the system `space_info` has enough free space (by allocating a 32MB system chunk if needed). When we call `check_system_chunk` the second time, it will see the free space in the `space_info` and successfully reserve the few KB needed for `remove_chunk_item`.
Reply: Based on a thorough review of the BTRFS codebase and its commit history over the past several years, here is a summary of the findings regarding the idioms, locking rules, and common patterns relevant to this bug and the proposed fix:
### 1. The Root Cause Context (Commit Root Searches)
The root cause you identified—that `find_free_dev_extent` searches the commit root and therefore cannot see space freed in the current transaction—is a known and intentional BTRFS design choice. This prevents the filesystem from reusing space freed in an uncommitted transaction, which could lead to severe corruption if the transaction were to abort.
### 2. `check_system_chunk` Error Handling
The `check_system_chunk` function (and its underlying `reserve_chunk_space`) is designed to return `void` and intentionally ignores failures to allocate a system chunk. The historical assumption (documented in `reserve_chunk_space`) is that if it fails, the caller might not actually need the space (e.g., if the required nodes were already COWed). If the space is truly needed, the subsequent tree modification (like `remove_chunk_item`) will fail with `-ENOSPC`.
`btrfs_remove_chunk` already attempts to handle this `-ENOSPC` gracefully by falling back to `btrfs_create_chunk` (introduced to handle cases where system space is fragmented or profiles are incompatible). However, as you noted, this fallback also fails because `btrfs_remove_dev_extents` has already modified the device tree in the current transaction.
### 3. The "Early Reserve and Release" Pattern
Reserving metadata space early (before making irreversible tree changes) to ensure it is available, and then releasing it, is a known pattern in BTRFS. For example, in `fs/btrfs/volumes.c` (`btrfs_finish_sprout` and `btrfs_rm_dev_item`), the code calls `btrfs_reserve_chunk_metadata()`, performs a tree search/modification, and then calls `btrfs_trans_release_chunk_metadata()`.
However, the proposed fix's approach of reserving, **immediately releasing**, and then re-reserving later is unique. It is a necessary adaptation here because `btrfs_remove_dev_extents` can trigger nested allocations that unconditionally clear the reservation (via `btrfs_trans_release_chunk_metadata`).
### 4. Checking `trans->chunk_bytes_reserved` (Novel Idiom)
The proposed method of detecting whether `check_system_chunk` failed by comparing `trans->chunk_bytes_reserved` before and after the call is a **novel, non-standard idiom**.
A search through the BTRFS codebase reveals no prior art that checks `chunk_bytes_reserved` in this manner to infer the success of `check_system_chunk`. In BTRFS, space reservation failures are almost universally handled by checking a returned integer error code (e.g., `ret = btrfs_chunk_alloc(...)`).
### Conclusion & Recommendations
**No exact past fixes were found that use this specific `chunk_bytes_reserved` check pattern.**
While the proposed fix correctly identifies the transactional deadlock and successfully works around it by failing gracefully before modifying the device tree, the way it detects the failure is non-idiomatic for BTRFS.
If you are to submit a fix based on historical BTRFS patterns, you have two main paths:
1. **The Pragmatic Workaround (Your Proposal):** Proceed with the `chunk_bytes_reserved` check, but heavily comment it. Acknowledge that it is a hack to work around the fact that `check_system_chunk` returns `void` and ignores errors, and that it is necessary to prevent a transaction abort.
2. **The Idiomatic Approach:** A more standard (though slightly more invasive) approach would be to introduce a variant of `check_system_chunk` or `reserve_chunk_space` that actually returns an `int` error code if it fails to allocate a system chunk, allowing `btrfs_remove_chunk` to check the return value directly rather than inferring it from transaction state variables.
|
| 216/2 |
2026/08/08 04:04 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 6948
output: 50
thoughts: 317
Thoughts: **Kernel Bug Research in BTRFS - My Approach**
Okay, so we're diving into this BTRFS kernel bug in `btrfs_remove_chunk`. My immediate concern is the root cause: an ENOSPC transaction abort during chunk removal, specifically within `btrfs_remove_chunk` in `fs/btrfs/volumes.c`. It looks like the process is a bit backwards. `btrfs_remove_chunk` first tweaks the device tree with `btrfs_remove_dev_extents`, and *then* tries to lock in the system space for the chunk item removal via `check_system_chunk`. The real problem is, if that `check_system_chunk` fails because `find_free_dev_extent` is looking at the commit root and can't see the recently freed space, the whole transaction gets aborted. Definitely a flawed sequence.
My current thought is to ensure we have the needed space *before* making any changes. The fix I'm mulling over is an early `check_system_chunk`. Reserve the needed space up-front, release that reservation (if all goes well) and *then* continue the process. This should hopefully head off the aborts.
Now, I need to understand what's been tried, what's similar, and what else might apply to this corner case. My plan is to start with `git-log`. I'm going to set a timeframe—let's say "Since=3 years"—to narrow the scope. I want to ferret out commits that address issues with a similar feel. Maybe we have the same kind of missing locks, incorrect refcounting, or errors in the error paths.
The crucial targets are commits touching `fs/btrfs/volumes.c` or related files. I should also search specifically for commits dealing with `btrfs_remove_chunk`, `check_system_chunk`, `ENOSPC` errors, transaction aborts, or even early space reservation in BTRFS. A targeted `git-log` search using `SymbolName="btrfs_remove_chunk"` would be helpful as well. Let's see what the history tells us, and if this space reservation issue has been addressed before.
|
| 217/2 |
2026/08/08 04:04 |
tool |
git-log |
0m
Args: map[Count:20 Since:3 years SourcePath:fs/btrfs/volumes.c SymbolName:btrfs_remove_chunk]
Results: map[Output:c84053d9f7f7 btrfs: update per-profile available estimation
6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
979e1dc3d69e btrfs: handle deletions from remapped block group
a929904cf73b btrfs: add unlikely annotations to branches leading to transaction abort
4d5a047e0736 btrfs: add space_info parameter for block group creation
7dc66abb5a47 btrfs: use a dedicated data structure for chunk maps
]
|
| 218/2 |
2026/08/08 04:04 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 7956
output: 48
thoughts: 168
Thoughts: **Analyzing Btrfs Error Handling and Locking**
Okay, so I need to get to the core of this bug fix, keeping in mind the lessons from similar past incidents. First, I need to check the commits they've identified, `6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()` and `979e1dc3d69e btrfs: handle deletions from remapped block group`. That's a good starting point to understand the immediate context and potentially identify any shared idioms or locking patterns relevant to error handling during block group operations, especially removals and remapping.
Wait a second, I need to be sure to summarize my findings based on past examples... So, looking at these commits will help with that...
Now, to broaden my search for potential solutions or established practices, I'll need to look at how similar issues were addressed in the past. To do that, a quick search for `check_system_chunk` within `fs/btrfs/volumes.c` or `fs/btrfs/block-group.c` is a good idea. That's where I would expect to find the most relevant code relating to block group management, particularly with respect to error checks. This will help me see if the system performs any specific checks when dealing with chunk operations, and how to go about handling errors. That should give me a solid picture of the existing patterns in the code. I need to be sure to document those findings in my summary. If I find nothing relevant, I need to state that too.
|
| 219/2 |
2026/08/08 04:04 |
tool |
git-log |
0m
Error:
Args: map[Count:20 Since:3 years SourcePath:fs/btrfs/volumes.c SymbolName:check_system_chunk]
Results:
|
| 220/2 |
2026/08/08 04:04 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 8551
output: 19
thoughts: 47
Thoughts: Okay, here's my thought process on that:
**Tracking Down the Definition of `check_system_chunk`**
Right, so I need to figure out where `check_system_chunk` is actually *defined*. I have a hunch that it might be tucked away in `fs/btrfs/block-group.c`, or possibly that it has been renamed or moved since I last worked with it. That's a common issue with these complex filesystems, things get refactored all the time. The first step is clear: I need to pinpoint the exact location of that definition. Let's get to it.
|
| 221/2 |
2026/08/08 04:04 |
tool |
grepper |
0m
Args: map[Expression:check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=1383=struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
--
fs/btrfs/block-group.c-1415- * more device items and remove one chunk item), but this is done at
fs/btrfs/block-group.c:1416: * btrfs_remove_chunk() through a call to check_system_chunk().
fs/btrfs/block-group.c-1417- */
--
fs/btrfs/block-group.c=3125=int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
--
fs/btrfs/block-group.c-3233- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/block-group.c:3234: check_system_chunk(trans, alloc_flags);
fs/btrfs/block-group.c-3235- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4140- */
fs/btrfs/block-group.c:4141: check_system_chunk(trans, flags);
fs/btrfs/block-group.c-4142-
--
fs/btrfs/block-group.c-4165- * single device, we are forced to allocate a new system chunk with a
fs/btrfs/block-group.c:4166: * SINGLE profile. Making check_system_chunk() iterate over all system
fs/btrfs/block-group.c-4167- * block groups and check if they have a usable profile and enough space
--
fs/btrfs/block-group.c-4173- * 2) We had enough free space info the system space_info, and one suitable
fs/btrfs/block-group.c:4174: * block group to allocate from when we called check_system_chunk()
fs/btrfs/block-group.c-4175- * above. However right after we called it, the only system block group
--
fs/btrfs/block-group.c-4181- * 3) We had one system block group with enough free space when we called
fs/btrfs/block-group.c:4182: * check_system_chunk(), but after that, right before we tried to
fs/btrfs/block-group.c-4183- * allocate the last extent buffer we needed, a discard operation came
--
fs/btrfs/block-group.c-4309- * needs to update the chunk btree (the only btree that uses system chunks), must
fs/btrfs/block-group.c:4310: * preallocate chunk space by calling either check_system_chunk() or
fs/btrfs/block-group.c-4311- * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
--
fs/btrfs/block-group.c-4316- *
fs/btrfs/block-group.c:4317: * The reservation of system space, done through check_system_chunk(), as well
fs/btrfs/block-group.c-4318- * as all the updates and insertions into the chunk btree must be done while
--
fs/btrfs/block-group.c=4341=int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4375- * before updating the chunk btree, by calling either
fs/btrfs/block-group.c:4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
fs/btrfs/block-group.c-4377- * It's possible that after a task reserves the space, it still ends up
--
fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3493-
fs/btrfs/volumes.c:3494: check_system_chunk(trans, map->type);
fs/btrfs/volumes.c-3495-
--
fs/btrfs/volumes.c-3498- * Normally we should not get -ENOSPC since we reserved space before
fs/btrfs/volumes.c:3499: * through the call to check_system_chunk().
fs/btrfs/volumes.c-3500- *
--
fs/btrfs/volumes.c-3561- * We are done with chunk btree updates and deletions, so release the
fs/btrfs/volumes.c:3562: * system space we previously reserved (with check_system_chunk()).
fs/btrfs/volumes.c-3563- */
]
|
| 222/2 |
2026/08/08 04:04 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 10338
output: 50
thoughts: 49
Thoughts: Here's my thought process, summarized as you requested:
**Investigating `check_system_chunk` and `btrfs_remove_chunk`**
Okay, so I need to find the definition of `check_system_chunk`. That's easy enough, it looks like it's located in `fs/btrfs/block-group.c`. Got it.
Now, I'm thinking, let's trace the development of this function and its relationship with `btrfs_remove_chunk`. I'm curious about how these two functions have evolved together over time. What changes have been made to them, and were the made in the same commit, or in coordination? I'll probably look for change logs or commit histories to understand the dependency or linkage between these two functions and how they've changed. That could give some insight into how they work together, any potential interactions, and possible implications for the system.
|
| 223/2 |
2026/08/08 04:05 |
tool |
git-log |
0m
Args: map[Count:20 Since:3 years SourcePath:fs/btrfs/block-group.c SymbolName:check_system_chunk]
Results:
|
| 224/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 10649
output: 34
thoughts: 63
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Troubleshooting Potential Mismatch in System Chunk Detection**
Alright, so I'm thinking, that `check_system_chunk` call, it's possible that the `-L` option isn't picking it up because the function definition doesn't fit the expected pattern. It could be some subtle difference in how the compiler handles it, an inlined function, a macro, or something non-standard. The limitations of `-L` become apparent when dealing with more complex codebases or unusual function definitions, and I've run into this before.
So, instead of banging my head against the wall with `-L`, I'm going to switch gears. My instinct is to reach for `git-log`, leveraging its flexibility. Using `git-log` combined with `MessageRegexps` is the best way to get around the limits of other tools. I'll search the commit messages directly for instances of "check_system_chunk" and "btrfs_remove_chunk". It's a more robust approach, especially considering the likely complexity of chunk management and potential for variations in function naming across different Btrfs versions. This provides greater control and allows us to get the right calls.
|
| 225/2 |
2026/08/08 04:05 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[check_system_chunk] Since:3 years]
Results: map[Output:0d85f5c2dd91 btrfs: fix assertion failure when splitting ordered extent after transaction abort
]
|
| 226/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 11053
output: 36
thoughts: 19
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Investigating Btrfs Chunk Removal in Commit History**
Right, so we're looking into `btrfs_remove_chunk`. That's a key function, and understanding its evolution is crucial. Beyond just finding code changes, I need to dig into the **context** of these changes. Commit messages are going to be critical here. They'll tell me *why* the changes were made – bug fixes, performance improvements, feature implementations, refactoring, or maybe even just clarifying documentation. I'm going to look specifically for those commit messages to get the story behind how chunk removal has changed over time. That's where the real insights are. I need to be able to understand the intent behind the code.
|
| 227/2 |
2026/08/08 04:05 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[btrfs_remove_chunk] Since:3 years]
Results: map[Output:6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
]
|
| 228/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 11325
output: 27
thoughts: 19
Thoughts: Okay, let's dive into commit `6f926597f983`.
|
| 229/2 |
2026/08/08 04:05 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 6f926597f9837577f5ada47eaa764fea4a2ca9a3
Author: Filipe Manana <fdmanana@suse.com>
Date: Fri Jan 23 09:49:57 2026 +0000
btrfs: abort transaction on error in btrfs_remove_block_group()
When btrfs_remove_block_group() fails we abort the transaction in its
single caller (btrfs_remove_chunk()). This makes it harder to find out
where exactly the failure happened, as several steps inside
btrfs_remove_block_group() can fail.
So make btrfs_remove_block_group() abort the transaction whenever an
error happens, instead of aborting in its caller.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 3a0521236ecd..7b723571501e 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -1108,8 +1108,10 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
bool remove_rsv = false;
block_group = btrfs_lookup_block_group(fs_info, map->start);
- if (!block_group)
+ if (unlikely(!block_group)) {
+ btrfs_abort_transaction(trans, -ENOENT);
return -ENOENT;
+ }
BUG_ON(!block_group->ro && !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED));
@@ -1143,8 +1145,9 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
btrfs_clear_data_reloc_bg(block_group);
path = btrfs_alloc_path();
- if (!path) {
+ if (unlikely(!path)) {
ret = -ENOMEM;
+ btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -1180,8 +1183,10 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
mutex_unlock(&trans->transaction->cache_write_mutex);
ret = btrfs_remove_free_space_inode(trans, inode, block_group);
- if (ret)
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
goto out;
+ }
write_lock(&fs_info->block_group_cache_lock);
rb_erase_cached(&block_group->cache_node,
@@ -1268,13 +1273,17 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
*/
if (!(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED)) {
ret = btrfs_remove_block_group_free_space(trans, block_group);
- if (ret)
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
goto out;
+ }
}
ret = remove_block_group_item(trans, path, block_group);
- if (ret < 0)
+ if (unlikely(ret < 0)) {
+ btrfs_abort_transaction(trans, ret);
goto out;
+ }
spin_lock(&block_group->lock);
/*
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index cff2412bc879..d33780082b8d 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3384,11 +3384,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
*/
btrfs_trans_release_chunk_metadata(trans);
+ /* On error, btrfs_remove_block_group() aborts the transaction. */
ret = btrfs_remove_block_group(trans, map);
- if (unlikely(ret)) {
- btrfs_abort_transaction(trans, ret);
- goto out;
- }
+ if (unlikely(ret))
+ ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
out:
if (trans->removing_chunk) {
]
|
| 230/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 12766
output: 19
thoughts: 53
Thoughts: **Refactoring `btrfs_remove_block_group`**
Okay, so this commit's primary function is moving the abort call within `btrfs_remove_block_group`. I need to fully understand the impact of this change. My immediate focus is to see how `check_system_chunk` is utilized within `fs/btrfs/block-group.c`. Understanding its call sites will give me a good sense of the control flow impacted by this adjustment and if there are any subtle interactions I need to be aware of. Let's see how this affects the overall integrity of the filesystem's block group operations.
|
| 231/2 |
2026/08/08 04:05 |
tool |
grepper |
0m
Args: map[Expression:check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=1383=struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
--
fs/btrfs/block-group.c-1415- * more device items and remove one chunk item), but this is done at
fs/btrfs/block-group.c:1416: * btrfs_remove_chunk() through a call to check_system_chunk().
fs/btrfs/block-group.c-1417- */
--
fs/btrfs/block-group.c=3125=int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
--
fs/btrfs/block-group.c-3233- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/block-group.c:3234: check_system_chunk(trans, alloc_flags);
fs/btrfs/block-group.c-3235- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4140- */
fs/btrfs/block-group.c:4141: check_system_chunk(trans, flags);
fs/btrfs/block-group.c-4142-
--
fs/btrfs/block-group.c-4165- * single device, we are forced to allocate a new system chunk with a
fs/btrfs/block-group.c:4166: * SINGLE profile. Making check_system_chunk() iterate over all system
fs/btrfs/block-group.c-4167- * block groups and check if they have a usable profile and enough space
--
fs/btrfs/block-group.c-4173- * 2) We had enough free space info the system space_info, and one suitable
fs/btrfs/block-group.c:4174: * block group to allocate from when we called check_system_chunk()
fs/btrfs/block-group.c-4175- * above. However right after we called it, the only system block group
--
fs/btrfs/block-group.c-4181- * 3) We had one system block group with enough free space when we called
fs/btrfs/block-group.c:4182: * check_system_chunk(), but after that, right before we tried to
fs/btrfs/block-group.c-4183- * allocate the last extent buffer we needed, a discard operation came
--
fs/btrfs/block-group.c-4309- * needs to update the chunk btree (the only btree that uses system chunks), must
fs/btrfs/block-group.c:4310: * preallocate chunk space by calling either check_system_chunk() or
fs/btrfs/block-group.c-4311- * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
--
fs/btrfs/block-group.c-4316- *
fs/btrfs/block-group.c:4317: * The reservation of system space, done through check_system_chunk(), as well
fs/btrfs/block-group.c-4318- * as all the updates and insertions into the chunk btree must be done while
--
fs/btrfs/block-group.c=4341=int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4375- * before updating the chunk btree, by calling either
fs/btrfs/block-group.c:4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
fs/btrfs/block-group.c-4377- * It's possible that after a task reserves the space, it still ends up
--
fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3493-
fs/btrfs/volumes.c:3494: check_system_chunk(trans, map->type);
fs/btrfs/volumes.c-3495-
--
fs/btrfs/volumes.c-3498- * Normally we should not get -ENOSPC since we reserved space before
fs/btrfs/volumes.c:3499: * through the call to check_system_chunk().
fs/btrfs/volumes.c-3500- *
--
fs/btrfs/volumes.c-3561- * We are done with chunk btree updates and deletions, so release the
fs/btrfs/volumes.c:3562: * system space we previously reserved (with check_system_chunk()).
fs/btrfs/volumes.c-3563- */
]
|
| 232/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 14560
output: 40
thoughts: 243
Thoughts: **Analyzing a Btrfs Chunk Reservation Fix**
Okay, so I'm looking at this code snippet, and my initial thought is, "Is this pattern of chunk reservation, check, and potentially release common in this filesystem?". We're locking a mutex, checking the reserved chunk bytes, running this `check_system_chunk` function, and then, crucially, *conditionally* releasing the reservation if it didn't change, which indicates a problem with space, returning -ENOSPC. Afterward we *unconditionally* release the metadata.
It seems a bit delicate, reserving then *potentially* releasing the chunk metadata. I'm wondering about the surrounding context. Specifically, I need to understand if the `check_system_chunk` function can modify `trans->chunk_bytes_reserved` in-place, which is what's implied here. It's a key detail to confirm the correctness. This conditional release with that specific condition implies this. It also implies that whatever functionality within the checked function *must* be transactional.
To get a clearer picture of how this whole chunk reservation and checking are used, and how `check_system_chunk` is typically employed, I think it's worth cross-referencing this code with another similar function. I think a good place to look would be `btrfs_inc_block_group_ro` in `fs/btrfs/block-group.c`. Seeing how `check_system_chunk` is used there might give me more insight into this design pattern and whether this conditional release and the checks for insufficient space are expected behavior. I need to fully understand if this is a standard transactional pattern, or something unique to this case.
|
| 233/2 |
2026/08/08 04:05 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:3220 LineCount:50]
Results: map[Contents:3220: * We have allocated a new chunk. We also need to activate that chunk to
3221: * grant metadata tickets for zoned filesystem.
3222: */
3223: ret = btrfs_zoned_activate_one_bg(space_info, true);
3224: if (ret < 0)
3225: goto out;
3226:
3227: ret = inc_block_group_ro(cache, false);
3228: if (ret == -ETXTBSY)
3229: goto unlock_out;
3230: out:
3231: if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
3232: alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
3233: mutex_lock(&fs_info->chunk_mutex);
3234: check_system_chunk(trans, alloc_flags);
3235: mutex_unlock(&fs_info->chunk_mutex);
3236: }
3237: unlock_out:
3238: mutex_unlock(&fs_info->ro_block_group_mutex);
3239:
3240: btrfs_end_transaction(trans);
3241: return ret;
3242: }
3243:
3244: void btrfs_dec_block_group_ro(struct btrfs_block_group *cache)
3245: {
3246: struct btrfs_space_info *sinfo = cache->space_info;
3247:
3248: BUG_ON(!cache->ro);
3249:
3250: spin_lock(&sinfo->lock);
3251: spin_lock(&cache->lock);
3252: if (!--cache->ro) {
3253: if (btrfs_is_zoned(cache->fs_info)) {
3254: /* Migrate zone_unusable bytes back */
3255: cache->zone_unusable =
3256: (cache->alloc_offset - cache->used - cache->pinned -
3257: cache->reserved) +
3258: (cache->length - cache->zone_capacity);
3259: btrfs_space_info_update_bytes_zone_unusable(sinfo, cache->zone_unusable);
3260: sinfo->bytes_readonly -= cache->zone_unusable;
3261: }
3262: sinfo->bytes_readonly -= btrfs_block_group_available_space(cache);
3263: list_del_init(&cache->ro_list);
3264: }
3265: spin_unlock(&cache->lock);
3266: spin_unlock(&sinfo->lock);
3267: }
3268:
3269: static int update_block_group_item(struct btrfs_trans_handle *trans,
]
|
| 234/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 16013
output: 40
thoughts: 59
Thoughts: Here's my summary, as the thought process unfolds:
**Digging into `btrfs_inc_block_group_ro`**
Okay, so I'm looking at `btrfs_inc_block_group_ro`. It's calling `check_system_chunk`, which, based on the name, seems like it's a critical check. But wait a minute, I'm noticing something that feels... off. It calls `check_system_chunk` and I don't immediately see it releasing any resources or handling an error condition, if one even *exists*! That's potentially a serious oversight, especially in such a crucial piece of the filesystem.
Let's not jump to conclusions. First, let me double-check the definition and behavior of `check_system_chunk`. I need to fully understand its responsibilities to determine if this apparent omission is actually a problem or if `check_system_chunk` handles any error conditions internally that I'm not seeing. Time to jump to its definition and understand what's really going on!
|
| 235/2 |
2026/08/08 04:05 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4574 LineCount:50]
Results: map[Contents:4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
4586:
4587: /*
4588: * Reserve space in the system space, if needed, for doing a modification to the
4589: * chunk btree.
4590: *
4591: * @trans: A transaction handle.
4592: * @is_item_insertion: Indicate if the modification is for inserting a new item
4593: * in the chunk btree or if it's for the deletion or update
4594: * of an existing item.
4595: *
4596: * This is used in a context where we need to update the chunk btree outside
4597: * block group allocation and removal, to avoid a deadlock with a concurrent
4598: * task that is allocating a metadata or data block group and therefore needs to
4599: * update the chunk btree while holding the chunk mutex. After the update to the
4600: * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
4601: *
4602: */
4603: void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
4604: bool is_item_insertion)
4605: {
4606: struct btrfs_fs_info *fs_info = trans->fs_info;
4607: u64 bytes;
4608:
4609: if (is_item_insertion)
4610: bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
4611: else
4612: bytes = btrfs_calc_metadata_size(fs_info, 1);
4613:
4614: mutex_lock(&fs_info->chunk_mutex);
4615: reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
4616: mutex_unlock(&fs_info->chunk_mutex);
4617: }
4618:
4619: void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
4620: {
4621: struct btrfs_block_group *block_group;
4622:
4623: block_group = btrfs_lookup_first_block_group(info, 0);
]
|
| 236/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 17224
output: 40
thoughts: 34
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Diving into System Chunk Allocation**
Right, `check_system_chunk` is a void function. That immediately tells me it's likely modifying something in place rather than returning a new handle or value. And it seems to be calling `reserve_chunk_space`. So, the core of the operation lies within `reserve_chunk_space`. I'll need to understand how *that* function works to fully grasp what's going on with the system chunks and their allocation. Let's see what the implementation of `reserve_chunk_space` does and, more importantly, *how* it's doing it, to gain insight into the overall memory management strategy.
|
| 237/2 |
2026/08/08 04:05 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4491 LineCount:80]
Results: map[Contents:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
4569:
4570: /*
]
|
| 238/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 18718
output: 19
thoughts: 294
Thoughts: **Analyzing a Potential Btrfs Chunk Reservation Bug**
Okay, so I'm looking at this chunk space reservation code in Btrfs, and something feels off. Specifically, it seems like `reserve_chunk_space` is ignoring failures during system chunk creation. The comment is pretty explicit: "Ignore failure to create system chunk. We might end up not needing it..." The code snippet confirms this: `btrfs_create_chunk` is called, and if it fails, the return value `bg` will indicate the error using `PTR_ERR`. Then, crucially, the subsequent `if (!ret)` is skipped. This means that if `btrfs_create_chunk` fails, `trans->chunk_bytes_reserved` isn't incremented.
The suggested fix, which involves capturing a starting value for `trans->chunk_bytes_reserved`, calling `check_system_chunk`, and then comparing to see if it changed, looks like the right approach. Essentially, the code is checking if the reservation *actually* happened. If the value hasn't changed after calling `check_system_chunk`, it releases the mutex, sets the return value to `-ENOSPC`, and exits. This ensures that a failure in chunk creation is properly handled and reflected as an out-of-space error.
I'm wondering if this pattern—checking for a change in a counter or reservation after a potentially failing operation—is used elsewhere in the codebase. It's a useful defensive programming technique. I think a quick search for `chunk_bytes_reserved` within the `fs/btrfs/` directory should reveal some similar patterns, and help me understand how widespread this check is, and whether there are other potential areas of concern.
|
| 239/2 |
2026/08/08 04:05 |
tool |
grepper |
0m
Args: map[Expression:chunk_bytes_reserved]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4565- if (!ret)
fs/btrfs/block-group.c:4566: trans->chunk_bytes_reserved += bytes;
fs/btrfs/block-group.c-4567- }
--
fs/btrfs/transaction.c=255=void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
--
fs/btrfs/transaction.c-258-
fs/btrfs/transaction.c:259: if (!trans->chunk_bytes_reserved)
fs/btrfs/transaction.c-260- return;
--
fs/btrfs/transaction.c-262- btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
fs/btrfs/transaction.c:263: trans->chunk_bytes_reserved, NULL);
fs/btrfs/transaction.c:264: trans->chunk_bytes_reserved = 0;
fs/btrfs/transaction.c-265-}
--
fs/btrfs/transaction.h=139=struct btrfs_trans_handle {
--
fs/btrfs/transaction.h-142- u64 delayed_refs_bytes_reserved;
fs/btrfs/transaction.h:143: u64 chunk_bytes_reserved;
fs/btrfs/transaction.h-144- unsigned long delayed_ref_updates;
]
|
| 240/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 19841
output: 36
thoughts: 161
Thoughts: **Analyzing the Chunk Reservation Vulnerability**
Okay, so it looks like this `chunk_bytes_reserved` isn't being validated elsewhere in the codebase with this specific early check. Hmm, right, the prompt wants a summary of idioms, locking rules, and common patterns for fixing bugs like this, based on past solutions. If there aren't any past solutions, I need to state that clearly.
Let's dive into the BTRFS codebase and see if we can find similar bug fixes. I need to find examples where there's an early check to prevent transaction aborts. Thinking through potential areas, `btrfs_remove_chunk`, `btrfs_remove_block_group`, and `btrfs_chunk_alloc` come to mind as potential candidates. I think I'll search the git logs for `check_system_chunk` again, focusing on the `fs/btrfs/` directory this time. Hopefully, that will give me some insight into how similar problems have been addressed in the past.
|
| 241/2 |
2026/08/08 04:05 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[ENOSPC abort] Since:3 years]
Results: map[Output:2c007acf7b31 scsi: elx: efct: Fix refcount leak in efct_hw_io_abort()
8070a50b8897 xfs: abort mount if xfs_fs_reserve_ag_blocks fails
40f2b11c1b7c btrfs: don't allow log trees to consume global reserve or overcommit metadata
574d93fc62e2 btrfs: be less aggressive with metadata overcommit when we can do full flushing
2672a26a7551 btrfs: use per-profile available space in calc_available_free_space()
52fead5eb8a7 btrfs: introduce the device layout aware per-profile available space
f9a4e3015db1 btrfs: reserve enough transaction items for qgroup ioctls
8ceaad6cd6e7 btrfs: do not ASSERT() when the fs flips RO inside btrfs_repair_io_failure()
b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
f4a9f219411f btrfs: do not delete unused block group if it may be used soon
a7f8de500e28 btrfs: return real error when orphan cleanup fails due to a transaction abort
1b6948acb854 btrfs: don't steal space from global rsv after a transaction abort
92fb94b69c6a btrfs: set cache_block_group_error if we find an error
]
|
| 242/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 20653
output: 48
thoughts: 25
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Investigating Potential Btrfs Corruption/Abortion Scenarios**
Alright, time to get to the heart of this problem. My initial instinct is to zero in on potential file system corruption or abort-related issues within the Btrfs code. Given the symptoms we're seeing, I'm going to start with a focused search. Let's see... I'll specifically look for the error code `ENOSPC` and any instances of the word `abort` within the `fs/btrfs/` directory. That's where the core Btrfs implementation resides, and those two terms together often point to problems where the file system runs out of space, which can lead to unexpected termination. This targeted approach is the best way to determine potential points of failure and areas that warrant a closer inspection. It'll narrow down the haystack, allowing me to assess if the root cause of my issue can be found.
|
| 243/2 |
2026/08/08 04:05 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[ENOSPC abort] PathPrefix:fs/btrfs/ Since:3 years]
Results: map[Output:40f2b11c1b7c btrfs: don't allow log trees to consume global reserve or overcommit metadata
574d93fc62e2 btrfs: be less aggressive with metadata overcommit when we can do full flushing
2672a26a7551 btrfs: use per-profile available space in calc_available_free_space()
52fead5eb8a7 btrfs: introduce the device layout aware per-profile available space
f9a4e3015db1 btrfs: reserve enough transaction items for qgroup ioctls
8ceaad6cd6e7 btrfs: do not ASSERT() when the fs flips RO inside btrfs_repair_io_failure()
b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
f4a9f219411f btrfs: do not delete unused block group if it may be used soon
a7f8de500e28 btrfs: return real error when orphan cleanup fails due to a transaction abort
1b6948acb854 btrfs: don't steal space from global rsv after a transaction abort
92fb94b69c6a btrfs: set cache_block_group_error if we find an error
]
|
| 244/2 |
2026/08/08 04:05 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 21272
output: 26
thoughts: 35
Thoughts: Okay, here's my take on those thoughts, framed as an expert in the field:
**Debugging a Btrfs Chunk Allocation Issue**
Alright, let's dive into this `b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation` commit. Seems like we're dealing with a nasty `EEXIST` error that's triggering an abort during chunk allocation in Btrfs. The clue is in "non-consecutive gaps". This likely means the allocator is hitting a problem when trying to find contiguous space. It's probably related to how the metadata structures are organized and how the allocator scans for available chunks. I need to understand how these "non-consecutive gaps" are getting created in the first place. Is it fragmentation? Is it due to interactions with the device's physical layout and block groups? I have a feeling the logic around how the allocator handles these discontiguous regions needs a close inspection. I'll need to focus on where the allocation routines are called, the checks for space availability, and how the algorithm then translates this to the underlying disk writes. I hope this commit provides a clear path through the problem, or at least a starting point. There's probably going to be a need for some tracing through the code to reproduce the issue.
|
| 245/2 |
2026/08/08 04:06 |
tool |
git-show |
0m
Args:
Results: map[Output:commit b14c5e04bd0f722ed631845599d52d03fcae1bc1
Author: Boris Burkov <boris@bur.io>
Date: Thu Jan 29 16:11:21 2026 -0800
btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
I have been observing a number of systems aborting at
insert_dev_extents() in btrfs_create_pending_block_groups(). The
following is a sample stack trace of such an abort coming from forced
chunk allocation (typically behind CONFIG_BTRFS_EXPERIMENTAL) but this
can theoretically happen to any DUP chunk allocation.
[81.801] ------------[ cut here ]------------
[81.801] BTRFS: Transaction aborted (error -17)
[81.801] WARNING: fs/btrfs/block-group.c:2876 at btrfs_create_pending_block_groups+0x721/0x770 [btrfs], CPU#1: bash/319
[81.802] Modules linked in: virtio_net btrfs xor zstd_compress raid6_pq null_blk
[81.803] CPU: 1 UID: 0 PID: 319 Comm: bash Kdump: loaded Not tainted 6.19.0-rc6+ #319 NONE
[81.803] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.17.0-2-2 04/01/2014
[81.804] RIP: 0010:btrfs_create_pending_block_groups+0x723/0x770 [btrfs]
[81.806] RSP: 0018:ffffa36241a6bce8 EFLAGS: 00010282
[81.806] RAX: 000000000000000d RBX: ffff8e699921e400 RCX: 0000000000000000
[81.807] RDX: 0000000002040001 RSI: 00000000ffffffef RDI: ffffffffc0608bf0
[81.807] RBP: 00000000ffffffef R08: ffff8e69830f6000 R09: 0000000000000007
[81.808] R10: ffff8e699921e5e8 R11: 0000000000000000 R12: ffff8e6999228000
[81.808] R13: ffff8e6984d82000 R14: ffff8e69966a69c0 R15: ffff8e69aa47b000
[81.809] FS: 00007fec6bdd9740(0000) GS:ffff8e6b1b379000(0000) knlGS:0000000000000000
[81.809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[81.810] CR2: 00005604833670f0 CR3: 0000000116679000 CR4: 00000000000006f0
[81.810] Call Trace:
[81.810] <TASK>
[81.810] __btrfs_end_transaction+0x3e/0x2b0 [btrfs]
[81.811] btrfs_force_chunk_alloc_store+0xcd/0x140 [btrfs]
[81.811] kernfs_fop_write_iter+0x15f/0x240
[81.812] vfs_write+0x264/0x500
[81.812] ksys_write+0x6c/0xe0
[81.812] do_syscall_64+0x66/0x770
[81.812] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[81.813] RIP: 0033:0x7fec6be66197
[81.814] RSP: 002b:00007fffb159dd30 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[81.815] RAX: ffffffffffffffda RBX: 00007fec6bdd9740 RCX: 00007fec6be66197
[81.815] RDX: 0000000000000002 RSI: 0000560483374f80 RDI: 0000000000000001
[81.816] RBP: 0000560483374f80 R08: 0000000000000000 R09: 0000000000000000
[81.816] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002
[81.817] R13: 00007fec6bfb85c0 R14: 00007fec6bfb5ee0 R15: 00005604833729c0
[81.817] </TASK>
[81.817] irq event stamp: 20039
[81.818] hardirqs last enabled at (20047): [<ffffffff99a68302>] __up_console_sem+0x52/0x60
[81.818] hardirqs last disabled at (20056): [<ffffffff99a682e7>] __up_console_sem+0x37/0x60
[81.819] softirqs last enabled at (19470): [<ffffffff999d2b46>] __irq_exit_rcu+0x96/0xc0
[81.819] softirqs last disabled at (19463): [<ffffffff999d2b46>] __irq_exit_rcu+0x96/0xc0
[81.820] ---[ end trace 0000000000000000 ]---
[81.820] BTRFS: error (device dm-7 state A) in btrfs_create_pending_block_groups:2876: errno=-17 Object already exists
Inspecting these aborts with drgn, I observed a pattern of overlapping
chunk_maps. Note how stripe 1 of the first chunk overlaps in physical
address with stripe 0 of the second chunk.
Physical Start Physical End Length Logical Type Stripe
----------------------------------------------------------------------------------------------------
0x0000000102500000 0x0000000142500000 1.0G 0x0000000641d00000 META|DUP 0/2
0x0000000142500000 0x0000000182500000 1.0G 0x0000000641d00000 META|DUP 1/2
0x0000000142500000 0x0000000182500000 1.0G 0x0000000601d00000 META|DUP 0/2
0x0000000182500000 0x00000001c2500000 1.0G 0x0000000601d00000 META|DUP 1/2
Now how could this possibly happen? All chunk allocation is protected by
the chunk_mutex so racing allocations should see a consistent view of
the CHUNK_ALLOCATED bit in the chunk allocation extent-io-tree
(device->alloc_state as set by chunk_map_device_set_bits()) The tree
itself is protected by a spin lock, and clearing/setting the bits is
always protected by fs_info->mapping_tree_lock, so no race is apparent.
It turns out that there is a subtle bug in the logic regarding chunk
allocations that have happened in the current transaction, known as
"pending extents". The chunk allocation as defined in
find_free_dev_extent() is a loop which searches the commit root of the
dev_root and looks for gaps between DEV_EXTENT items. For those gaps, it
then checks alloc_state bitmap for any pending extents and adjusts the
hole that it finds accordingly. However, the logic in that adjustment
assumes that the first pending extent is the only one in that range.
e.g., given a layout with two non-consecutive pending extents in a hole
passed to dev_extent_hole_check() via *hole_start and *hole_size:
|----pending A----| real hole |----pending B----|
| candidate hole |
*hole_start *hole_start + *hole_size
the code incorrectly returns a "hole" from the end of pending extent A
until the passed in hole end, failing to account for pending B.
However, it is not entirely obvious that it is actually possible to
produce such a layout. I was able to reproduce it, but with some
contortions: I continued to use the force chunk allocation sysfs file
and I introduced a long delay (10 seconds) into the start of the cleaner
thread. I also prevented the unused bgs cleaning logic from ever
deleting metadata bgs. These help make it easier to deterministically
produce the condition but shouldn't really matter if you imagine the
conditions happening by race/luck. Allocations/frees can happen
concurrently with the cleaner thread preparing to process an unused
extent and both create some used chunks with an unused chunk
interleaved, all during one transaction. Then btrfs_delete_unused_bgs()
sees the unused one and clears it, leaving a range with several pending
chunk allocations and a gap in the middle.
The basic idea is that the unused_bgs cleanup work happens on a worker
so if we allocate 3 block groups in one transaction, then the cleaner
work kicked off by the previous transaction comes through and deletes
the middle one of the 3, then the commit root shows no dev extents and
we have the bad pattern in the extent-io-tree. One final consideration
is that the code happens to loop to the next hole if there are no more
extents at all, so we need one more dev extent way past the area we are
working in. Something like the following demonstrates the technique:
# push the BG frontier out to 20G
fallocate -l 20G $mnt/foo
# allocate one more that will prevent the "no more dev extents" luck
fallocate -l 1G $mnt/sticky
# sync
sync
# clear out the allocation area
rm $mnt/foo
sync
_cleaner
# let everything quiesce
sleep 20
sync
# dev tree should have one bg 20G out and the rest at the beginning..
# sort of like an empty FS but with a random sticky chunk.
# kick off the cleaner in the background, remember it will sleep 10s
# before doing interesting work
_cleaner &
sleep 3
# create 3 trivial block groups, all empty, all immediately marked as unused.
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
echo 1 > "$(_btrfs_sysfs_space_info $dev data)/force_chunk_alloc"
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
# let the cleaner thread definitely finish, it will remove the data bg
sleep 10
# this allocation sees the non-consecutive pending metadata chunks with
# data chunk gap of 1G and allocates a 2G extent in that hole. ENOSPC!
echo 1 > "$(_btrfs_sysfs_space_info $dev metadata)/force_chunk_alloc"
As for the fix, it is not that obvious. I could not see a trivial way to
do it even by adding backup loops into find_free_dev_extent(), so I
opted to change the semantics of dev_extent_hole_check() to not stop
looping until it finds a sufficiently big hole. For clarity, this also
required changing the helper function contains_pending_extent() into two
new helpers which find the first pending extent and the first suitable
hole in a range.
I attempted to clean up the documentation and range calculations to be
as consistent and clear as possible for the future.
I also looked at the zoned case and concluded that the loop there is
different and not to be unified with this one. As far as I can tell, the
zoned check will only further constrain the hole so looping back to find
more holes is acceptable. Though given that zoned really only appends, I
find it highly unlikely that it is susceptible to this bug.
Fixes: 1b9845081633 ("Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole")
Reported-by: Dimitrios Apostolou <jimis@gmx.net>
Closes: https://lore.kernel.org/linux-btrfs/q7760374-q1p4-029o-5149-26p28421s468@tzk.arg/
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index d33780082b8d..329a922893b4 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1509,30 +1509,158 @@ struct btrfs_device *btrfs_scan_one_device(const char *path,
}
/*
- * Try to find a chunk that intersects [start, start + len] range and when one
- * such is found, record the end of it in *start
+ * Find the first pending extent intersecting a range.
+ *
+ * @device: the device to search
+ * @start: start of the range to check
+ * @len: length of the range to check
+ * @pending_start: output pointer for the start of the found pending extent
+ * @pending_end: output pointer for the end of the found pending extent (inclusive)
+ *
+ * Search for a pending chunk allocation that intersects the half-open range
+ * [start, start + len).
+ *
+ * Return: true if a pending extent was found, false otherwise.
+ * If the return value is true, store the first pending extent in
+ * [*pending_start, *pending_end]. Otherwise, the two output variables
+ * may still be modified, to something outside the range and should not
+ * be used.
*/
-static bool contains_pending_extent(struct btrfs_device *device, u64 *start,
- u64 len)
+static bool first_pending_extent(struct btrfs_device *device, u64 start, u64 len,
+ u64 *pending_start, u64 *pending_end)
{
- u64 physical_start, physical_end;
-
lockdep_assert_held(&device->fs_info->chunk_mutex);
- if (btrfs_find_first_extent_bit(&device->alloc_state, *start,
- &physical_start, &physical_end,
+ if (btrfs_find_first_extent_bit(&device->alloc_state, start,
+ pending_start, pending_end,
CHUNK_ALLOCATED, NULL)) {
- if (in_range(physical_start, *start, len) ||
- in_range(*start, physical_start,
- physical_end + 1 - physical_start)) {
- *start = physical_end + 1;
+ if (in_range(*pending_start, start, len) ||
+ in_range(start, *pending_start, *pending_end + 1 - *pending_start)) {
return true;
}
}
return false;
}
+/*
+ * Find the first real hole accounting for pending extents.
+ *
+ * @device: the device containing the candidate hole
+ * @start: input/output pointer for the hole start position
+ * @len: input/output pointer for the hole length
+ * @min_hole_size: the size of hole we are looking for
+ *
+ * Given a potential hole specified by [*start, *start + *len), check for pending
+ * chunk allocations within that range. If pending extents are found, the hole is
+ * adjusted to represent the first true free space that is large enough when
+ * accounting for pending chunks.
+ *
+ * Note that this function must handle various cases involving non consecutive
+ * pending extents.
+ *
+ * Returns: true if a suitable hole was found and false otherwise.
+ * If the return value is true, then *start and *len are set to represent the hole.
+ * If the return value is false, then *start is set to the largest hole we
+ * found and *len is set to its length.
+ * If there are no holes at all, then *start is set to the end of the range and
+ * *len is set to 0.
+ */
+static bool find_hole_in_pending_extents(struct btrfs_device *device, u64 *start,
+ u64 *len, u64 min_hole_size)
+{
+ u64 pending_start, pending_end;
+ u64 end;
+ u64 max_hole_start = 0;
+ u64 max_hole_len = 0;
+
+ lockdep_assert_held(&device->fs_info->chunk_mutex);
+
+ if (*len == 0)
+ return false;
+
+ end = *start + *len - 1;
+
+ /*
+ * Loop until we either see a large enough hole or check every pending
+ * extent overlapping the candidate hole.
+ * At every hole that we observe, record it if it is the new max.
+ * At the end of the iteration, set the output variables to the max hole.
+ */
+ while (true) {
+ if (first_pending_extent(device, *start, *len, &pending_start, &pending_end)) {
+ /*
+ * Case 1: the pending extent overlaps the start of
+ * candidate hole. That means the true hole is after the
+ * pending extent, but we need to find the next pending
+ * extent to properly size the hole. In the next loop,
+ * we will reduce to case 2 or 3.
+ * e.g.,
+ *
+ * |----pending A----| real hole |----pending B----|
+ * | candidate hole |
+ * *start end
+ */
+ if (pending_start <= *start) {
+ *start = pending_end + 1;
+ goto next;
+ }
+ /*
+ * Case 2: The pending extent starts after *start (and overlaps
+ * [*start, end), so the first hole just goes up to the start
+ * of the pending extent.
+ * e.g.,
+ *
+ * | real hole |----pending A----|
+ * | candidate hole |
+ * *start end
+ */
+ *len = pending_start - *start;
+ if (*len > max_hole_len) {
+ max_hole_start = *start;
+ max_hole_len = *len;
+ }
+ if (*len >= min_hole_size)
+ break;
+ /*
+ * If the hole wasn't big enough, then we advance past
+ * the pending extent and keep looking.
+ */
+ *start = pending_end + 1;
+ goto next;
+ } else {
+ /*
+ * Case 3: There is no pending extent overlapping the
+ * range [*start, *start + *len - 1], so the only remaining
+ * hole is the remaining range.
+ * e.g.,
+ *
+ * | candidate hole |
+ * | real hole |
+ * *start end
+ */
+
+ if (*len > max_hole_len) {
+ max_hole_start = *start;
+ max_hole_len = *len;
+ }
+ break;
+ }
+next:
+ if (*start > end)
+ break;
+ *len = end - *start + 1;
+ }
+ if (max_hole_len) {
+ *start = max_hole_start;
+ *len = max_hole_len;
+ } else {
+ *start = end + 1;
+ *len = 0;
+ }
+ return max_hole_len >= min_hole_size;
+}
+
static u64 dev_extent_search_start(struct btrfs_device *device)
{
switch (device->fs_devices->chunk_alloc_policy) {
@@ -1597,59 +1725,57 @@ static bool dev_extent_hole_check_zoned(struct btrfs_device *device,
}
/*
- * Check if specified hole is suitable for allocation.
+ * Validate and adjust a hole for chunk allocation
+ *
+ * @device: the device containing the candidate hole
+ * @hole_start: input/output pointer for the hole start position
+ * @hole_size: input/output pointer for the hole size
+ * @num_bytes: minimum allocation size required
*
- * @device: the device which we have the hole
- * @hole_start: starting position of the hole
- * @hole_size: the size of the hole
- * @num_bytes: the size of the free space that we need
+ * Check if the specified hole is suitable for allocation and adjust it if
+ * necessary. The hole may be modified to skip over pending chunk allocations
+ * and to satisfy stricter zoned requirements on zoned filesystems.
*
- * This function may modify @hole_start and @hole_size to reflect the suitable
- * position for allocation. Returns 1 if hole position is updated, 0 otherwise.
+ * For regular (non-zoned) allocation, if the hole after adjustment is smaller
+ * than @num_bytes, the search continues past additional pending extents until
+ * either a sufficiently large hole is found or no more pending extents exist.
+ *
+ * Return: true if a suitable hole was found and false otherwise.
+ * If the return value is true, then *hole_start and *hole_size are set to
+ * represent the hole we found.
+ * If the return value is false, then *hole_start is set to the largest
+ * hole we found and *hole_size is set to its length.
+ * If there are no holes at all, then *hole_start is set to the end of the range
+ * and *hole_size is set to 0.
*/
static bool dev_extent_hole_check(struct btrfs_device *device, u64 *hole_start,
u64 *hole_size, u64 num_bytes)
{
- bool changed = false;
- u64 hole_end = *hole_start + *hole_size;
+ bool found = false;
+ const u64 hole_end = *hole_start + *hole_size - 1;
- for (;;) {
- /*
- * Check before we set max_hole_start, otherwise we could end up
- * sending back this offset anyway.
- */
- if (contains_pending_extent(device, hole_start, *hole_size)) {
- if (hole_end >= *hole_start)
- *hole_size = hole_end - *hole_start;
- else
- *hole_size = 0;
- changed = true;
- }
+ ASSERT(*hole_size > 0);
- switch (device->fs_devices->chunk_alloc_policy) {
- default:
- btrfs_warn_unknown_chunk_allocation(device->fs_devices->chunk_alloc_policy);
- fallthrough;
- case BTRFS_CHUNK_ALLOC_REGULAR:
- /* No extra check */
- break;
- case BTRFS_CHUNK_ALLOC_ZONED:
- if (dev_extent_hole_check_zoned(device, hole_start,
- hole_size, num_bytes)) {
- changed = true;
- /*
- * The changed hole can contain pending extent.
- * Loop again to check that.
- */
- continue;
- }
- break;
- }
+again:
+ *hole_size = hole_end - *hole_start + 1;
+ found = find_hole_in_pending_extents(device, hole_start, hole_size, num_bytes);
+ if (!found)
+ return found;
+ ASSERT(*hole_size >= num_bytes);
+ switch (device->fs_devices->chunk_alloc_policy) {
+ default:
+ btrfs_warn_unknown_chunk_allocation(device->fs_devices->chunk_alloc_policy);
+ fallthrough;
+ case BTRFS_CHUNK_ALLOC_REGULAR:
+ return found;
+ case BTRFS_CHUNK_ALLOC_ZONED:
+ if (dev_extent_hole_check_zoned(device, hole_start, hole_size, num_bytes))
+ goto again;
break;
}
- return changed;
+ return found;
}
/*
@@ -1708,7 +1834,7 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
ret = -ENOMEM;
goto out;
}
-again:
+
if (search_start >= search_end ||
test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state)) {
ret = -ENOSPC;
@@ -1795,11 +1921,7 @@ static int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
*/
if (search_end > search_start) {
hole_size = search_end - search_start;
- if (dev_extent_hole_check(device, &search_start, &hole_size,
- num_bytes)) {
- btrfs_release_path(path);
- goto again;
- }
+ dev_extent_hole_check(device, &search_start, &hole_size, num_bytes);
if (hole_size > max_hole_size) {
max_hole_start = search_start;
@@ -5022,6 +5144,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
u64 diff;
u64 start;
u64 free_diff = 0;
+ u64 pending_start, pending_end;
new_size = round_down(new_size, fs_info->sectorsize);
start = new_size;
@@ -5067,7 +5190,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
* in-memory chunks are synced to disk so that the loop below sees them
* and relocates them accordingly.
*/
- if (contains_pending_extent(device, &start, diff)) {
+ if (first_pending_extent(device, start, diff, &pending_start, &pending_end)) {
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_commit_transaction(trans);
if (ret)
]
|
| 246/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 29248
output: 36
thoughts: 66
Thoughts: Okay, here's my take on those thoughts, framed as my own:
**Analyzing a Code Commit**
Alright, so this commit's focus is clearly on `find_free_dev_extent` and those pending extents. My initial reaction is that it doesn't immediately jump out as being connected to that early `check_system_chunk` pattern we were looking at. Gotta be careful with assumptions, but that's the first impression.
Now, let's see... since we're diving into chunk removal, I think it's worth revisiting that `btrfs_remove_chunk` function. I'll fire up the git log again, but this time I'll use the `Since="5 years"` argument. That should give us a relevant timeframe for potential changes and how the surrounding code has evolved. This approach might give me a better understanding of how the pending extents code interacts, or doesn't interact, with chunk removal.
|
| 247/2 |
2026/08/08 04:06 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[btrfs_remove_chunk] Since:5 years]
Results: map[Output:6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
]
|
| 248/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 29598
output: 35
thoughts: 24
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm an expert in this field, and rendered in the first person:
**Investigating a Ten-Year-Old Code Issue**
Alright, I'm thinking we need to delve into the history to understand the root of this issue. My initial thought is to go straight to the git logs. We're dealing with something potentially related to system chunk checks, so let's zero in on that. I'm going to start by searching for instances of `check_system_chunk`. Given the age of the project and the nature of the problem, I'm setting a pretty wide time window – "10 years" should give us a good view back in time. This approach should allow me to filter out most of the noise and focus on commits that might be relevant to any past changes around system chunk verification. This will give a good start to determine how the current behavior came to be. I expect this initial search will hopefully give me some key commits to analyze.
|
| 249/2 |
2026/08/08 04:06 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[check_system_chunk] Since:10 years]
Results: map[Output:0d85f5c2dd91 btrfs: fix assertion failure when splitting ordered extent after transaction abort
eafa4fd0ad06 btrfs: fix exhaustion of the system chunk array due to concurrent allocations
a9143bd31c6a btrfs: block-group: Rework documentation of check_system_chunk function
451a2c130342 btrfs: Remove fs_info from check_system_chunk
]
|
| 250/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 29992
output: 23
thoughts: 30
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Fixing a Btrfs Chunk Array Exhaustion Bug**
Right, let's dive into this `eafa4fd0ad06 btrfs` commit: "fix exhaustion of the system chunk array due to concurrent allocations." The title alone immediately tells me this is likely a race condition issue, probably related to how the Btrfs filesystem manages its underlying block device space. Chunk arrays... that suggests we're dealing with how Btrfs slices up the available storage into manageably sized units, perhaps for RAID or other redundancy schemes.
"Exhaustion due to concurrent allocations"... Okay, so multiple threads or processes are trying to grab chunks simultaneously. That's a classic concurrency problem. The system's chunk array – I'm guessing this is some sort of in-memory data structure that tracks which chunks are free, in-use, or reserved – is getting hammered. Likely, there's a flaw in the locking or atomicity of the allocation code. Maybe there are insufficient locks around the chunk array itself? Or the locking granularity is too coarse, leading to contention. Could be a read-modify-write problem, where multiple allocators are seeing the same free space and all trying to claim it, ultimately resulting in an overlap.
I'd be looking closely at the specific functions involved in chunk allocation: how they determine if a chunk is free, how they mark it as allocated, and how they handle errors if allocation fails. I'd be looking for potential race conditions: a missing `mutex_lock/unlock`, a flawed use of `atomic` operations, or perhaps a lack of proper barrier synchronization. I'd need to consider how this interacts with the journal, too, because transactions and concurrent updates add another layer of complexity. If the fix is a simple lock add, that's pretty straightforward, though it'll be interesting to see if it causes any performance regression. If there's a more involved refactoring of the allocation logic, it'll take a lot of scrutiny to make sure the fix is sound. Ultimately, I'd want to understand the *precise* sequence of operations that lead to the exhaustion, so I can be sure the fix truly addresses the root cause and doesn't introduce any new vulnerabilities. Time to dig in!
|
| 251/2 |
2026/08/08 04:06 |
tool |
git-show |
0m
Args:
Results: map[Output:commit eafa4fd0ad06074da8be4e28ff93b4dca9ffa407
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed Mar 31 11:55:50 2021 +0100
btrfs: fix exhaustion of the system chunk array due to concurrent allocations
When we are running out of space for updating the chunk tree, that is,
when we are low on available space in the system space info, if we have
many task concurrently allocating block groups, via fallocate for example,
many of them can end up all allocating new system chunks when only one is
needed. In extreme cases this can lead to exhaustion of the system chunk
array, which has a size limit of 2048 bytes, and results in a transaction
abort with errno EFBIG, producing a trace in dmesg like the following,
which was triggered on a PowerPC machine with a node/leaf size of 64K:
[1359.518899] ------------[ cut here ]------------
[1359.518980] BTRFS: Transaction aborted (error -27)
[1359.519135] WARNING: CPU: 3 PID: 16463 at ../fs/btrfs/block-group.c:1968 btrfs_create_pending_block_groups+0x340/0x3c0 [btrfs]
[1359.519152] Modules linked in: (...)
[1359.519239] Supported: Yes, External
[1359.519252] CPU: 3 PID: 16463 Comm: stress-ng Tainted: G X 5.3.18-47-default #1 SLE15-SP3
[1359.519274] NIP: c008000000e36fe8 LR: c008000000e36fe4 CTR: 00000000006de8e8
[1359.519293] REGS: c00000056890b700 TRAP: 0700 Tainted: G X (5.3.18-47-default)
[1359.519317] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 48008222 XER: 00000007
[1359.519356] CFAR: c00000000013e170 IRQMASK: 0
[1359.519356] GPR00: c008000000e36fe4 c00000056890b990 c008000000e83200 0000000000000026
[1359.519356] GPR04: 0000000000000000 0000000000000000 0000d52a3b027651 0000000000000007
[1359.519356] GPR08: 0000000000000003 0000000000000001 0000000000000007 0000000000000000
[1359.519356] GPR12: 0000000000008000 c00000063fe44600 000000001015e028 000000001015dfd0
[1359.519356] GPR16: 000000000000404f 0000000000000001 0000000000010000 0000dd1e287affff
[1359.519356] GPR20: 0000000000000001 c000000637c9a000 ffffffffffffffe5 0000000000000000
[1359.519356] GPR24: 0000000000000004 0000000000000000 0000000000000100 ffffffffffffffc0
[1359.519356] GPR28: c000000637c9a000 c000000630e09230 c000000630e091d8 c000000562188b08
[1359.519561] NIP [c008000000e36fe8] btrfs_create_pending_block_groups+0x340/0x3c0 [btrfs]
[1359.519613] LR [c008000000e36fe4] btrfs_create_pending_block_groups+0x33c/0x3c0 [btrfs]
[1359.519626] Call Trace:
[1359.519671] [c00000056890b990] [c008000000e36fe4] btrfs_create_pending_block_groups+0x33c/0x3c0 [btrfs] (unreliable)
[1359.519729] [c00000056890ba90] [c008000000d68d44] __btrfs_end_transaction+0xbc/0x2f0 [btrfs]
[1359.519782] [c00000056890bae0] [c008000000e309ac] btrfs_alloc_data_chunk_ondemand+0x154/0x610 [btrfs]
[1359.519844] [c00000056890bba0] [c008000000d8a0fc] btrfs_fallocate+0xe4/0x10e0 [btrfs]
[1359.519891] [c00000056890bd00] [c0000000004a23b4] vfs_fallocate+0x174/0x350
[1359.519929] [c00000056890bd50] [c0000000004a3cf8] ksys_fallocate+0x68/0xf0
[1359.519957] [c00000056890bda0] [c0000000004a3da8] sys_fallocate+0x28/0x40
[1359.519988] [c00000056890bdc0] [c000000000038968] system_call_exception+0xe8/0x170
[1359.520021] [c00000056890be20] [c00000000000cb70] system_call_common+0xf0/0x278
[1359.520037] Instruction dump:
[1359.520049] 7d0049ad 40c2fff4 7c0004ac 71490004 40820024 2f83fffb 419e0048 3c620000
[1359.520082] e863bcb8 7ec4b378 48010d91 e8410018 <0fe00000> 3c820000 e884bcc8 7ec6b378
[1359.520122] ---[ end trace d6c186e151022e20 ]---
The following steps explain how we can end up in this situation:
1) Task A is at check_system_chunk(), either because it is allocating a
new data or metadata block group, at btrfs_chunk_alloc(), or because
it is removing a block group or turning a block group RO. It does not
matter why;
2) Task A sees that there is not enough free space in the system
space_info object, that is 'left' is < 'thresh'. And at this point
the system space_info has a value of 0 for its 'bytes_may_use'
counter;
3) As a consequence task A calls btrfs_alloc_chunk() in order to allocate
a new system block group (chunk) and then reserves 'thresh' bytes in
the chunk block reserve with the call to btrfs_block_rsv_add(). This
changes the chunk block reserve's 'reserved' and 'size' counters by an
amount of 'thresh', and changes the 'bytes_may_use' counter of the
system space_info object from 0 to 'thresh'.
Also during its call to btrfs_alloc_chunk(), we end up increasing the
value of the 'total_bytes' counter of the system space_info object by
8MiB (the size of a system chunk stripe). This happens through the
call chain:
btrfs_alloc_chunk()
create_chunk()
btrfs_make_block_group()
btrfs_update_space_info()
4) After it finishes the first phase of the block group allocation, at
btrfs_chunk_alloc(), task A unlocks the chunk mutex;
5) At this point the new system block group was added to the transaction
handle's list of new block groups, but its block group item, device
items and chunk item were not yet inserted in the extent, device and
chunk trees, respectively. That only happens later when we call
btrfs_finish_chunk_alloc() through a call to
btrfs_create_pending_block_groups();
Note that only when we update the chunk tree, through the call to
btrfs_finish_chunk_alloc(), we decrement the 'reserved' counter
of the chunk block reserve as we COW/allocate extent buffers,
through:
btrfs_alloc_tree_block()
btrfs_use_block_rsv()
btrfs_block_rsv_use_bytes()
And the system space_info's 'bytes_may_use' is decremented everytime
we allocate an extent buffer for COW operations on the chunk tree,
through:
btrfs_alloc_tree_block()
btrfs_reserve_extent()
find_free_extent()
btrfs_add_reserved_bytes()
If we end up COWing less chunk btree nodes/leaves than expected, which
is the typical case since the amount of space we reserve is always
pessimistic to account for the worst possible case, we release the
unused space through:
btrfs_create_pending_block_groups()
btrfs_trans_release_chunk_metadata()
btrfs_block_rsv_release()
block_rsv_release_bytes()
btrfs_space_info_free_bytes_may_use()
But before task A gets into btrfs_create_pending_block_groups()...
6) Many other tasks start allocating new block groups through fallocate,
each one does the first phase of block group allocation in a
serialized way, since btrfs_chunk_alloc() takes the chunk mutex
before calling check_system_chunk() and btrfs_alloc_chunk().
However before everyone enters the final phase of the block group
allocation, that is, before calling btrfs_create_pending_block_groups(),
new tasks keep coming to allocate new block groups and while at
check_system_chunk(), the system space_info's 'bytes_may_use' keeps
increasing each time a task reserves space in the chunk block reserve.
This means that eventually some other task can end up not seeing enough
free space in the system space_info and decide to allocate yet another
system chunk.
This may repeat several times if yet more new tasks keep allocating
new block groups before task A, and all the other tasks, finish the
creation of the pending block groups, which is when reserved space
in excess is released. Eventually this can result in exhaustion of
system chunk array in the superblock, with btrfs_add_system_chunk()
returning EFBIG, resulting later in a transaction abort.
Even when we don't reach the extreme case of exhausting the system
array, most, if not all, unnecessarily created system block groups
end up being unused since when finishing creation of the first
pending system block group, the creation of the following ones end
up not needing to COW nodes/leaves of the chunk tree, so we never
allocate and deallocate from them, resulting in them never being
added to the list of unused block groups - as a consequence they
don't get deleted by the cleaner kthread - the only exceptions are
if we unmount and mount the filesystem again, which adds any unused
block groups to the list of unused block groups, if a scrub is
run, which also adds unused block groups to the unused list, and
under some circumstances when using a zoned filesystem or async
discard, which may also add unused block groups to the unused list.
So fix this by:
*) Tracking the number of reserved bytes for the chunk tree per
transaction, which is the sum of reserved chunk bytes by each
transaction handle currently being used;
*) When there is not enough free space in the system space_info,
if there are other transaction handles which reserved chunk space,
wait for some of them to complete in order to have enough excess
reserved space released, and then try again. Otherwise proceed with
the creation of a new system chunk.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 85077c95b4f7..293f3169be80 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3273,6 +3273,7 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
*/
void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
+ struct btrfs_transaction *cur_trans = trans->transaction;
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
@@ -3287,6 +3288,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
+again:
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
@@ -3305,6 +3307,58 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ u64 reserved = atomic64_read(&cur_trans->chunk_bytes_reserved);
+
+ /*
+ * If there's not available space for the chunk tree (system
+ * space) and there are other tasks that reserved space for
+ * creating a new system block group, wait for them to complete
+ * the creation of their system block group and release excess
+ * reserved space. We do this because:
+ *
+ * *) We can end up allocating more system chunks than necessary
+ * when there are multiple tasks that are concurrently
+ * allocating block groups, which can lead to exhaustion of
+ * the system array in the superblock;
+ *
+ * *) If we allocate extra and unnecessary system block groups,
+ * despite being empty for a long time, and possibly forever,
+ * they end not being added to the list of unused block groups
+ * because that typically happens only when deallocating the
+ * last extent from a block group - which never happens since
+ * we never allocate from them in the first place. The few
+ * exceptions are when mounting a filesystem or running scrub,
+ * which add unused block groups to the list of unused block
+ * groups, to be deleted by the cleaner kthread.
+ * And even when they are added to the list of unused block
+ * groups, it can take a long time until they get deleted,
+ * since the cleaner kthread might be sleeping or busy with
+ * other work (deleting subvolumes, running delayed iputs,
+ * defrag scheduling, etc);
+ *
+ * This is rare in practice, but can happen when too many tasks
+ * are allocating blocks groups in parallel (via fallocate())
+ * and before the one that reserved space for a new system block
+ * group finishes the block group creation and releases the space
+ * reserved in excess (at btrfs_create_pending_block_groups()),
+ * other tasks end up here and see free system space temporarily
+ * not enough for updating the chunk tree.
+ *
+ * We unlock the chunk mutex before waiting for such tasks and
+ * lock it again after the wait, otherwise we would deadlock.
+ * It is safe to do so because allocating a system chunk is the
+ * first thing done while allocating a new block group.
+ */
+ if (reserved > trans->chunk_bytes_reserved) {
+ const u64 min_needed = reserved - thresh;
+
+ mutex_unlock(&fs_info->chunk_mutex);
+ wait_event(cur_trans->chunk_reserve_wait,
+ atomic64_read(&cur_trans->chunk_bytes_reserved) <=
+ min_needed);
+ mutex_lock(&fs_info->chunk_mutex);
+ goto again;
+ }
/*
* Ignore failure to create system chunk. We might end up not
@@ -3319,8 +3373,10 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
thresh, BTRFS_RESERVE_NO_FLUSH);
- if (!ret)
+ if (!ret) {
+ atomic64_add(thresh, &cur_trans->chunk_bytes_reserved);
trans->chunk_bytes_reserved += thresh;
+ }
}
}
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index acff6bb49a97..97c5e7396bce 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -260,6 +260,7 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_transaction *cur_trans = trans->transaction;
if (!trans->chunk_bytes_reserved)
return;
@@ -268,6 +269,8 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
+ atomic64_sub(trans->chunk_bytes_reserved, &cur_trans->chunk_bytes_reserved);
+ cond_wake_up(&cur_trans->chunk_reserve_wait);
trans->chunk_bytes_reserved = 0;
}
@@ -383,6 +386,8 @@ static noinline int join_transaction(struct btrfs_fs_info *fs_info,
spin_lock_init(&cur_trans->dropped_roots_lock);
INIT_LIST_HEAD(&cur_trans->releasing_ebs);
spin_lock_init(&cur_trans->releasing_ebs_lock);
+ atomic64_set(&cur_trans->chunk_bytes_reserved, 0);
+ init_waitqueue_head(&cur_trans->chunk_reserve_wait);
list_add_tail(&cur_trans->list, &fs_info->trans_list);
extent_io_tree_init(fs_info, &cur_trans->dirty_pages,
IO_TREE_TRANS_DIRTY_PAGES, fs_info->btree_inode);
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index dd7c3eea08ad..364cfbb4c5c5 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -96,6 +96,13 @@ struct btrfs_transaction {
spinlock_t releasing_ebs_lock;
struct list_head releasing_ebs;
+
+ /*
+ * The number of bytes currently reserved, by all transaction handles
+ * attached to this transaction, for metadata extents of the chunk tree.
+ */
+ atomic64_t chunk_bytes_reserved;
+ wait_queue_head_t chunk_reserve_wait;
};
#define __TRANS_FREEZABLE (1U << 0)
]
|
| 252/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 36519
output: 25
thoughts: 501
Thoughts: **Analyzing Chunk Metadata Management in Btrfs**
Okay, so this commit's highlighting something interesting: `check_system_chunk` is being used in a way that involves both locking and unlocking `fs_info->chunk_mutex` internally. Now, I see the proposed fix involves: grabbing the mutex, reserving chunk space, calling `check_system_chunk`, and then conditionally unlocking the mutex *and* releasing metadata if space isn't sufficient. This approach of reserving space early and releasing it if unused catches my attention; I wonder if this pattern is implemented anywhere else in the code base.
My initial thought is to check some other areas where chunk allocation is crucial, like `btrfs_remove_block_group`, `btrfs_chunk_alloc`, or perhaps even `btrfs_inc_block_group_ro`. I'm starting to get a better grasp.
In `btrfs_inc_block_group_ro`, I see `check_system_chunk` is called, and then `btrfs_end_transaction` is called. Ah, but it locks, calls `check_system_chunk`, and unlocks the mutex. It reserves chunk metadata, but it doesn't immediately release it. It just reserves it and then ends the transaction. But wait, `btrfs_end_transaction` will handle releasing the metadata if it isn't actually needed. Okay, that makes sense.
So, to get a handle on the bigger picture of chunk metadata usage, I'm going to search for instances of `btrfs_trans_release_chunk_metadata` to map out the call graph and see how this specific release operation fits into the overall Btrfs transaction handling.
|
| 253/2 |
2026/08/08 04:06 |
tool |
grepper |
0m
Args: map[Expression:btrfs_trans_release_chunk_metadata]
Results: map[Output:fs/btrfs/block-group.c=2920=void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
--
fs/btrfs/block-group.c-3003- }
fs/btrfs/block-group.c:3004: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/block-group.c-3005-}
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4222-out:
fs/btrfs/block-group.c:4223: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/block-group.c-4224-
--
fs/btrfs/block-group.c=4574=void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
--
fs/btrfs/block-group.c-4599- * update the chunk btree while holding the chunk mutex. After the update to the
fs/btrfs/block-group.c:4600: * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
fs/btrfs/block-group.c-4601- *
--
fs/btrfs/relocation.c=2638=static int relocate_cowonly_block(struct btrfs_trans_handle *trans,
--
fs/btrfs/relocation.c-2668- if (root == root->fs_info->chunk_root)
fs/btrfs/relocation.c:2669: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/relocation.c-2670- if (ret > 0)
--
fs/btrfs/relocation.c=4738=static int remove_chunk_stripes(struct btrfs_trans_handle *trans,
--
fs/btrfs/relocation.c-4759- }
fs/btrfs/relocation.c:4760: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/relocation.c-4761- return ret;
--
fs/btrfs/relocation.c-4774- btrfs_release_path(path);
fs/btrfs/relocation.c:4775: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/relocation.c-4776-
--
fs/btrfs/transaction.c=243=static inline int extwriter_counter_read(struct btrfs_transaction *trans)
--
fs/btrfs/transaction.c-254- */
fs/btrfs/transaction.c:255:void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
fs/btrfs/transaction.c-256-{
--
fs/btrfs/transaction.c=1076=static int __btrfs_end_transaction(struct btrfs_trans_handle *trans,
--
fs/btrfs/transaction.c-1093-
fs/btrfs/transaction.c:1094: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/transaction.c-1095-
--
fs/btrfs/transaction.c=2230=int btrfs_commit_transaction(struct btrfs_trans_handle *trans)
--
fs/btrfs/transaction.c-2553-
fs/btrfs/transaction.c:2554: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/transaction.c-2555-
--
fs/btrfs/transaction.c-2660- btrfs_cleanup_pending_block_groups(trans);
fs/btrfs/transaction.c:2661: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/transaction.c-2662- trans->block_rsv = NULL;
--
fs/btrfs/transaction.h=319=void btrfs_add_dropped_root(struct btrfs_trans_handle *trans,
fs/btrfs/transaction.h-320- struct btrfs_root *root);
fs/btrfs/transaction.h:321:void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans);
fs/btrfs/transaction.h-322-void __cold __btrfs_abort_transaction(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=2067=static int btrfs_add_dev_item(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-2087- &key, sizeof(*dev_item));
fs/btrfs/volumes.c:2088: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-2089- if (ret)
--
fs/btrfs/volumes.c=2135=static int btrfs_rm_dev_item(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-2152- ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
fs/btrfs/volumes.c:2153: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-2154- if (ret > 0)
--
fs/btrfs/volumes.c=2778=static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
--
fs/btrfs/volumes.c-2802- ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
fs/btrfs/volumes.c:2803: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-2804- if (ret < 0)
--
fs/btrfs/volumes.c=3144=int btrfs_grow_device(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c-3183- ret = btrfs_update_device(trans, device);
fs/btrfs/volumes.c:3184: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-3185-
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3563- */
fs/btrfs/volumes.c:3564: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-3565-
--
fs/btrfs/volumes.c=5226=int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
--
fs/btrfs/volumes.c-5409- ret = btrfs_update_device(trans, device);
fs/btrfs/volumes.c:5410: btrfs_trans_release_chunk_metadata(trans);
fs/btrfs/volumes.c-5411- if (unlikely(ret < 0)) {
]
|
| 254/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 39432
output: 39
thoughts: 50
Thoughts: Okay, let's dive into this.
**Investigating `btrfs_shrink_device` in btrfs**
Alright, I need to understand this `btrfs_shrink_device` function within `fs/btrfs/volumes.c`. The first thing that jumps out is that it calls `btrfs_trans_release_chunk_metadata(trans)`. My gut feeling is to understand the context of the transaction, and how this function plays with the metadata. I need to understand if the function performs any sort of early check before attempting to do anything with the chunk metadata. Understanding this will be key to understanding the full implications of its role in the file system.
|
| 255/2 |
2026/08/08 04:06 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:5226 LineCount:100]
Results: map[Contents:5226: int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
5227: {
5228: struct btrfs_fs_info *fs_info = device->fs_info;
5229: struct btrfs_root *root = fs_info->dev_root;
5230: struct btrfs_trans_handle *trans;
5231: struct btrfs_dev_extent *dev_extent = NULL;
5232: struct btrfs_path *path;
5233: u64 length;
5234: u64 chunk_offset;
5235: int ret;
5236: int slot;
5237: int failed = 0;
5238: bool retried = false;
5239: struct extent_buffer *l;
5240: struct btrfs_key key;
5241: struct btrfs_super_block *super_copy = fs_info->super_copy;
5242: u64 old_total = btrfs_super_total_bytes(super_copy);
5243: u64 old_size = btrfs_device_get_total_bytes(device);
5244: u64 diff;
5245: u64 start;
5246: u64 free_diff = 0;
5247: u64 pending_start, pending_end;
5248:
5249: new_size = round_down(new_size, fs_info->sectorsize);
5250: start = new_size;
5251: diff = round_down(old_size - new_size, fs_info->sectorsize);
5252:
5253: if (test_bit(BTRFS_DEV_STATE_REPLACE_TGT, &device->dev_state))
5254: return -EINVAL;
5255:
5256: path = btrfs_alloc_path();
5257: if (!path)
5258: return -ENOMEM;
5259:
5260: path->reada = READA_BACK;
5261:
5262: trans = btrfs_start_transaction(root, 0);
5263: if (IS_ERR(trans)) {
5264: btrfs_free_path(path);
5265: return PTR_ERR(trans);
5266: }
5267:
5268: mutex_lock(&fs_info->chunk_mutex);
5269:
5270: btrfs_device_set_total_bytes(device, new_size);
5271: if (test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state)) {
5272: device->fs_devices->total_rw_bytes -= diff;
5273:
5274: /*
5275: * The new free_chunk_space is new_size - used, so we have to
5276: * subtract the delta of the old free_chunk_space which included
5277: * old_size - used. If used > new_size then just subtract this
5278: * entire device's free space.
5279: */
5280: if (device->bytes_used < new_size)
5281: free_diff = (old_size - device->bytes_used) -
5282: (new_size - device->bytes_used);
5283: else
5284: free_diff = old_size - device->bytes_used;
5285: atomic64_sub(free_diff, &fs_info->free_chunk_space);
5286: }
5287:
5288: btrfs_update_per_profile_avail(fs_info);
5289: /*
5290: * Once the device's size has been set to the new size, ensure all
5291: * in-memory chunks are synced to disk so that the loop below sees them
5292: * and relocates them accordingly.
5293: */
5294: if (btrfs_first_pending_extent(device, start, diff, &pending_start, &pending_end)) {
5295: mutex_unlock(&fs_info->chunk_mutex);
5296: ret = btrfs_commit_transaction(trans);
5297: if (ret)
5298: goto done;
5299: } else {
5300: mutex_unlock(&fs_info->chunk_mutex);
5301: btrfs_end_transaction(trans);
5302: }
5303:
5304: again:
5305: key.objectid = device->devid;
5306: key.type = BTRFS_DEV_EXTENT_KEY;
5307: key.offset = (u64)-1;
5308:
5309: do {
5310: mutex_lock(&fs_info->reclaim_bgs_lock);
5311: ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
5312: if (ret < 0) {
5313: mutex_unlock(&fs_info->reclaim_bgs_lock);
5314: goto done;
5315: }
5316:
5317: ret = btrfs_previous_item(root, path, 0, key.type);
5318: if (ret) {
5319: mutex_unlock(&fs_info->reclaim_bgs_lock);
5320: if (ret < 0)
5321: goto done;
5322: ret = 0;
5323: btrfs_release_path(path);
5324: break;
5325: }
]
|
| 256/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 41296
output: 39
thoughts: 39
Thoughts: Okay, let's see...
**Analyzing `btrfs_shrink_device` and `btrfs_remove_chunk`**
Hang on a second. It seems like `btrfs_shrink_device` only goes ahead and commits the transaction if it finds pending extents. That's a key detail I might have overlooked before. I need to understand the implications of this. Now, with that fresh in my mind, let me go back and give `btrfs_remove_chunk` another, more careful look. I'll re-evaluate its behavior in light of this new piece of information about how `btrfs_shrink_device` handles transactions. I need to make sure I grasp the interaction between these two functions completely.
|
| 257/2 |
2026/08/08 04:06 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3453 LineCount:100]
Results: map[Contents:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
]
|
| 258/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 43226
output: 38
thoughts: 607
Thoughts: **Analyzing a Btrfs Chunk Removal Issue**
Okay, so I'm looking at this `btrfs_remove_chunk` function, and it seems the logic already *should* handle the scenario. Specifically:
```c
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
```
The issue I'm seeing is that `btrfs_remove_dev_extents` is called *before* `check_system_chunk`. If `check_system_chunk` fails to allocate a system chunk, it silently ignores the error. This leads to `remove_chunk_item` failing with `-ENOSPC`. Then, the code *tries* to allocate a system chunk again:
```c
if (ret == -ENOSPC) {
sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
if (IS_ERR(sys_bg)) {
ret = PTR_ERR(sys_bg);
btrfs_abort_transaction(trans, ret);
goto out;
}
```
...and this subsequent `btrfs_create_chunk` fails *again* with `-ENOSPC`. The reason for this double failure is that `find_free_dev_extent` searches the commit root, and the device extents have *just* been freed in the current transaction, so they are not visible in the commit root.
So, the proposed solution seems reasonable: move the `check_system_chunk` call *before* `btrfs_remove_dev_extents`. This early check will prevent the problem.
```c
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
```
I like that it reserves space early, and then releases it:
`btrfs_trans_release_chunk_metadata(trans)`
Now, the question is, is this "early check and release" pattern used elsewhere in the Btrfs code? Let's quickly search for `btrfs_trans_release_chunk_metadata` in the relevant files, like `fs/btrfs/relocation.c` and `fs/btrfs/volumes.c`. Ah, here's `btrfs_finish_sprout` in `fs/btrfs/volumes.c`. Let's take a look.
|
| 259/2 |
2026/08/08 04:06 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:2778 LineCount:50]
Results: map[Contents:2778: static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
2779: {
2780: BTRFS_DEV_LOOKUP_ARGS(args);
2781: struct btrfs_fs_info *fs_info = trans->fs_info;
2782: struct btrfs_root *root = fs_info->chunk_root;
2783: BTRFS_PATH_AUTO_FREE(path);
2784: struct extent_buffer *leaf;
2785: struct btrfs_dev_item *dev_item;
2786: struct btrfs_device *device;
2787: struct btrfs_key key;
2788: u8 fs_uuid[BTRFS_FSID_SIZE];
2789: u8 dev_uuid[BTRFS_UUID_SIZE];
2790: int ret;
2791:
2792: path = btrfs_alloc_path();
2793: if (!path)
2794: return -ENOMEM;
2795:
2796: key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
2797: key.type = BTRFS_DEV_ITEM_KEY;
2798: key.offset = 0;
2799:
2800: while (1) {
2801: btrfs_reserve_chunk_metadata(trans, false);
2802: ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
2803: btrfs_trans_release_chunk_metadata(trans);
2804: if (ret < 0)
2805: return ret;
2806:
2807: leaf = path->nodes[0];
2808: next_slot:
2809: if (path->slots[0] >= btrfs_header_nritems(leaf)) {
2810: ret = btrfs_next_leaf(root, path);
2811: if (ret > 0)
2812: break;
2813: if (ret < 0)
2814: return ret;
2815: leaf = path->nodes[0];
2816: btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
2817: btrfs_release_path(path);
2818: continue;
2819: }
2820:
2821: btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
2822: if (key.objectid != BTRFS_DEV_ITEMS_OBJECTID ||
2823: key.type != BTRFS_DEV_ITEM_KEY)
2824: break;
2825:
2826: dev_item = btrfs_item_ptr(leaf, path->slots[0],
2827: struct btrfs_dev_item);
]
|
| 260/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 45442
output: 38
thoughts: 168
Thoughts: **Analyzing Btrfs Chunk Metadata Reservation**
Okay, let me break down what I'm seeing in `btrfs_finish_sprout`. The code block there is making me pause, specifically this part:
```c
btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
btrfs_trans_release_chunk_metadata(trans);
```
My initial thought is that `btrfs_search_slot` is being called with `ins_len = 0` and `cow = 1`. That's key. It indicates that the function will be *modifying* the B-tree, and the `cow = 1` implies a CoW operation is going to take place. This means that the metadata reservation is *consumed* by the B-tree operation. It reserves space, does the deed, and then, whatever's leftover from the reserved space is released. So, it's not some kind of early sanity check. It's more of a space reservation strategy, executing the operation, and then tidying up.
The whole process just strikes me as a bit odd. I'm left wondering about potential interactions with other parts of the filesystem. What is the corresponding process in a function like `btrfs_rm_dev_item`? I think I need to dig deeper into the allocation strategies to fully understand the rationale here.
|
| 261/2 |
2026/08/08 04:06 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:2135 LineCount:30]
Results: map[Contents:2135: static int btrfs_rm_dev_item(struct btrfs_trans_handle *trans,
2136: struct btrfs_device *device)
2137: {
2138: struct btrfs_root *root = device->fs_info->chunk_root;
2139: int ret;
2140: BTRFS_PATH_AUTO_FREE(path);
2141: struct btrfs_key key;
2142:
2143: path = btrfs_alloc_path();
2144: if (!path)
2145: return -ENOMEM;
2146:
2147: key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
2148: key.type = BTRFS_DEV_ITEM_KEY;
2149: key.offset = device->devid;
2150:
2151: btrfs_reserve_chunk_metadata(trans, false);
2152: ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
2153: btrfs_trans_release_chunk_metadata(trans);
2154: if (ret > 0)
2155: return -ENOENT;
2156: if (ret < 0)
2157: return ret;
2158:
2159: return btrfs_del_item(trans, root, path);
2160: }
2161:
2162: /*
2163: * Verify that @num_devices satisfies the RAID profile constraints in the whole
2164: * filesystem. It's up to the caller to adjust that number regarding eg. device
]
|
| 262/2 |
2026/08/08 04:06 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 46457
output: 40
thoughts: 700
Thoughts: **Analyzing Chunk Allocation in Btrfs**
Okay, so I'm looking at this code and trying to understand the chunk metadata reservation process, and something seems a little convoluted. Initially, I see it reserving chunk metadata, doing a `btrfs_search_slot`, releasing the metadata, and then calling `btrfs_del_item`. My immediate thought is, why is the reservation being released *before* the deletion? I quickly realized that `btrfs_del_item` itself likely doesn't need to allocate chunks directly. It's the `btrfs_search_slot` call, especially with `cow=1`, that handles the Copy-on-Write and therefore, the potential chunk allocation. `btrfs_del_item` then just modifies the already COWed leaf.
The proposed fix introduces a mutex lock and this interesting check:
```c
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
```
My first question is: is this a standard pattern? It looks like it's trying to determine if `check_system_chunk` failed to allocate a system chunk by comparing the value of `trans->chunk_bytes_reserved` before and after the call. If the values are the same, it implies no chunk was successfully allocated. A very hacky way to check.
Let's dissect `check_system_chunk`. It seems to delegate to `reserve_chunk_space`, which is the one that calls `btrfs_create_chunk`:
```c
if (left < bytes) {
bg = btrfs_create_chunk(trans, space_info, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
} else {
// ...
}
}
if (!ret) {
ret = btrfs_block_rsv_add(fs_info,
&fs_info->chunk_block_rsv,
bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
```
If `btrfs_create_chunk` fails, `ret` will be non-zero, and `trans->chunk_bytes_reserved` remains unchanged. But if, the pre-existing block has space, and `btrfs_block_rsv_add` suceeds, `trans->chunk_bytes_reserved` will be incremented. The crucial point is: if `trans->chunk_bytes_reserved` has the same value after `check_system_chunk` is called, it means *something* failed within that function, either `btrfs_create_chunk` or `btrfs_block_rsv_add`. Since `check_system_chunk` returns `void`, this is the only way to detect a failure. It seems a bit indirect to me.
I also see that `btrfs_reserve_chunk_metadata` returns `void`, so it would seem this could be changed there.
So, in `btrfs_chunk_alloc`...
|
| 263/2 |
2026/08/08 04:07 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4341 LineCount:50]
Results: map[Contents:4341: int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
4342: struct btrfs_space_info *space_info, u64 flags,
4343: enum btrfs_chunk_alloc_enum force)
4344: {
4345: struct btrfs_fs_info *fs_info = trans->fs_info;
4346: struct btrfs_block_group *ret_bg;
4347: bool wait_for_alloc = false;
4348: bool should_alloc = false;
4349: bool from_extent_allocation = false;
4350: int ret = 0;
4351:
4352: if (force == CHUNK_ALLOC_FORCE_FOR_EXTENT) {
4353: from_extent_allocation = true;
4354: force = CHUNK_ALLOC_FORCE;
4355: }
4356:
4357: /* Don't re-enter if we're already allocating a chunk */
4358: if (trans->allocating_chunk)
4359: return -ENOSPC;
4360: /*
4361: * Allocation of system chunks can not happen through this path, as we
4362: * could end up in a deadlock if we are allocating a data or metadata
4363: * chunk and there is another task modifying the chunk btree.
4364: *
4365: * This is because while we are holding the chunk mutex, we will attempt
4366: * to add the new chunk item to the chunk btree or update an existing
4367: * device item in the chunk btree, while the other task that is modifying
4368: * the chunk btree is attempting to COW an extent buffer while holding a
4369: * lock on it and on its parent - if the COW operation triggers a system
4370: * chunk allocation, then we can deadlock because we are holding the
4371: * chunk mutex and we may need to access that extent buffer or its parent
4372: * in order to add the chunk item or update a device item.
4373: *
4374: * Tasks that want to modify the chunk tree should reserve system space
4375: * before updating the chunk btree, by calling either
4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
4377: * It's possible that after a task reserves the space, it still ends up
4378: * here - this happens in the cases described above at do_chunk_alloc().
4379: * The task will have to either retry or fail.
4380: */
4381: if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
4382: return -ENOSPC;
4383:
4384: do {
4385: spin_lock(&space_info->lock);
4386: if (force < space_info->force_alloc)
4387: force = space_info->force_alloc;
4388: should_alloc = should_alloc_chunk(fs_info, space_info, force);
4389: if (space_info->full) {
4390: /* No more free physical space */
]
|
| 264/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 48936
output: 19
thoughts: 586
Thoughts: **Analyzing Btrfs Chunk Removal and System Chunk Allocation**
Okay, let's break this down. My initial thought process is this: I've got a situation where `btrfs_chunk_alloc` is returning `-ENOSPC` when the `BTRFS_BLOCK_GROUP_SYSTEM` flag is set. That means system chunks are managed specifically by `check_system_chunk`, calling `reserve_chunk_space`, which ultimately calls `btrfs_create_chunk`. Got it.
Now, I'm revisiting `btrfs_remove_chunk`. I see it calls `btrfs_remove_dev_extents` first. If *that* succeeds, it moves on. It locks `chunk_mutex`, marks the transaction as removing a chunk, and then calls `check_system_chunk`. If `check_system_chunk` fails *after* `btrfs_remove_dev_extents` has run, it doesn't reserve the space needed, so `remove_chunk_item` fails with `-ENOSPC`.
Here's the problem: Then, it attempts to allocate a system chunk using the same `btrfs_create_chunk` method. But this `btrfs_create_chunk` *also* fails with `-ENOSPC`. And the reason? `btrfs_remove_dev_extents` already removed the extents in the *current* transaction. So when `find_free_dev_extent` tries to look at the commit root, it can't *see* the space that was just freed. This transaction has already been changed.
So, the proposed fix is an early `check_system_chunk` *before* we call `btrfs_remove_dev_extents`. This seems like a reasonable approach. But I need to fully understand how we'll know if `check_system_chunk` actually *fails* in the early check. The text suggests that the proposed fix will inspect `trans->chunk_bytes_reserved`. That makes sense. It's the "indicator" of success.
Is this the *only* place this mechanism is used? I need to know if we're following an established practice or if this is an ad-hoc fix. I'll search the btrfs codebase for `chunk_bytes_reserved` to see if this pattern is used anywhere else. Understanding the broader context will confirm the proposed change is well-integrated and won't introduce new issues.
|
| 265/2 |
2026/08/08 04:07 |
tool |
grepper |
0m
Args: map[Expression:chunk_bytes_reserved]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4565- if (!ret)
fs/btrfs/block-group.c:4566: trans->chunk_bytes_reserved += bytes;
fs/btrfs/block-group.c-4567- }
--
fs/btrfs/transaction.c=255=void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
--
fs/btrfs/transaction.c-258-
fs/btrfs/transaction.c:259: if (!trans->chunk_bytes_reserved)
fs/btrfs/transaction.c-260- return;
--
fs/btrfs/transaction.c-262- btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
fs/btrfs/transaction.c:263: trans->chunk_bytes_reserved, NULL);
fs/btrfs/transaction.c:264: trans->chunk_bytes_reserved = 0;
fs/btrfs/transaction.c-265-}
--
fs/btrfs/transaction.h=139=struct btrfs_trans_handle {
--
fs/btrfs/transaction.h-142- u64 delayed_refs_bytes_reserved;
fs/btrfs/transaction.h:143: u64 chunk_bytes_reserved;
fs/btrfs/transaction.h-144- unsigned long delayed_ref_updates;
]
|
| 266/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 50513
output: 40
thoughts: 53
Thoughts: **Analyzing Btrfs Chunk Reservation**
Okay, there's a specific check for `trans->chunk_bytes_reserved` here, and I'm pretty sure I haven't seen this particular bit of code duplicated elsewhere. Let me just pause and verify. It’s always a good idea to double-check these assumptions before continuing.
Ah, yes, let me quickly navigate to `fs/btrfs/block-group.c` and take a look at `btrfs_reserve_chunk_metadata`. I need to ensure I'm not missing any similar reserve calculations or interactions. Just want to make sure the specific behavior related to `trans->chunk_bytes_reserved` here is unique and correctly handled within the btrfs codebase, especially in context of metadata chunk allocation.
|
| 267/2 |
2026/08/08 04:07 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4587 LineCount:40]
Results: map[Contents:4587: /*
4588: * Reserve space in the system space, if needed, for doing a modification to the
4589: * chunk btree.
4590: *
4591: * @trans: A transaction handle.
4592: * @is_item_insertion: Indicate if the modification is for inserting a new item
4593: * in the chunk btree or if it's for the deletion or update
4594: * of an existing item.
4595: *
4596: * This is used in a context where we need to update the chunk btree outside
4597: * block group allocation and removal, to avoid a deadlock with a concurrent
4598: * task that is allocating a metadata or data block group and therefore needs to
4599: * update the chunk btree while holding the chunk mutex. After the update to the
4600: * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
4601: *
4602: */
4603: void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
4604: bool is_item_insertion)
4605: {
4606: struct btrfs_fs_info *fs_info = trans->fs_info;
4607: u64 bytes;
4608:
4609: if (is_item_insertion)
4610: bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
4611: else
4612: bytes = btrfs_calc_metadata_size(fs_info, 1);
4613:
4614: mutex_lock(&fs_info->chunk_mutex);
4615: reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
4616: mutex_unlock(&fs_info->chunk_mutex);
4617: }
4618:
4619: void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
4620: {
4621: struct btrfs_block_group *block_group;
4622:
4623: block_group = btrfs_lookup_first_block_group(info, 0);
4624: while (block_group) {
4625: btrfs_wait_block_group_cache_done(block_group);
4626: spin_lock(&block_group->lock);
]
|
| 268/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 51475
output: 40
thoughts: 130
Thoughts: **My Analysis of `btrfs_reserve_chunk_metadata` and Related Error Handling**
Okay, so I'm looking at `btrfs_reserve_chunk_metadata`, which seems to be about pre-allocating space *before* modifying the chunk btree. It's used outside of the standard block group allocation/removal routines, which is important. And I see that it seemingly ignores failures when creating a system chunk, as it calls `reserve_chunk_space`. That's a potential area of concern, especially if `reserve_chunk_space` fails. This could lead to out-of-space errors later down the line if we're not careful.
The task is to distill past approaches to similar ENOSPC (No space left on device) issues and suggest relevant idioms/patterns for a potential fix. My mental checklist for similar bugs: what did we do before? The main strategies, as I see it, are:
1. **Early Reservation:** Reserving space *before* attempting the critical operation that might fail due to lack of space, ensuring that resources are available before the more complex operations happen.
2. **Robust Error Checking:** Implementing more thorough checks, or checking that reservation was successful, and then providing a strategy for recovery.
3. **Idempotency and Recovery:** In case of failure, what do we do? Do we need to ensure the action is idempotent, so that retries or alternative approaches are possible?
So, I need to look at commit history for situations where ENOSPC leads to file system aborts. I'll search for commits that deal with *early space reservation* or *enhanced error checking* to find related patterns and see what's been done in the past. This should give me a better understanding of how the community addresses similar issues.
|
| 269/2 |
2026/08/08 04:07 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[ENOSPC early reserve] Since:3 years]
Results: map[Output:5422fe71d26d ext4: avoid infinite loops caused by residual data
061c99169706 perf/hw_breakpoint: Return EOPNOTSUPP for unsupported breakpoint type
2f6397e448e6 btrfs: don't refill whole delayed refs block reserve when starting transaction
e85a0adacf17 btrfs: ensure releasing squota reserve on head refs
3022fd7af960 shmem: _add_to_page_cache() before shmem_inode_acct_blocks()
28270e25c69a btrfs: always reserve space for delayed refs when starting transaction
13bb483d32ab btrfs: zoned: activate metadata block group on write time
]
|
| 270/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 52226
output: 48
thoughts: 25
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Diving Back into the Btrfs Depths**
Right, let's circle back to that `ENOSPC` issue and those pesky aborts we've been seeing. I'm going to re-examine the `fs/btrfs/` directory, specifically looking for instances of both `ENOSPC` (out of space errors) and the word `abort`. The goal is to get a handle on the relationship. Maybe there is an overlooked interaction I missed, or perhaps an edge case I didn't identify previously. I'm hoping to discover the root cause, or at least uncover new clues. The kernel can be complex, and these types of things can be tricky to diagnose, so I'm going to need a thorough, second look at that code path.
|
| 271/2 |
2026/08/08 04:07 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[ENOSPC abort] PathPrefix:fs/btrfs/ Since:5 years]
Results: map[Output:40f2b11c1b7c btrfs: don't allow log trees to consume global reserve or overcommit metadata
574d93fc62e2 btrfs: be less aggressive with metadata overcommit when we can do full flushing
2672a26a7551 btrfs: use per-profile available space in calc_available_free_space()
52fead5eb8a7 btrfs: introduce the device layout aware per-profile available space
f9a4e3015db1 btrfs: reserve enough transaction items for qgroup ioctls
8ceaad6cd6e7 btrfs: do not ASSERT() when the fs flips RO inside btrfs_repair_io_failure()
b14c5e04bd0f btrfs: fix EEXIST abort due to non-consecutive gaps in chunk allocation
f4a9f219411f btrfs: do not delete unused block group if it may be used soon
a7f8de500e28 btrfs: return real error when orphan cleanup fails due to a transaction abort
1b6948acb854 btrfs: don't steal space from global rsv after a transaction abort
92fb94b69c6a btrfs: set cache_block_group_error if we find an error
2d82a40aa7d6 btrfs: fix deadlock when aborting transaction during relocation with scrub
09e44868f1e0 btrfs: do not abort transaction on failure to update log root
765c3fe99bcd btrfs: introduce BTRFS_RESERVE_FLUSH_EMERGENCY
8e327b9c0de8 btrfs: dump all space infos if we abort transaction due to ENOSPC
193b4e83986d btrfs: do not BUG_ON() on failure to update inode when setting xattr
c18e3235646a btrfs: reserve extra space for the free space tree
9506f9538206 btrfs: include the free space tree in the global rsv minimum calculation
]
|
| 272/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 53012
output: 27
thoughts: 30
Thoughts: Okay, here's my take on those thoughts, framed as an expert's inner monologue:
**Diving into Btrfs Space Reservation**
Alright, let's take a look at the commit `c18e3235646a` relating to Btrfs, specifically about reserving extra space for the free space tree. My initial reaction is that this is likely addressing a crucial aspect of Btrfs's design and performance. The free space tree, of course, is the heart of Btrfs's ability to manage its block allocation and avoid fragmentation, it is essential to the filesystem's proper operation. This usually means that reserving *extra* space for it suggests a proactive measure to prevent potential issues down the line. It sounds as though it is potentially related to preventing out-of-space errors, which can quickly degrade performance, and potentially lead to data loss if not addressed.
I'm immediately considering several potential rationales behind this. Perhaps they've identified a scenario where the free space tree itself can become fragmented or experience significant growth under specific workloads or data patterns. Or, maybe there's an increased need to ensure the free space tree can grow without impacting the filesystem's read/write performance. This is particularly important for metadata operations. Another possibility is that this is simply a preventative measure to reduce the chances of critical metadata running out of space. In any case, my interest is piqued. I'll need to dig into the commit details to understand the specific problem being addressed, the chosen solution, and the potential performance implications. It’s important to see if they've taken into account the impact of the reserved space on usable space and filesystem overhead. I'm keen to see if this affects the filesystem's reported capacity.
|
| 273/2 |
2026/08/08 04:07 |
tool |
git-show |
0m
Args:
Results: map[Output:commit c18e3235646a8ba74d013067a6475c8d262d3776
Author: Josef Bacik <josef@toxicpanda.com>
Date: Thu Dec 2 15:34:32 2021 -0500
btrfs: reserve extra space for the free space tree
Filipe reported a problem where sometimes he'd get an ENOSPC abort when
running delayed refs with generic/619 and the free space tree enabled.
This is partly because we do not reserve space for modifying the free
space tree, nor do we have a block rsv associated with that tree.
The delayed_refs_rsv tracks the amount of space required to run delayed
refs. This means 1 modification means 1 change to the extent root.
With the free space tree this turns into 2 changes, because modifying 1
extent means updating the extent tree and potentially updating the free
space tree to either remove that entry or add the free space. Thus if
we have the FST enabled, simply double the reservation size for our
modification.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-rsv.c b/fs/btrfs/block-rsv.c
index b3086f252ad0..b3ee49b0b1e8 100644
--- a/fs/btrfs/block-rsv.c
+++ b/fs/btrfs/block-rsv.c
@@ -426,6 +426,7 @@ void btrfs_init_root_block_rsv(struct btrfs_root *root)
switch (root->root_key.objectid) {
case BTRFS_CSUM_TREE_OBJECTID:
case BTRFS_EXTENT_TREE_OBJECTID:
+ case BTRFS_FREE_SPACE_TREE_OBJECTID:
root->block_rsv = &fs_info->delayed_refs_rsv;
break;
case BTRFS_ROOT_TREE_OBJECTID:
diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c
index da9d20813147..4176df149d04 100644
--- a/fs/btrfs/delayed-ref.c
+++ b/fs/btrfs/delayed-ref.c
@@ -84,6 +84,17 @@ void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr)
u64 num_bytes = btrfs_calc_insert_metadata_size(fs_info, nr);
u64 released = 0;
+ /*
+ * We have to check the mount option here because we could be enabling
+ * the free space tree for the first time and don't have the compat_ro
+ * option set yet.
+ *
+ * We need extra reservations if we have the free space tree because
+ * we'll have to modify that tree as well.
+ */
+ if (btrfs_test_opt(fs_info, FREE_SPACE_TREE))
+ num_bytes *= 2;
+
released = btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL);
if (released)
trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
@@ -108,6 +119,17 @@ void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans)
num_bytes = btrfs_calc_insert_metadata_size(fs_info,
trans->delayed_ref_updates);
+ /*
+ * We have to check the mount option here because we could be enabling
+ * the free space tree for the first time and don't have the compat_ro
+ * option set yet.
+ *
+ * We need extra reservations if we have the free space tree because
+ * we'll have to modify that tree as well.
+ */
+ if (btrfs_test_opt(fs_info, FREE_SPACE_TREE))
+ num_bytes *= 2;
+
spin_lock(&delayed_rsv->lock);
delayed_rsv->size += num_bytes;
delayed_rsv->full = 0;
]
|
| 274/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 54640
output: 37
thoughts: 35
Thoughts: Okay, here's my interpretation of those thoughts, crafted with the perspective of someone deeply familiar with the topic:
**Analyzing the Reservation Size Increase**
Hmm, this commit *just* doubles the reservation size, does it? That's a pretty straightforward change, but the implications could be wide-ranging depending on *why* it's needed. I need to understand the context. Doubling it suggests that the current allocation strategy is somehow falling short, or that usage has dramatically increased, or that we are planning for an increase in the size of the overall file system.
Alright, a simple size increase alone is not a huge issue, but it does highlight the fact that a deeper investigation is required to see what the reason is and also ensure this does not cause issues for users.
I think a good place to start would be to look at the commit history for anything related to `btrfs_remove_chunk`. Let me pull up the git log and restrict it to the last ten years, roughly around the time when this system matured. That function is probably involved in chunk management and is probably linked to whatever caused the change in the allocation size. Seeing how that function has been modified or interacted with over time could give me some clues as to the underlying problem. I am going to run that search now.
|
| 275/2 |
2026/08/08 04:07 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[btrfs_remove_chunk] Since:10 years]
Results: map[Output:6f926597f983 btrfs: abort transaction on error in btrfs_remove_block_group()
c41ec4529d34 btrfs: fix lockdep warning due to seqcount_mutex on 32bit arch
64bc6c2a3453 btrfs: Remove superfluous check form btrfs_remove_chunk
9e7cc91a6d18 btrfs: fix fsfreeze hang caused by delayed iputs deal
]
|
| 276/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 55112
output: 26
thoughts: 33
Thoughts: Here's my take on those thoughts, tailored for an expert audience:
**Analyzing the Btrfs Chunk Removal Patch**
Okay, so we're diving into commit `64bc6c2a3453`, specifically the one addressing a superfluous check within `btrfs_remove_chunk`. My immediate thought is, "Ah, unnecessary code streamlining – always a good thing." I'm curious what this "superfluous check" actually *was*. Given the function name, I'm expecting it was likely a redundant validation of some pre-existing condition, probably related to the chunk's state or the intended operation.
My initial hypothesis: It's probably a performance optimization. Btrfs is already a complex system, and every unnecessary conditional adds a bit of overhead, especially in a potentially hot code path like chunk removal. So, this cleanup should translate to a tiny gain in efficiency, perhaps noticeable under heavy I/O load or with extremely large file systems. It could also improve code readability, making the chunk removal logic easier to understand and maintain. I'm keen to see if the removal of the check opens any new opportunities for further optimization in this area. Perhaps it also simplifies some error handling paths, or removes a potential source of false negatives in certain error cases. Ultimately, it should be a net benefit, contributing to the overall stability and performance of the filesystem. I should take a look at the commit log itself.
|
| 277/2 |
2026/08/08 04:07 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 64bc6c2a34535515f8319b2054795f0ca0b842f9
Author: Nikolay Borisov <nborisov@suse.com>
Date: Fri Oct 26 14:43:19 2018 +0300
btrfs: Remove superfluous check form btrfs_remove_chunk
It's unnecessary to check map->stripes[i].dev for NULL given its value
is already set and dereferenced above the the check. No functional
changes.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index fc9a3d8f6238..0839fae337f6 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2848,13 +2848,11 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
mutex_unlock(&fs_info->chunk_mutex);
}
- if (map->stripes[i].dev) {
- ret = btrfs_update_device(trans, map->stripes[i].dev);
- if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
- btrfs_abort_transaction(trans, ret);
- goto out;
- }
+ ret = btrfs_update_device(trans, device);
+ if (ret) {
+ mutex_unlock(&fs_devices->device_list_mutex);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
}
}
mutex_unlock(&fs_devices->device_list_mutex);
]
|
| 278/2 |
2026/08/08 04:07 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 56062
output: 20
thoughts: 740
Thoughts: **My Assessment of the `btrfs_remove_chunk` ENOSPC Bug and Proposed Fix**
Alright, let's break this down. My initial thought is that we've got a classic ENOSPC bug in `btrfs_remove_chunk`. The crux is that `check_system_chunk` is silently failing to allocate a system chunk due to `find_free_dev_extent` only looking at the commit root, and not picking up space *just* freed by `btrfs_remove_dev_extents`. `check_system_chunk` ignores the allocation failure, and the subsequent `remove_chunk_item` call naturally fails with the expected ENOSPC.
My gut reaction to the proposed fix – the early `check_system_chunk` before the transaction - is sound. It forces the system to acknowledge the issue upfront and abort the process if necessary. We reserve space, check, release, then act.
Now, let’s consider the BTRFS context. I've taken a look around.
1. The fact that `check_system_chunk` ignores allocation errors is *known*. We've seen it in `reserve_chunk_space`. It's designed to rely on the *caller* handling potential ENOSPC or assume that the space *might not be needed*.
2. I’ve found that `btrfs_remove_chunk` already *has* a retry. If `remove_chunk_item` hits ENOSPC, it tries allocating a chunk again. The problem is that second attempt *also* fails for the same reason – the `find_free_dev_extent` routine is still searching the commit root, missing the freed extents from the ongoing transaction.
3. The pattern of *early* reservation, a *check*, and then a *release* if we need to abort - this is familiar in BTRFS. We see this in `btrfs_finish_sprout` and `btrfs_rm_dev_item`. We reserve chunk metadata, do the operation, and then release it, to manage those edge cases.
4. Here’s the rub, and this is where I’m getting more analytical: the proposed fix relies on checking `trans->chunk_bytes_reserved` *before* and *after* the `check_system_chunk` call. This *is not* a common pattern in BTRFS. Generally, these allocation functions return an error code like `-ENOSPC`. `check_system_chunk` itself returns `void`, which is awkward. We're using a sort of side-effect detection mechanism here.
5. Yes, the behavior of `find_free_dev_extent` not seeing freed space in the current transaction is *expected* to happen. It's meant to search the commit root, so we don't end up reusing potentially invalid space in case of a transaction abort. This is a deliberate design choice, and something we can’t fix.
6. Looking back at `fs/btrfs/volumes.c` and `fs/btrfs/block-group.c`, I'm not seeing any recent examples (last 3-5 years) of this *specific* approach of comparing `trans->chunk_bytes_reserved` to detect `check_system_chunk` failures.
So, while the proposed solution effectively works around the ENOSPC, the means by which it *detects* failure are non-standard. The *approach* of reserving space early, checking, and releasing fits into BTRFS’s complex space reservation, but the error checking is a little unusual. A cleaner approach might involve changing `check_system_chunk` itself (or a variant), or even the underlying reservation function, to actually *return* an error. However, given the nature of the issue, and the current constraints, it does seem this is the most prudent solution.
Let's do a quick double check to make sure `check_system_chunk` never did return an error, just in case.
|
| 279/2 |
2026/08/08 04:08 |
tool |
grepper |
0m
Error:
Args: map[Expression:int check_system_chunk]
Results:
|
| 280/2 |
2026/08/08 04:08 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 57706
output: 35
thoughts: 1150
Thoughts: **BTRFS Chunk Allocation Analysis and Proposed Fix**
Alright, let's break this down. My initial observation remains: `check_system_chunk` is the target. I need to understand its behavior and how the proposed fix addresses the core issue.
Firstly, I've confirmed the initial call locations. `check_system_chunk` is called in `btrfs_inc_block_group_ro` before the end of the transaction and in `do_chunk_alloc` before a data/metadata chunk is allocated. The goal is to catch system chunk allocation failures early.
The key is this: `check_system_chunk` is a `void` function. It intentionally ignores allocation failures. The design philosophy is that if reserving space for system chunks fails, the caller might not actually need it (COW-ed nodes, etc.). The expectation is that if space *is* actually required later, the subsequent tree modifications, like removing a chunk item in `btrfs_remove_chunk`, will fail with `-ENOSPC`. However, the root cause is that, because of the commit root, the `find_free_dev_extent` function doesn't see freed space in the current transaction, and the attempt to recover from this `-ENOSPC` via chunk allocation within `btrfs_remove_chunk` is also stymied. It all points to `btrfs_remove_dev_extents` that's freeing space being invisible to later chunk allocations within the *same* transaction.
So, the proposed fix introduces this novel pattern: reserve chunk space, immediately check, and re-reserve later. The trick is to compare `trans->chunk_bytes_reserved` before and after calling `check_system_chunk`. If `trans->chunk_bytes_reserved` remains the same, it implies `check_system_chunk` failed to allocate. **Critically, I haven't seen this exact idiom before within the BTRFS codebase.** This is a departure from established patterns.
Now, while BTRFS *does* use a reserve-and-release strategy for chunk space around tree modifications, like in `btrfs_finish_sprout` and `btrfs_rm_dev_item` (with `btrfs_reserve_chunk_metadata` and `btrfs_trans_release_chunk_metadata`), it usually holds the reservation *during* the tree modification. The uniqueness of this approach is that it reserves, *immediately* releases, and then re-reserves later. The reason is that `btrfs_remove_dev_extents` might trigger nested allocations, potentially clearing the reservation.
Therefore, although this technique is not standard BTRFS fare, it is a clever and reasonable solution, especially since `check_system_chunk` is a void function. However, the `chunk_bytes_reserved` comparison is the red flag.
**In the context of a fix, here's what I'd recommend:** While this "early check and release" strategy makes logical sense in BTRFS's transactional system to catch allocation failures before they cause transaction aborts, the approach of checking `trans->chunk_bytes_reserved` to infer success is non-idiomatic. Ideally, I'd want to modify `check_system_chunk` or the underlying `reserve_chunk_space` to return an error code or to create a helper function that explicitly checks for available system space and *signals* failure. That said, it may be impractical to change core BTRFS functions, so using the non-standard check could be a reasonable compromise. The key is to document this non-standard approach clearly. At least, I'd want to weigh the cost of changing the `void` function to return an error code, or introducing a `btrfs_check_system_space` helper, versus sticking with the current approach.
|
| 281/2 |
2026/08/08 04:08 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[check_system_chunk] Since:10 years]
Results: map[Output:0d85f5c2dd91 btrfs: fix assertion failure when splitting ordered extent after transaction abort
eafa4fd0ad06 btrfs: fix exhaustion of the system chunk array due to concurrent allocations
a9143bd31c6a btrfs: block-group: Rework documentation of check_system_chunk function
451a2c130342 btrfs: Remove fs_info from check_system_chunk
]
|
| 282/2 |
2026/08/08 04:08 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 59832
output: 26
thoughts: 31
Thoughts: Okay, let's dive into this commit, `0d85f5c2dd91 btrfs: fix assertion failure when splitting ordered extent after transaction abort`. I've got a feeling this is going to be interesting, and hopefully, not too time-consuming. My immediate thought is, "btrfs, right, gotta be careful there."
**Fixing a Btrfs Assertion in a Tricky Situation**
First, I need to understand the failure. "Assertion failure" means something's gone wrong in the code's self-checking mechanism, likely a logical inconsistency. And the context - splitting an ordered extent *after* a transaction abort - suggests a delicate dance with the filesystem's journaling and concurrency. Transactions in btrfs are, obviously, how we manage a write that might not completely succeed. A split implies we're trying to divide a data block, likely due to fragmentation or allocation needs, and "ordered extent" is another btrfs specific internal data structure term. The fact that the problem crops up after an abort makes me suspect some sort of corruption in the transaction state or some issue with undoing partial operations. This could be a race condition, a subtle data structure inconsistency, or something related to how btrfs tracks metadata changes during the abort process. I'm already formulating some hypotheses around how `btrfs_preallocate`, `btrfs_end_extent_io` or similar functions might have an issue when handling the commit. Let's see what the commit description, and ideally the diff, actually say. This could be a good one.
|
| 283/2 |
2026/08/08 04:08 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 0d85f5c2dd91df6b5da454406756f463ba923b69
Author: Filipe Manana <fdmanana@suse.com>
Date: Mon Jan 13 15:01:08 2025 +0000
btrfs: fix assertion failure when splitting ordered extent after transaction abort
If while we are doing a direct IO write a transaction abort happens, we
mark all existing ordered extents with the BTRFS_ORDERED_IOERR flag (done
at btrfs_destroy_ordered_extents()), and then after that if we enter
btrfs_split_ordered_extent() and the ordered extent has bytes left
(meaning we have a bio that doesn't cover the whole ordered extent, see
details at btrfs_extract_ordered_extent()), we will fail on the following
assertion at btrfs_split_ordered_extent():
ASSERT(!(flags & ~BTRFS_ORDERED_TYPE_FLAGS));
because the BTRFS_ORDERED_IOERR flag is set and the definition of
BTRFS_ORDERED_TYPE_FLAGS is just the union of all flags that identify the
type of write (regular, nocow, prealloc, compressed, direct IO, encoded).
Fix this by returning an error from btrfs_extract_ordered_extent() if we
find the BTRFS_ORDERED_IOERR flag in the ordered extent. The error will
be the error that resulted in the transaction abort or -EIO if no
transaction abort happened.
This was recently reported by syzbot with the following trace:
FAULT_INJECTION: forcing a failure.
name failslab, interval 1, probability 0, space 0, times 1
CPU: 0 UID: 0 PID: 5321 Comm: syz.0.0 Not tainted 6.13.0-rc5-syzkaller #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
fail_dump lib/fault-inject.c:53 [inline]
should_fail_ex+0x3b0/0x4e0 lib/fault-inject.c:154
should_failslab+0xac/0x100 mm/failslab.c:46
slab_pre_alloc_hook mm/slub.c:4072 [inline]
slab_alloc_node mm/slub.c:4148 [inline]
__do_kmalloc_node mm/slub.c:4297 [inline]
__kmalloc_noprof+0xdd/0x4c0 mm/slub.c:4310
kmalloc_noprof include/linux/slab.h:905 [inline]
kzalloc_noprof include/linux/slab.h:1037 [inline]
btrfs_chunk_alloc_add_chunk_item+0x244/0x1100 fs/btrfs/volumes.c:5742
reserve_chunk_space+0x1ca/0x2c0 fs/btrfs/block-group.c:4292
check_system_chunk fs/btrfs/block-group.c:4319 [inline]
do_chunk_alloc fs/btrfs/block-group.c:3891 [inline]
btrfs_chunk_alloc+0x77b/0xf80 fs/btrfs/block-group.c:4187
find_free_extent_update_loop fs/btrfs/extent-tree.c:4166 [inline]
find_free_extent+0x42d1/0x5810 fs/btrfs/extent-tree.c:4579
btrfs_reserve_extent+0x422/0x810 fs/btrfs/extent-tree.c:4672
btrfs_new_extent_direct fs/btrfs/direct-io.c:186 [inline]
btrfs_get_blocks_direct_write+0x706/0xfa0 fs/btrfs/direct-io.c:321
btrfs_dio_iomap_begin+0xbb7/0x1180 fs/btrfs/direct-io.c:525
iomap_iter+0x697/0xf60 fs/iomap/iter.c:90
__iomap_dio_rw+0xeb9/0x25b0 fs/iomap/direct-io.c:702
btrfs_dio_write fs/btrfs/direct-io.c:775 [inline]
btrfs_direct_write+0x610/0xa30 fs/btrfs/direct-io.c:880
btrfs_do_write_iter+0x2a0/0x760 fs/btrfs/file.c:1397
do_iter_readv_writev+0x600/0x880
vfs_writev+0x376/0xba0 fs/read_write.c:1050
do_pwritev fs/read_write.c:1146 [inline]
__do_sys_pwritev2 fs/read_write.c:1204 [inline]
__se_sys_pwritev2+0x196/0x2b0 fs/read_write.c:1195
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1281f85d29
RSP: 002b:00007f12819fe038 EFLAGS: 00000246 ORIG_RAX: 0000000000000148
RAX: ffffffffffffffda RBX: 00007f1282176080 RCX: 00007f1281f85d29
RDX: 0000000000000001 RSI: 0000000020000240 RDI: 0000000000000005
RBP: 00007f12819fe090 R08: 0000000000000000 R09: 0000000000000003
R10: 0000000000007000 R11: 0000000000000246 R12: 0000000000000002
R13: 0000000000000000 R14: 00007f1282176080 R15: 00007ffcb9e23328
</TASK>
BTRFS error (device loop0 state A): Transaction aborted (error -12)
BTRFS: error (device loop0 state A) in btrfs_chunk_alloc_add_chunk_item:5745: errno=-12 Out of memory
BTRFS info (device loop0 state EA): forced readonly
assertion failed: !(flags & ~BTRFS_ORDERED_TYPE_FLAGS), in fs/btrfs/ordered-data.c:1234
------------[ cut here ]------------
kernel BUG at fs/btrfs/ordered-data.c:1234!
Oops: invalid opcode: 0000 [#1] PREEMPT SMP KASAN NOPTI
CPU: 0 UID: 0 PID: 5321 Comm: syz.0.0 Not tainted 6.13.0-rc5-syzkaller #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
RIP: 0010:btrfs_split_ordered_extent+0xd8d/0xe20 fs/btrfs/ordered-data.c:1234
RSP: 0018:ffffc9000d1df2b8 EFLAGS: 00010246
RAX: 0000000000000057 RBX: 000000000006a000 RCX: 9ce21886c4195300
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000000000
RBP: 0000000000000091 R08: ffffffff817f0a3c R09: 1ffff92001a3bdf4
R10: dffffc0000000000 R11: fffff52001a3bdf5 R12: 1ffff1100a45f401
R13: ffff8880522fa018 R14: dffffc0000000000 R15: 000000000006a000
FS: 00007f12819fe6c0(0000) GS:ffff88801fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000557750bd7da8 CR3: 00000000400ea000 CR4: 0000000000352ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
btrfs_extract_ordered_extent fs/btrfs/direct-io.c:702 [inline]
btrfs_dio_submit_io+0x4be/0x6d0 fs/btrfs/direct-io.c:737
iomap_dio_submit_bio fs/iomap/direct-io.c:85 [inline]
iomap_dio_bio_iter+0x1022/0x1740 fs/iomap/direct-io.c:447
__iomap_dio_rw+0x13b7/0x25b0 fs/iomap/direct-io.c:703
btrfs_dio_write fs/btrfs/direct-io.c:775 [inline]
btrfs_direct_write+0x610/0xa30 fs/btrfs/direct-io.c:880
btrfs_do_write_iter+0x2a0/0x760 fs/btrfs/file.c:1397
do_iter_readv_writev+0x600/0x880
vfs_writev+0x376/0xba0 fs/read_write.c:1050
do_pwritev fs/read_write.c:1146 [inline]
__do_sys_pwritev2 fs/read_write.c:1204 [inline]
__se_sys_pwritev2+0x196/0x2b0 fs/read_write.c:1195
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1281f85d29
RSP: 002b:00007f12819fe038 EFLAGS: 00000246 ORIG_RAX: 0000000000000148
RAX: ffffffffffffffda RBX: 00007f1282176080 RCX: 00007f1281f85d29
RDX: 0000000000000001 RSI: 0000000020000240 RDI: 0000000000000005
RBP: 00007f12819fe090 R08: 0000000000000000 R09: 0000000000000003
R10: 0000000000007000 R11: 0000000000000246 R12: 0000000000000002
R13: 0000000000000000 R14: 00007f1282176080 R15: 00007ffcb9e23328
</TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:btrfs_split_ordered_extent+0xd8d/0xe20 fs/btrfs/ordered-data.c:1234
RSP: 0018:ffffc9000d1df2b8 EFLAGS: 00010246
RAX: 0000000000000057 RBX: 000000000006a000 RCX: 9ce21886c4195300
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000000000
RBP: 0000000000000091 R08: ffffffff817f0a3c R09: 1ffff92001a3bdf4
R10: dffffc0000000000 R11: fffff52001a3bdf5 R12: 1ffff1100a45f401
R13: ffff8880522fa018 R14: dffffc0000000000 R15: 000000000006a000
FS: 00007f12819fe6c0(0000) GS:ffff88801fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000557750bd7da8 CR3: 00000000400ea000 CR4: 0000000000352ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
In this case the transaction abort was due to (an injected) memory
allocation failure when attempting to allocate a new chunk.
Reported-by: syzbot+f60d8337a5c8e8d92a77@syzkaller.appspotmail.com
Link: https://lore.kernel.org/linux-btrfs/6777f2dd.050a0220.178762.0045.GAE@google.com/
Fixes: 52b1fdca23ac ("btrfs: handle completed ordered extents in btrfs_split_ordered_extent")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 30eceaf829a7..4aca7475fd82 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -1229,6 +1229,18 @@ struct btrfs_ordered_extent *btrfs_split_ordered_extent(
*/
if (WARN_ON_ONCE(len >= ordered->num_bytes))
return ERR_PTR(-EINVAL);
+ /*
+ * If our ordered extent had an error there's no point in continuing.
+ * The error may have come from a transaction abort done either by this
+ * task or some other concurrent task, and the transaction abort path
+ * iterates over all existing ordered extents and sets the flag
+ * BTRFS_ORDERED_IOERR on them.
+ */
+ if (unlikely(flags & (1U << BTRFS_ORDERED_IOERR))) {
+ const int fs_error = BTRFS_FS_ERROR(fs_info);
+
+ return fs_error ? ERR_PTR(fs_error) : ERR_PTR(-EIO);
+ }
/* We cannot split partially completed ordered extents. */
if (ordered->bytes_left) {
ASSERT(!(flags & ~BTRFS_ORDERED_TYPE_FLAGS));
]
|
| 284/2 |
2026/08/08 04:08 |
llm |
history-explorer |
0m
Model:
Tokens:
input: 65306
output: 909
thoughts: 647
Thoughts: **My Assessment of the Proposed Fix**
Alright, I've got a handle on the situation. The core of this issue is pretty clear: the `find_free_dev_extent` function, by design, doesn't see space freed within the *current* uncommitted transaction, specifically those cleared by `btrfs_remove_dev_extents`. This is a crucial safety mechanism to prevent corruption if the transaction aborts. The `check_system_chunk` function, and its underlying `reserve_chunk_space`, is being... let's say "optimistic" and ignoring the potential `-ENOSPC` errors. It *hopes* a later tree modification will catch the real failure, which, in most cases is fine.
The existing fallback within `btrfs_remove_chunk` to create a new system chunk also fails for the exact same reason: `find_free_dev_extent` is blind to the freed space. We *know* that reserving metadata space *early*, before making irreversible changes, and then releasing it is a common idiom in BTRFS. We've seen it in `btrfs_finish_sprout` and `btrfs_rm_dev_item` for instance, but they hold the reservation through the entire operation. However, the proposed fix needs to reserve, immediately release, and *then* re-reserve. The reason for the unusual immediate release is that nested allocations triggered by `btrfs_remove_dev_extents` would have immediately cleared the reservation.
Now, the really interesting part, and where the fix gets *clever*, is the way it checks for `check_system_chunk` failure: by comparing `trans->chunk_bytes_reserved` before and after the call. I have to admit, *that's* a novel idiom! I haven't seen that precise technique used anywhere else in the BTRFS codebase. I'm used to checking the error return of the underlying functions or explicitly checking for space.
Ultimately, this aligns with the transactional safety goals of BTRFS by failing gracefully before actually modifying the device tree. The use of the `chunk_bytes_reserved` check is not standard. A more 'by-the-book' solution, would involve tweaking `check_system_chunk` or `reserve_chunk_space` to return proper error codes, or crafting an entirely new helper. However, given the constraints of not changing the signatures of these core, widely-used functions, this proposed early check-and-release pattern looks like a pragmatic and necessary workaround, even if it is a bit unique.
|
| 285/1 |
2026/08/08 04:08 |
action |
kernel-scratch-checkouter |
0m
Results: map[KernelScratchSrc:/app/workdir/cache/tmp/tmp1754874449]
|
| 286/1 |
2026/08/08 04:09 |
loop |
|
26m
|
| 287/2 |
2026/08/08 04:09 |
iteration |
0 |
26m
|
| 288/3 |
2026/08/08 04:09 |
agent |
patch-generator |
3m
Model:
Instruction:
You are an experienced Linux kernel developer tasked with creating a fix for a kernel bug.
You will be given a crash report, an initial explanation of the root cause done by another
kernel expert, and a summary of how similar bugs were fixed in the past.
Use the codeeditor tool to do code edits.
Note: you will not see your changes when looking at the code using codesearch tools.
Use the patch-diff tool to review the modifications you applied.
Your final reply should contain explanation of what you did in the patch and why
(details not present in the initial explanation of the bug).
Your fix must not just prevent the given crash, but also be the best fix for the underlying
root cause from the software engineering point of view. There can be several ways to fix the
same bug. Consider alternatives, and pick the best one. For example, additional checks may be
added at different locations/functions, it's usually better to place them earlier in the
execution to avoid multiple checks at various locations later.
Frequently the same coding mistake is done in several locations in the source code.
Check if your fix should be extended/applied to similar cases around to fix other similar bugs.
But don't go too wide, don't try to fix problems kernel-wide, fix similar issues
in the same file only.
If you are changing post-conditions of a function, consider all callers of the functions,
and if they need to be updated to handle new post-conditions. For example, if you make
a function that previously never returned a NULL, return NULL, consider if callers
need to be updated to handle NULL return value.
If you will end up removing the WARN_ON macro because the condition can legitimately happen,
add a pr_err/dev_err/... (whatever is the macro for printing runtime errors used in the file)
call that logs that the unlikely condition has happened. The pr_err/dev_err/... message
must not include "WARNING" nor "BUG" strings.
Prefer calling several tools at the same time to save round-trips.
Prompt:
The crash that corresponds to the bug is:
BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
The explanation of the root cause of the bug is:
An analysis of the crash reveals that it is caused by a transaction abort (`-ENOSPC`) in `btrfs_remove_chunk` during a chunk relocation/balance operation. The abort happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
### Root Cause
1. **Order of Operations**: During chunk removal, `btrfs_remove_chunk` first removes the device extents from the device tree by calling `btrfs_remove_dev_extents`.
2. **System Space Reservation**: It then calls `check_system_chunk` to reserve system space for removing the chunk item from the chunk tree. If the system space is full, `check_system_chunk` attempts to allocate a new system chunk.
3. **Commit Root Search**: To allocate a new chunk, `find_free_dev_extent` is called. However, `find_free_dev_extent` searches the *commit root* of the device tree to prevent reusing space freed in the current transaction (which could lead to corruption if the transaction aborts).
4. **Invisible Freed Space**: Because the device extents were just freed in the *current* transaction by `btrfs_remove_dev_extents`, `find_free_dev_extent` does not see this freed space. If the device is otherwise full, it returns `-ENOSPC`.
5. **Ignored Failure**: `check_system_chunk` ignores the failure to allocate a system chunk and does not reserve any space in `trans->chunk_bytes_reserved`.
6. **Transaction Abort**: `remove_chunk_item` is then called, which fails with `-ENOSPC` because it needs to COW a node in the chunk tree but no system space was reserved. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully (returning `-ENOSPC` would leave the filesystem in an inconsistent state with the chunk item present but device extents missing). Thus, it is forced to abort the transaction.
### The Fix
We must ensure that we have enough system space *before* we modify the device tree. However, we cannot simply move the `check_system_chunk` call before `btrfs_remove_dev_extents` and keep the reservation, because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations (`btrfs_trans_release_chunk_metadata`).
Instead, we can call `check_system_chunk` early to *ensure* the system `space_info` has enough free space (allocating a system chunk if necessary), and then immediately release the `chunk_block_rsv` reservation. If it fails to allocate a system chunk, we can fail gracefully by returning `-ENOSPC` before any trees are modified. Then, after `btrfs_remove_dev_extents`, we call `check_system_chunk` again to make the actual reservation for `remove_chunk_item`.
Here is the proposed fix for `fs/btrfs/volumes.c` in `btrfs_remove_chunk`:
```c
map = em->map_lookup;
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
/*
* Take the device list mutex to prevent races with the final phase of
* a device replace operation...
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
// ... existing btrfs_remove_dev_extents logic ...
}
mutex_unlock(&fs_devices->device_list_mutex);
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
### Why this works:
1. **Graceful Failure**: By checking `trans->chunk_bytes_reserved`, we can detect if `check_system_chunk` failed to allocate a system chunk. If it failed, we return `-ENOSPC` *before* any trees are modified, allowing the balance operation to fail cleanly instead of crashing the kernel.
2. **No Metadata Leaks**: We immediately release the early reservation using `btrfs_trans_release_chunk_metadata(trans)`.
3. **Safe from Nested Allocations**: Because we release the reservation, we don't care if `btrfs_remove_dev_extents` triggers a nested chunk allocation that clears `trans->chunk_bytes_reserved`.
4. **Guaranteed Space**: The early `check_system_chunk` ensures the system `space_info` has enough free space (by allocating a 32MB system chunk if needed). When we call `check_system_chunk` the second time, it will see the free space in the `space_info` and successfully reserve the few KB needed for `remove_chunk_item`.
Historical context on how similar bugs were fixed in the past:
Based on a thorough review of the BTRFS codebase and its commit history over the past several years, here is a summary of the findings regarding the idioms, locking rules, and common patterns relevant to this bug and the proposed fix:
### 1. The Root Cause Context (Commit Root Searches)
The root cause you identified—that `find_free_dev_extent` searches the commit root and therefore cannot see space freed in the current transaction—is a known and intentional BTRFS design choice. This prevents the filesystem from reusing space freed in an uncommitted transaction, which could lead to severe corruption if the transaction were to abort.
### 2. `check_system_chunk` Error Handling
The `check_system_chunk` function (and its underlying `reserve_chunk_space`) is designed to return `void` and intentionally ignores failures to allocate a system chunk. The historical assumption (documented in `reserve_chunk_space`) is that if it fails, the caller might not actually need the space (e.g., if the required nodes were already COWed). If the space is truly needed, the subsequent tree modification (like `remove_chunk_item`) will fail with `-ENOSPC`.
`btrfs_remove_chunk` already attempts to handle this `-ENOSPC` gracefully by falling back to `btrfs_create_chunk` (introduced to handle cases where system space is fragmented or profiles are incompatible). However, as you noted, this fallback also fails because `btrfs_remove_dev_extents` has already modified the device tree in the current transaction.
### 3. The "Early Reserve and Release" Pattern
Reserving metadata space early (before making irreversible tree changes) to ensure it is available, and then releasing it, is a known pattern in BTRFS. For example, in `fs/btrfs/volumes.c` (`btrfs_finish_sprout` and `btrfs_rm_dev_item`), the code calls `btrfs_reserve_chunk_metadata()`, performs a tree search/modification, and then calls `btrfs_trans_release_chunk_metadata()`.
However, the proposed fix's approach of reserving, **immediately releasing**, and then re-reserving later is unique. It is a necessary adaptation here because `btrfs_remove_dev_extents` can trigger nested allocations that unconditionally clear the reservation (via `btrfs_trans_release_chunk_metadata`).
### 4. Checking `trans->chunk_bytes_reserved` (Novel Idiom)
The proposed method of detecting whether `check_system_chunk` failed by comparing `trans->chunk_bytes_reserved` before and after the call is a **novel, non-standard idiom**.
A search through the BTRFS codebase reveals no prior art that checks `chunk_bytes_reserved` in this manner to infer the success of `check_system_chunk`. In BTRFS, space reservation failures are almost universally handled by checking a returned integer error code (e.g., `ret = btrfs_chunk_alloc(...)`).
### Conclusion & Recommendations
**No exact past fixes were found that use this specific `chunk_bytes_reserved` check pattern.**
While the proposed fix correctly identifies the transactional deadlock and successfully works around it by failing gracefully before modifying the device tree, the way it detects the failure is non-idiomatic for BTRFS.
If you are to submit a fix based on historical BTRFS patterns, you have two main paths:
1. **The Pragmatic Workaround (Your Proposal):** Proceed with the `chunk_bytes_reserved` check, but heavily comment it. Acknowledge that it is a hack to work around the fact that `check_system_chunk` returns `void` and ignores errors, and that it is necessary to prevent a transaction abort.
2. **The Idiomatic Approach:** A more standard (though slightly more invasive) approach would be to introduce a variant of `check_system_chunk` or `reserve_chunk_space` that actually returns an `int` error code if it fails to allocate a system chunk, allowing `btrfs_remove_chunk` to check the return value directly rather than inferring it from transaction state variables.
Reply: I have implemented a fix for the transaction abort issue in `btrfs_remove_chunk`.
### What I did in the patch:
1. **Changed `check_system_chunk` and `reserve_chunk_space` to return `int`:** Instead of returning `void` and ignoring errors, these functions now return an `int` error code if they fail to allocate a system chunk. This is the standard, idiomatic way to handle errors in BTRFS, avoiding the need for hacky workarounds like checking `trans->chunk_bytes_reserved` before and after the call.
2. **Early System Space Check:** In `btrfs_remove_chunk`, I added an early call to `check_system_chunk` *before* `btrfs_remove_dev_extents` modifies the device tree. We check the return value directly.
3. **Graceful Failure:** If the early `check_system_chunk` fails (e.g., returns `-ENOSPC`), we can now fail gracefully and return the error before any irreversible tree modifications have occurred, preventing the transaction abort.
4. **Immediate Reservation Release:** If the early check succeeds, we immediately release the reserved metadata space using `btrfs_trans_release_chunk_metadata(trans)`. This is crucial because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations. We then proceed to call `check_system_chunk` a second time later in the function to make the actual reservation, which is now guaranteed to succeed since we ensured the system `space_info` has enough free space.
### Why this is the best approach:
- **Idiomatic Error Handling:** By modifying `check_system_chunk` to return an error code, we align with BTRFS's standard error handling practices. Other callers of `check_system_chunk` (like `btrfs_inc_block_group_ro` and `do_chunk_alloc`) can safely continue to ignore the return value, as they don't require the same strict early-failure guarantees.
- **Prevents Transaction Aborts:** The root cause of the crash was that `btrfs_remove_chunk` was forced to abort the transaction because it couldn't fail gracefully after modifying the device tree. By checking for space *before* modifying the tree, we allow the balance/relocation operation to fail cleanly with `-ENOSPC`.
- **Safe from Nested Allocations:** The "reserve and immediately release" pattern perfectly works around the fact that `btrfs_remove_dev_extents` can clear our reservations. We use the first call purely to guarantee that the system `space_info` has enough capacity (allocating a new system chunk if necessary), and the second call to make the actual reservation for the chunk tree modification.
|
| 289/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 8396
output: 38
thoughts: 393
Thoughts: **Analyzing a Btrfs Chunk Removal Bug**
Okay, so I've got a tricky Btrfs bug to wrestle with. It seems that during `btrfs_remove_chunk`, a transaction is aborting with `-ENOSPC`, which, in this context, is because we are running out of system space when trying to remove a chunk item. The root cause is that our current approach in `btrfs_remove_chunk` is... well, it's problematic. We're removing device extents *before* calling `check_system_chunk` to reserve system space. The transaction fails because `check_system_chunk`, when system space is scarce, tries to allocate a new system chunk.
The issue arises because `check_system_chunk` calls `find_free_dev_extent`, which searches the commit root of the device tree. Unfortunately, because we just freed device extents *in the current transaction*, `find_free_dev_extent` can't see the newly freed space. Consequently, if the device is otherwise full, it returns `-ENOSPC`. This is where the problematic logic exists, as `check_system_chunk` proceeds to ignore the `-ENOSPC` error, failing to reserve system space. Then, `remove_chunk_item` fails with `-ENOSPC` and the transaction aborts because the device tree has already been modified. We are attempting to modify an item, but the space to do so does not exist.
The suggested fix is pretty straightforward: let's call `check_system_chunk` *earlier*, before `btrfs_remove_dev_extents`, to ensure the system space is actually available. If `check_system_chunk` fails, we can gracefully exit and the transaction would be able to roll back. However, we can't keep the reservation if the check passes. This is because nested chunk allocations, triggered by `btrfs_remove_dev_extents`, can release *all* chunk metadata reservations. So, we'll need to release the reservation immediately after we've checked.
The historical context is also giving me pause, particularly concerning `trans->chunk_bytes_reserved`. This seems to be a non-standard way to infer success from `check_system_chunk`. I'm going to need to heavily comment that code or, better yet, consider introducing a variant of `check_system_chunk` that returns a proper error code (`int`). I always favor explicit error handling.
Right, time to dive into `fs/btrfs/volumes.c` and take a closer look at `check_system_chunk` itself. Let's see what we can find.
|
| 290/4 |
2026/08/08 04:09 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 291/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 9719
output: 40
thoughts: 11
Thoughts: Okay, here's my attempt at summarizing that thought process, keeping in mind I'm speaking to a fellow expert:
**Investigating `reserve_chunk_space`**
Alright, time to dive into this. I've got to understand how we're handling memory allocation within this specific chunking mechanism. My first thought is to take a close look at the `reserve_chunk_space` function. This feels like the most likely entry point to understanding how we're pre-allocating or dynamically requesting the memory that the chunk utilizes. I want to check for any potential gotchas or edge cases right away, and see how memory is reserved here.
|
| 292/4 |
2026/08/08 04:09 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 293/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 11172
output: 53
thoughts: 136
Thoughts: **Analyzing Chunk Reservation and Error Handling**
Okay, so I'm looking at this `reserve_chunk_space` function, and something's bugging me. It seems like it's silently ignoring errors coming back from `btrfs_create_chunk` and `btrfs_zoned_activate_one_bg`. The really concerning part is that if either of those return a non-zero value, indicating an error, it *doesn't* update `trans->chunk_bytes_reserved`. Essentially, we're not tracking the failure to allocate space correctly.
Wait a second, let me clarify: if there's an error (`ret != 0`), it skips the crucial `btrfs_block_rsv_add` call and *doesn't* increment `trans->chunk_bytes_reserved`. That's a definite problem. It sounds like the proposed solution is to make `check_system_chunk` return an integer error code, which makes perfect sense in light of this. I need to understand where `check_system_chunk` is used in the overall flow, to know how far reaching the change might be. Let's see...
|
| 294/4 |
2026/08/08 04:09 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:0 Name:check_system_chunk SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_inc_block_group_ro SourceFile:fs/btrfs/block-group.c SourceLine:3234] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3494] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:do_chunk_alloc SourceFile:fs/btrfs/block-group.c SourceLine:4141]] TruncatedOutput:false]
|
| 295/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 11797
output: 40
thoughts: 50
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Analyzing Block Group Management in Btrfs**
Alright, so I see this construct cropping up in a few key areas within the Btrfs codebase: `btrfs_inc_block_group_ro`, `btrfs_remove_chunk`, and `do_chunk_alloc`. My immediate inclination is to drill down into `btrfs_remove_chunk`. That function, in particular, always sparks my interest. It represents a significant point where allocation and data placement are impacted. I want to see what assumptions it makes, and how it handles the interaction between metadata and data blocks. I’m thinking about how the block group structures are manipulated and whether the increment being mentioned hints at reference counting, locking, or some other form of resource management. Understanding this particular function is probably critical to understanding the bigger picture around block group changes.
|
| 296/4 |
2026/08/08 04:09 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_chunk]
Results: map[SourceCode:3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
3505: * check_system_space() above, a scrub turned the only system block group
3506: * with enough free space into RO mode.
3507: * This is explained with more detail at do_chunk_alloc().
3508: *
3509: * So if we get -ENOSPC, allocate a new system chunk and retry once.
3510: */
3511: if (ret == -ENOSPC) {
3512: const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
3513: struct btrfs_block_group *sys_bg;
3514: struct btrfs_space_info *space_info;
3515:
3516: space_info = btrfs_find_space_info(fs_info, sys_flags);
3517: if (unlikely(!space_info)) {
3518: ret = -EINVAL;
3519: btrfs_abort_transaction(trans, ret);
3520: goto out;
3521: }
3522:
3523: sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
3524: if (IS_ERR(sys_bg)) {
3525: ret = PTR_ERR(sys_bg);
3526: btrfs_abort_transaction(trans, ret);
3527: goto out;
3528: }
3529:
3530: ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
3531: if (unlikely(ret)) {
3532: btrfs_abort_transaction(trans, ret);
3533: goto out;
3534: }
3535:
3536: ret = remove_chunk_item(trans, map, chunk_offset);
3537: if (unlikely(ret)) {
3538: btrfs_abort_transaction(trans, ret);
3539: goto out;
3540: }
3541: } else if (unlikely(ret)) {
3542: btrfs_abort_transaction(trans, ret);
3543: goto out;
3544: }
3545:
3546: trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
3547:
3548: if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
3549: ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
3550: if (unlikely(ret)) {
3551: btrfs_abort_transaction(trans, ret);
3552: goto out;
3553: }
3554: }
3555:
3556: btrfs_update_per_profile_avail(fs_info);
3557: mutex_unlock(&fs_info->chunk_mutex);
3558: trans->removing_chunk = false;
3559:
3560: /*
3561: * We are done with chunk btree updates and deletions, so release the
3562: * system space we previously reserved (with check_system_chunk()).
3563: */
3564: btrfs_trans_release_chunk_metadata(trans);
3565:
3566: /* On error, btrfs_remove_block_group() aborts the transaction. */
3567: ret = btrfs_remove_block_group(trans, map);
3568: if (unlikely(ret))
3569: ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
3570:
3571: out:
3572: if (trans->removing_chunk) {
3573: mutex_unlock(&fs_info->chunk_mutex);
3574: trans->removing_chunk = false;
3575: }
3576: /* once for us */
3577: btrfs_free_chunk_map(map);
3578: return ret;
3579: }
SourceFile:fs/btrfs/volumes.c]
|
| 297/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 14178
output: 40
thoughts: 43
Thoughts: Okay, let's break this down.
**Evaluating a Proposed Change**
The idea is to modify the function `check_system_chunk` so that instead of its current return type, it returns an integer representing an error code. I need to consider the implications of that shift. What are the benefits of using an integer return type for error signaling in this specific context? What are the potential drawbacks? How would this impact the existing codebase? Are there any potential backwards compatibility issues to address? I'll need to analyze the surrounding code to determine how `check_system_chunk` is currently used and assess the necessary modifications. Ultimately, I'll need to decide if this proposed change is a net positive for the project.
|
| 298/4 |
2026/08/08 04:09 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 299/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 14733
output: 40
thoughts: 46
Thoughts: Okay, let's break this down.
**Refactoring the Chunk Allocation System**
Right, so I'm looking at this chunk allocation system, and it seems like we're changing the return type of `check_system_chunk` to `int`. If we do that – and it sounds like a good move, potentially for performance or to represent specific error codes in a clearer way – then we *absolutely* need to also modify the return type of `reserve_chunk_space`. It's a cascading dependency, and consistency is key here. Otherwise, the interface breaks, and the whole thing goes haywire. I need to make sure that these functions have related return types, so that it will continue working. Okay, time to double-check `reserve_chunk_space` itself to make sure that everything's going to work.
|
| 300/4 |
2026/08/08 04:09 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:reserve_chunk_space]
Results: map[SourceCode:4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
SourceFile:fs/btrfs/block-group.c]
|
| 301/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 16261
output: 53
thoughts: 387
Thoughts: **Reflecting on `check_system_chunk` Return Value**
Okay, so I'm looking at this `check_system_chunk` function, and the idea is to change it to return an integer. If we do that, we could simplify things and just return `ret` directly from `reserve_chunk_space`. Seems cleaner, right? But hold on... the comment in the code immediately brings up a potential issue. It says to ignore failures in creating the system chunk because we might not even *need* it, especially if the data's already been COW'd. Returning the `ret` directly from `check_system_chunk` means callers could fail prematurely, even when they didn't actually require the chunk. That's not ideal.
Ah, but wait, `btrfs_remove_chunk` is the real place where a failure to allocate system chunks is critical, right? That needs to happen *early* if we're going to fail at all. So, if we return `ret` from `check_system_chunk`, we *can* check it within `btrfs_remove_chunk`.
However, the question is, even if we return the `ret`, we still might fail even if we didn't need the space! On the other hand, `btrfs_remove_chunk` will subsequently call `btrfs_remove_dev_extents`, which is going to *modify* the device tree. Failing gracefully up front, even if it's a bit conservative, is better than potentially aborting the transaction later down the line. That's a huge risk! We're better off failing early if we might need it, than risking corruption later.
So, maybe we don't change it at all; maybe we add a new function such as `btrfs_check_system_chunk_early` or something, or just change `check_system_chunk` to return an `int` and check it in `btrfs_remove_chunk`? I need to analyze the current callers of `check_system_chunk`.
|
| 302/4 |
2026/08/08 04:09 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/block-group.c IncludeSnippetLines:1 Name:check_system_chunk SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_inc_block_group_ro SourceFile:fs/btrfs/block-group.c SourceLine:3234 SourceSnippet:3233: mutex_lock(&fs_info->chunk_mutex);
3234: check_system_chunk(trans, alloc_flags);
3235: mutex_unlock(&fs_info->chunk_mutex);
] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3494 SourceSnippet:3493:
3494: check_system_chunk(trans, map->type);
3495:
] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:do_chunk_alloc SourceFile:fs/btrfs/block-group.c SourceLine:4141 SourceSnippet:4140: */
4141: check_system_chunk(trans, flags);
4142:
]] TruncatedOutput:false]
|
| 303/4 |
2026/08/08 04:09 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 17472
output: 1685
thoughts: 461
Thoughts: **Refactoring Btrfs Chunk Allocation Error Handling**
Okay, so I'm looking at this Btrfs code, and I've noticed a recurring pattern of ignoring return values from functions that are critical to chunk allocation. Specifically, `btrfs_inc_block_group_ro` and `do_chunk_alloc` are both guilty of this. This is a potential source of problems if those functions fail silently.
The initial instinct was to change `check_system_chunk` to return an `int`. That way, we could handle the error directly in the two current call sites and check the result more thoroughly in `btrfs_remove_chunk`.
However, the proposed fix shown in the problem description is a bit different. It's essentially checking for failure by comparing `trans->chunk_bytes_reserved` before and after the call to `check_system_chunk`. This is... well, it's a novel approach, to put it mildly. It's not the most conventional or easily understood way of handling errors, and it introduces this sort of indirect check. The historical context confirms that this is not standard practice.
Thinking more about it, I'm leaning toward a more direct approach. Let's make `reserve_chunk_space` and `check_system_chunk` return an `int` error code. This way, we get that explicit error reporting that will provide clear indications of failure. It's more standard, cleaner, and ultimately, easier to reason about. Let's start with that approach.
|
| 304/4 |
2026/08/08 04:10 |
tool |
codeeditor |
0m
Args: map[CurrentCode:static void reserve_chunk_space(struct btrfs_trans_handle *trans,
u64 bytes,
u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
int ret = 0;
/*
* Needed because we can end up allocating a system chunk and for an
* atomic and race free space reservation in the chunk block reserve.
*/
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
left, bytes, type);
btrfs_dump_space_info(info, 0, false);
}
if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
*/
bg = btrfs_create_chunk(trans, space_info, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
} else {
int activate_ret;
/*
* We have a new chunk. We also need to activate it for
* zoned filesystem.
*/
activate_ret = btrfs_zoned_activate_one_bg(info, true);
if (activate_ret < 0) {
ret = activate_ret;
} else {
/*
* If we fail to add the chunk item here, we end
* up trying again at phase 2 of chunk allocation,
* at btrfs_create_pending_block_groups(). So
* ignore any error here. An ENOSPC here could
* happen, due to the cases described at
* do_chunk_alloc() - the system block group we
* just created was just turned into RO mode by a
* scrub for example, or a running discard
* temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
}
}
if (!ret) {
ret = btrfs_block_rsv_add(fs_info,
&fs_info->chunk_block_rsv,
bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
} NewCode:static int reserve_chunk_space(struct btrfs_trans_handle *trans,
u64 bytes,
u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
int ret = 0;
/*
* Needed because we can end up allocating a system chunk and for an
* atomic and race free space reservation in the chunk block reserve.
*/
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
left, bytes, type);
btrfs_dump_space_info(info, 0, false);
}
if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
*/
bg = btrfs_create_chunk(trans, space_info, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
} else {
int activate_ret;
/*
* We have a new chunk. We also need to activate it for
* zoned filesystem.
*/
activate_ret = btrfs_zoned_activate_one_bg(info, true);
if (activate_ret < 0) {
ret = activate_ret;
} else {
/*
* If we fail to add the chunk item here, we end
* up trying again at phase 2 of chunk allocation,
* at btrfs_create_pending_block_groups(). So
* ignore any error here. An ENOSPC here could
* happen, due to the cases described at
* do_chunk_alloc() - the system block group we
* just created was just turned into RO mode by a
* scrub for example, or a running discard
* temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
}
}
if (!ret) {
ret = btrfs_block_rsv_add(fs_info,
&fs_info->chunk_block_rsv,
bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
return ret;
} SourceFile:fs/btrfs/block-group.c]
Results:
|
| 305/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 19949
output: 431
thoughts: 26
Thoughts: Here's my take on the changes we're making:
**Refactoring `check_system_chunk` in `fs/btrfs/block-group.c`**
Alright, time to dive back into `fs/btrfs/block-group.c` and specifically focus on that `check_system_chunk` function. I remember we were discussing potential optimizations or perhaps addressing some edge cases there. Let's see... what's the goal here? Is it purely performance related, or are we shoring up some error handling, maybe even preventing potential data corruption scenarios? I need to review the surrounding code to understand the specific context. I'll need to meticulously analyze how `check_system_chunk` interacts with the chunk allocation and system metadata structures. I'll be looking for opportunities to simplify the logic, potentially reduce redundant checks (if any), and ensure that the code is robust and efficient. I need to make sure that whatever changes I make don't introduce any regressions or unforeseen side effects, so I will have to be extra diligent in my testing. I'll need to think about potential performance trade-offs too; what's more important - speed or safety in this specific part of the code?
|
| 306/4 |
2026/08/08 04:10 |
tool |
codeeditor |
0m
Args: map[CurrentCode:/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
u64 bytes;
/* num_devs device items to update and 1 chunk item to add or remove. */
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
reserve_chunk_space(trans, bytes, type);
} NewCode:/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
u64 bytes;
/* num_devs device items to update and 1 chunk item to add or remove. */
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
return reserve_chunk_space(trans, bytes, type);
} SourceFile:fs/btrfs/block-group.c]
Results:
|
| 307/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 20671
output: 40
thoughts: 21
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience:
**Refactoring the Btrfs Block Group Prototype**
Alright, time to get down to brass tacks. I'm looking at the `fs/btrfs/block-group.h` file. We need to make a fundamental change to the block group prototype. I know this particular area well, and the impact of the modification needs to be carefully considered. It's crucial we maintain compatibility where possible. But the refactor is necessary. I need to make sure this change is properly thought out. Let's start with a review of existing Btrfs block group structures.
|
| 308/4 |
2026/08/08 04:10 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 309/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 21194
output: 19
thoughts: 15
Thoughts: Here's my thought process, as if I were considering this myself:
**Tracing the Declaration of `check_system_chunk`**
Okay, first things first, I need to pinpoint the declaration of this `check_system_chunk` function. It's crucial for understanding how and where it's being used. I need to know its signature: what arguments it takes, and what it returns. That'll give me a strong foundation for the analysis. Knowing the declaration context (header files, modules, etc.) will also help identify dependencies and the overall architecture. I'll probably start with a grep, a find, or my IDE's "find declaration" feature. If this is a codebase I'm familiar with, I'll have a good idea of where to look based on project structure. If not, a broad search might be the best first step. Gotta track it down so I can continue the investigation.
|
| 310/4 |
2026/08/08 04:10 |
tool |
grepper |
0m
Args: map[Expression:check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=1383=struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
--
fs/btrfs/block-group.c-1415- * more device items and remove one chunk item), but this is done at
fs/btrfs/block-group.c:1416: * btrfs_remove_chunk() through a call to check_system_chunk().
fs/btrfs/block-group.c-1417- */
--
fs/btrfs/block-group.c=3125=int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
--
fs/btrfs/block-group.c-3233- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/block-group.c:3234: check_system_chunk(trans, alloc_flags);
fs/btrfs/block-group.c-3235- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4140- */
fs/btrfs/block-group.c:4141: check_system_chunk(trans, flags);
fs/btrfs/block-group.c-4142-
--
fs/btrfs/block-group.c-4165- * single device, we are forced to allocate a new system chunk with a
fs/btrfs/block-group.c:4166: * SINGLE profile. Making check_system_chunk() iterate over all system
fs/btrfs/block-group.c-4167- * block groups and check if they have a usable profile and enough space
--
fs/btrfs/block-group.c-4173- * 2) We had enough free space info the system space_info, and one suitable
fs/btrfs/block-group.c:4174: * block group to allocate from when we called check_system_chunk()
fs/btrfs/block-group.c-4175- * above. However right after we called it, the only system block group
--
fs/btrfs/block-group.c-4181- * 3) We had one system block group with enough free space when we called
fs/btrfs/block-group.c:4182: * check_system_chunk(), but after that, right before we tried to
fs/btrfs/block-group.c-4183- * allocate the last extent buffer we needed, a discard operation came
--
fs/btrfs/block-group.c-4309- * needs to update the chunk btree (the only btree that uses system chunks), must
fs/btrfs/block-group.c:4310: * preallocate chunk space by calling either check_system_chunk() or
fs/btrfs/block-group.c-4311- * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
--
fs/btrfs/block-group.c-4316- *
fs/btrfs/block-group.c:4317: * The reservation of system space, done through check_system_chunk(), as well
fs/btrfs/block-group.c-4318- * as all the updates and insertions into the chunk btree must be done while
--
fs/btrfs/block-group.c=4341=int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4375- * before updating the chunk btree, by calling either
fs/btrfs/block-group.c:4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
fs/btrfs/block-group.c-4377- * It's possible that after a task reserves the space, it still ends up
--
fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3493-
fs/btrfs/volumes.c:3494: check_system_chunk(trans, map->type);
fs/btrfs/volumes.c-3495-
--
fs/btrfs/volumes.c-3498- * Normally we should not get -ENOSPC since we reserved space before
fs/btrfs/volumes.c:3499: * through the call to check_system_chunk().
fs/btrfs/volumes.c-3500- *
--
fs/btrfs/volumes.c-3561- * We are done with chunk btree updates and deletions, so release the
fs/btrfs/volumes.c:3562: * system space we previously reserved (with check_system_chunk()).
fs/btrfs/volumes.c-3563- */
]
|
| 311/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 23009
output: 188
thoughts: 19
Thoughts: Okay, here's my summary, tailored for an expert like you:
**Diving into `fs/btrfs/block-group.h`**
Alright, the objective here is clear: I need to make some modifications within the `fs/btrfs/block-group.h` file. This means I'm likely dealing with fundamental data structures and definitions related to block groups within the Btrfs filesystem. This file is crucial for how Btrfs organizes its storage, handles allocation, and manages the on-disk format at the block group level. Thinking about what needs changing will depend on the problem I'm trying to solve. Am I optimizing allocation, modifying metadata structures, adding new block group types, addressing a bug, or maybe making changes for new features? Knowing the context of this change is key, of course. Whatever the reason, I know this file requires careful consideration, as even minor adjustments here can have widespread implications on the filesystem's stability and performance. I need to be sure I understand the current structures inside and out, consider all potential side effects, and make sure that any changes are thoroughly tested. Let's get to it!
|
| 312/4 |
2026/08/08 04:10 |
tool |
codeeditor |
0m
Args: map[CurrentCode:int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans, NewCode:int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans, SourceFile:fs/btrfs/block-group.h]
Results:
|
| 313/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 23466
output: 477
thoughts: 26
Thoughts: Here's my take on those thoughts, tailored for an expert audience:
**Diving into `btrfs_remove_chunk`**
Okay, time to get my hands dirty with `btrfs_remove_chunk` in `fs/btrfs/volumes.c`. This is where the rubber meets the road when dealing with chunk removal, and I'm interested to see how it's handled. I need to refresh my memory on the interactions here, especially around the logic dealing with data extents and metadata, and how the chunk's removal impacts the overall block group allocation. I'm going to want to focus on the error handling around removing the chunks, making sure that it deals with potential issues gracefully. Also I need to double-check that this is thread-safe and can handle concurrent operations correctly. I think I'll start by tracing the function's execution flow and outlining its key responsibilities. I'll need to pay close attention to the lock acquisition and release strategy; race conditions in this area can lead to major data corruption, so this requires a meticulous review. I'll also want to see the chunk removal's impact on any RAID configurations and if this is handled correctly. Let's see what's actually going on here.
|
| 314/4 |
2026/08/08 04:10 |
tool |
codeeditor |
0m
Args: map[CurrentCode: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
if (IS_ERR(map)) {
DEBUG_WARN("errr %ld reading chunk map at offset %llu",
PTR_ERR(map), chunk_offset);
return PTR_ERR(map);
}
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
NewCode: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
if (IS_ERR(map)) {
DEBUG_WARN("errr %ld reading chunk map at offset %llu",
PTR_ERR(map), chunk_offset);
return PTR_ERR(map);
}
/*
* Check if we have enough system space to remove the chunk.
* If we don't, and we can't allocate a new system chunk, we fail gracefully
* before modifying the device tree.
*/
mutex_lock(&fs_info->chunk_mutex);
ret = check_system_chunk(trans, map->type);
if (ret) {
mutex_unlock(&fs_info->chunk_mutex);
goto out;
}
/*
* Release the block reserve. We just wanted to ensure the system
* space_info has enough free space. We will reserve it again later.
*/
btrfs_trans_release_chunk_metadata(trans);
mutex_unlock(&fs_info->chunk_mutex);
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
SourceFile:fs/btrfs/volumes.c]
Results:
|
| 315/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 24241
output: 15
thoughts: 2058
Thoughts: **Analyzing System Chunk Allocation and Error Handling**
Okay, so I've been digging into how system chunks are handled during operations like balancing in Btrfs, specifically focusing on error conditions. The core issue is this: `check_system_chunk` is supposed to ensure we have enough space for system chunks. If `reserve_chunk_space` fails to allocate a system chunk, it returns an error. This can be due to various reasons, like a failed chunk creation (`btrfs_create_chunk`), or an issue during zoned filesystem activation (`btrfs_zoned_activate_one_bg`).
The problem I'm seeing is that `check_system_chunk` *ignores* this return value! This is because it might not *actually* need the space right away. It's possible the nodes or leafs it was going to COW were already updated or created in the transaction, rendering the system chunk unnecessary at that time. However, this creates a potential for a delayed failure down the line.
If `check_system_chunk` fails *and* the caller proceeds to, say, `btrfs_remove_chunk` and *that* fails early, we *could* have failed a balance that was otherwise okay. However, if the operation proceeds as though the `check_system_chunk` had succeeded, and it *does* need the space later, we'd end up aborting the *entire* transaction, which is far worse. So, failing early is the right approach.
I confirmed that `check_system_chunk` is called in `btrfs_inc_block_group_ro` and `do_chunk_alloc` where the return value is ignored. This is fine. So, I plan to change `check_system_chunk` to return an integer and check it in the callers where the failure is potentially serious (e.g., in `btrfs_remove_chunk`).
Now, I was thinking about a scenario where `check_system_chunk` returns success, but we *still* get `-ENOSPC` later when `remove_chunk_item` is called. It turns out `btrfs_remove_chunk` already has a fallback. It tries to allocate a new system chunk if it gets `-ENOSPC` from `remove_chunk_item`. However, this fallback mechanism also uses `btrfs_create_chunk`, which ultimately calls `find_free_dev_extent`, which searches the commit root.
So here's the kicker: if `remove_chunk_item` fails with `-ENOSPC` because we *didn't* have system space, the fallback *also* fails with `-ENOSPC` if the device is full and the only free space was freed in the current transaction. This is where it aborts the transaction!
The crux of the issue is this: If `check_system_chunk` fails to allocate system chunk, and that error is ignored, and `remove_chunk_item` fails, the *fallback* would also fail and trigger an abort. With my change to fail `check_system_chunk`, the system will fail *early* before we get to the fallback mechanism.
It is possible that even if `check_system_chunk` returns success, `remove_chunk_item` could still fail later with `-ENOSPC` due to an incompatible profile or other concurrent factors, such as a running scrub. The fallback could trigger, but might fail. This is possible *even without* my change, because `check_system_chunk` only checks `left < bytes` on `BTRFS_BLOCK_GROUP_SYSTEM` space info and *doesn't* check profiles. This has been a pre-existing issue.
So by making `check_system_chunk` return the error, and checking for it right away, we prevent a more severe outcome in the `btrfs_remove_chunk` process: We will fail early and gracefully.
|
| 316/4 |
2026/08/08 04:10 |
tool |
patch-diff |
0m
Args:
Results: map[Output:diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4481,122 +4481,124 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
{
u64 num_dev;
num_dev = btrfs_raid_array[btrfs_bg_flags_to_raid_index(type)].devs_max;
if (!num_dev)
num_dev = fs_info->fs_devices->rw_devices;
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
int ret = 0;
/*
* Needed because we can end up allocating a system chunk and for an
* atomic and race free space reservation in the chunk block reserve.
*/
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
left, bytes, type);
btrfs_dump_space_info(info, 0, false);
}
if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
*/
bg = btrfs_create_chunk(trans, space_info, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
} else {
int activate_ret;
/*
* We have a new chunk. We also need to activate it for
* zoned filesystem.
*/
activate_ret = btrfs_zoned_activate_one_bg(info, true);
if (activate_ret < 0) {
ret = activate_ret;
} else {
/*
* If we fail to add the chunk item here, we end
* up trying again at phase 2 of chunk allocation,
* at btrfs_create_pending_block_groups(). So
* ignore any error here. An ENOSPC here could
* happen, due to the cases described at
* do_chunk_alloc() - the system block group we
* just created was just turned into RO mode by a
* scrub for example, or a running discard
* temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
}
}
if (!ret) {
ret = btrfs_block_rsv_add(fs_info,
&fs_info->chunk_block_rsv,
bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
u64 bytes;
/* num_devs device items to update and 1 chunk item to add or remove. */
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
* Reserve space in the system space, if needed, for doing a modification to the
* chunk btree.
*
* @trans: A transaction handle.
* @is_item_insertion: Indicate if the modification is for inserting a new item
* in the chunk btree or if it's for the deletion or update
* of an existing item.
*
* This is used in a context where we need to update the chunk btree outside
* block group allocation and removal, to avoid a deadlock with a concurrent
* task that is allocating a metadata or data block group and therefore needs to
* update the chunk btree while holding the chunk mutex. After the update to the
* chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
*
*/
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -371,21 +371,21 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes, bool alloc);
int btrfs_add_reserved_bytes(struct btrfs_block_group *cache,
u64 ram_bytes, u64 num_bytes, bool delalloc,
bool force_wrong_size_class);
void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, u64 num_bytes,
bool is_delalloc);
int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
u64 physical, u64 **logical, int *naddrs, int *stripe_len);
static inline u64 btrfs_data_alloc_profile(struct btrfs_fs_info *fs_info)
{
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3453,127 +3453,145 @@ int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chun
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_chunk_map *map;
int ret;
map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
if (IS_ERR(map)) {
DEBUG_WARN("errr %ld reading chunk map at offset %llu",
PTR_ERR(map), chunk_offset);
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
*
* 1) Just like with the first phase of the chunk allocation, we must
* reserve system space, do all chunk btree updates and deletions, and
* update the system chunk array in the superblock while holding this
* mutex. This is for similar reasons as explained on the comment at
* the top of btrfs_chunk_alloc();
*
* 2) Prevent races with the final phase of a device replace operation
* that replaces the device object associated with the map's stripes,
* because the device object's id can change at any time during that
* final phase of the device replace operation
* (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
* replaced device and then see it with an ID of
* BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
* the device item, which does not exists on the chunk btree.
* The finishing phase of device replace acquires both the
* device_list_mutex and the chunk_mutex, in that order, so we are
* safe by just acquiring the chunk_mutex.
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
/*
* Normally we should not get -ENOSPC since we reserved space before
* through the call to check_system_chunk().
*
* Despite our system space_info having enough free space, we may not
* be able to allocate extents from its block groups, because all have
* an incompatible profile, which will force us to allocate a new system
* block group with the right profile, or right after we called
* check_system_space() above, a scrub turned the only system block group
* with enough free space into RO mode.
* This is explained with more detail at do_chunk_alloc().
*
* So if we get -ENOSPC, allocate a new system chunk and retry once.
*/
if (ret == -ENOSPC) {
const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *sys_bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, sys_flags);
if (unlikely(!space_info)) {
ret = -EINVAL;
btrfs_abort_transaction(trans, ret);
goto out;
}
sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
if (IS_ERR(sys_bg)) {
ret = PTR_ERR(sys_bg);
btrfs_abort_transaction(trans, ret);
goto out;
}
ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
ret = remove_chunk_item(trans, map, chunk_offset);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
} else if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
}
btrfs_update_per_profile_avail(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
/*
* We are done with chunk btree updates and deletions, so release the
* system space we previously reserved (with check_system_chunk()).
*/
btrfs_trans_release_chunk_metadata(trans);
/* On error, btrfs_remove_block_group() aborts the transaction. */
ret = btrfs_remove_block_group(trans, map);
if (unlikely(ret))
ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
out:
if (trans->removing_chunk) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
}
/* once for us */
btrfs_free_chunk_map(map);
return ret;
}
]
|
| 317/4 |
2026/08/08 04:10 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 31254
output: 46
thoughts: 1167
Thoughts: **Analyzing BTRFS Chunk Allocation and Error Handling**
Okay, so I've been reviewing these changes and they look syntactically correct and follow the established idioms. The shift to returning an `int` from `reserve_chunk_space` and `check_system_chunk` is standard for BTRFS error handling – good. It makes the error checking more explicit and consistent.
In `btrfs_remove_chunk`, the new return value check on `check_system_chunk` before calling `btrfs_remove_dev_extents` is a solid defensive measure. We fail gracefully if system chunk allocation isn't possible *before* making device tree modifications. That's exactly the right approach. Then releasing the reservation is correct given what `btrfs_remove_dev_extents` can do.
Now, let's step through `check_system_chunk`'s behavior and make sure all the bases are covered. I wanted to verify that failures within `check_system_chunk` itself are properly propagated. Specifically, I needed to check what happens if `btrfs_create_chunk`, `btrfs_zoned_activate_one_bg`, or `btrfs_chunk_alloc_add_chunk_item` fail. `reserve_chunk_space` correctly propagates the error from `btrfs_create_chunk` and `btrfs_zoned_activate_one_bg`. The code correctly ignores errors from `btrfs_chunk_alloc_add_chunk_item`, since the BTRFS design allows for retries later in the chunk allocation process. The comment makes it clear that failure is possible but acceptable in that specific spot. That's by design.
It seems the current code logic is that if `reserve_chunk_space` returns an error, it means we failed to allocate a system chunk. But I started wondering if there were other places where we call `check_system_chunk`, and whether we should be checking its return value there. I looked at the call in `btrfs_inc_block_group_ro`. It's called when making a block group read-only. I reasoned that if it *fails* to allocate a system chunk there, it might be fine, since we might not *actually* need the system chunk at that point (the read-only block group might not have any nodes that require COW, for example). If we *do* need the chunk and it failed, the subsequent tree modification would fail with `-ENOSPC` and would be handled there. Also, `btrfs_inc_block_group_ro` does *not* do any tree modifications itself, it just ensures there is enough space. So it's fine to ignore the return value here.
The same logic applies to `do_chunk_alloc`. It checks for system space before allocating data/metadata chunks. If that check fails, the chunk will fail later, which is acceptable, so ignoring the return is fine.
To be extra cautious, I decided to check for compiler warnings about ignoring the return value. Since `check_system_chunk` returns `int`, it usually doesn't generate a warning unless the function is marked with `__must_check`. `check_system_chunk` is not marked with that attribute, so no warning is expected. That's good; it confirms that the current code behavior is not implicitly broken.
Finally, I reviewed `btrfs_remove_chunk` again to make certain I hadn't missed anything. I confirmed that `check_system_chunk` is called to ensure there is space before any modification, and the reservation is released later by `btrfs_trans_release_chunk_metadata`, which is the correct thing to do. So the overall flow in `btrfs_remove_chunk` appears correct and safe. All looks good!
|
| 318/4 |
2026/08/08 04:11 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:btrfs_trans_release_chunk_metadata]
Results: map[SourceCode: 248: /*
249: * To be called after doing the chunk btree updates right after allocating a new
250: * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
251: * chunk after all chunk btree updates and after finishing the second phase of
252: * chunk allocation (btrfs_create_pending_block_groups()) in case some block
253: * group had its chunk item insertion delayed to the second phase.
254: */
255: void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
256: {
257: struct btrfs_fs_info *fs_info = trans->fs_info;
258:
259: if (!trans->chunk_bytes_reserved)
260: return;
261:
262: btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
263: trans->chunk_bytes_reserved, NULL);
264: trans->chunk_bytes_reserved = 0;
265: }
SourceFile:fs/btrfs/transaction.c]
|
| 319/4 |
2026/08/08 04:11 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 33638
output: 19
thoughts: 498
Thoughts: **Analyzing the Chunk Metadata Release**
Okay, so `btrfs_trans_release_chunk_metadata` *does* indeed release `trans->chunk_bytes_reserved` and sets it to zero. That's exactly what I'd expect and want. Perfect.
Now, let's just double-check that this function isn't called anywhere else in this context that could mess things up. Ah, yes, it's also called at the very end of `btrfs_remove_chunk`:
```c
/*
* We are done with chunk btree updates and deletions, so release the
* system space we previously reserved (with check_system_chunk()).
*/
btrfs_trans_release_chunk_metadata(trans);
```
That's fine. That releases the reservation made by the second `check_system_chunk`, so that's all good.
Wait a minute... What happens if `btrfs_remove_dev_extents` fails?
```c
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
```
If it fails, we `goto out`, and, importantly, `trans->chunk_bytes_reserved` will already be zero, so we're not leaking any reservation. That's correct; there's no problem there.
Okay, let's revisit the core of the bug's description. The abort happens because we run out of system space when removing a chunk item. We couldn't allocate a new system chunk.
The idea is that we can call `check_system_chunk` early *to ensure* that we'll have enough space. We'll allocate a system chunk if necessary. Then, *immediately* release the `chunk_block_rsv` reservation. If `check_system_chunk` fails to allocate, we can return `-ENOSPC` and fail gracefully *before* modifying anything in the trees. After `btrfs_remove_dev_extents`, we call `check_system_chunk` again to reserve for `remove_chunk_item`.
That's exactly what I implemented, except, rather than relying on `trans->chunk_bytes_reserved` to infer a failure in `check_system_chunk`, I changed `check_system_chunk` to return an `int` error code. This is the more idiomatic approach and makes the code cleaner.
Alright, now let's make sure that `check_system_chunk` isn't used in other places in a way that would be incompatible with this change.
|
| 320/4 |
2026/08/08 04:11 |
tool |
grepper |
0m
Args: map[Expression:check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=1383=struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
--
fs/btrfs/block-group.c-1415- * more device items and remove one chunk item), but this is done at
fs/btrfs/block-group.c:1416: * btrfs_remove_chunk() through a call to check_system_chunk().
fs/btrfs/block-group.c-1417- */
--
fs/btrfs/block-group.c=3125=int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
--
fs/btrfs/block-group.c-3233- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/block-group.c:3234: check_system_chunk(trans, alloc_flags);
fs/btrfs/block-group.c-3235- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4140- */
fs/btrfs/block-group.c:4141: check_system_chunk(trans, flags);
fs/btrfs/block-group.c-4142-
--
fs/btrfs/block-group.c-4165- * single device, we are forced to allocate a new system chunk with a
fs/btrfs/block-group.c:4166: * SINGLE profile. Making check_system_chunk() iterate over all system
fs/btrfs/block-group.c-4167- * block groups and check if they have a usable profile and enough space
--
fs/btrfs/block-group.c-4173- * 2) We had enough free space info the system space_info, and one suitable
fs/btrfs/block-group.c:4174: * block group to allocate from when we called check_system_chunk()
fs/btrfs/block-group.c-4175- * above. However right after we called it, the only system block group
--
fs/btrfs/block-group.c-4181- * 3) We had one system block group with enough free space when we called
fs/btrfs/block-group.c:4182: * check_system_chunk(), but after that, right before we tried to
fs/btrfs/block-group.c-4183- * allocate the last extent buffer we needed, a discard operation came
--
fs/btrfs/block-group.c-4309- * needs to update the chunk btree (the only btree that uses system chunks), must
fs/btrfs/block-group.c:4310: * preallocate chunk space by calling either check_system_chunk() or
fs/btrfs/block-group.c-4311- * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
--
fs/btrfs/block-group.c-4316- *
fs/btrfs/block-group.c:4317: * The reservation of system space, done through check_system_chunk(), as well
fs/btrfs/block-group.c-4318- * as all the updates and insertions into the chunk btree must be done while
--
fs/btrfs/block-group.c=4341=int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4375- * before updating the chunk btree, by calling either
fs/btrfs/block-group.c:4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
fs/btrfs/block-group.c-4377- * It's possible that after a task reserves the space, it still ends up
--
fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3493-
fs/btrfs/volumes.c:3494: check_system_chunk(trans, map->type);
fs/btrfs/volumes.c-3495-
--
fs/btrfs/volumes.c-3498- * Normally we should not get -ENOSPC since we reserved space before
fs/btrfs/volumes.c:3499: * through the call to check_system_chunk().
fs/btrfs/volumes.c-3500- *
--
fs/btrfs/volumes.c-3561- * We are done with chunk btree updates and deletions, so release the
fs/btrfs/volumes.c:3562: * system space we previously reserved (with check_system_chunk()).
fs/btrfs/volumes.c-3563- */
]
|
| 321/4 |
2026/08/08 04:11 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 36310
output: 53
thoughts: 843
Thoughts: **Analyzing Chunk Removal in Btrfs**
Okay, so I've been looking at this chunk removal code, specifically within `fs/btrfs`. My initial modifications involved changing the return types of `reserve_chunk_space` and `check_system_chunk` to `int`, and updating the header declaration. I've also integrated an early `check_system_chunk` call in `btrfs_remove_chunk` to check if a chunk can be removed. I need to make sure I haven't introduced any issues with the way reservations and releases are handled.
First, I need to confirm the flow. In `btrfs_remove_chunk`, `check_system_chunk` is called. It seems that the code then releases the reservation. Now, could `check_system_chunk` return 0 (success) but not actually increment `trans->chunk_bytes_reserved`? Let's trace it back. I see in `reserve_chunk_space` that if `btrfs_block_rsv_add` *fails*, then `trans->chunk_bytes_reserved` isn't incremented. But if `btrfs_block_rsv_add` *succeeds* then `trans->chunk_bytes_reserved` is incremented.
So, if `check_system_chunk` returns 0, then the chunk bytes are *indeed* incremented, and then *subsequently* released. That should be safe.
However, what if `check_system_chunk` returns an error *after* some amount of bytes are already reserved in `trans->chunk_bytes_reserved`? It will release ALL of them! In the function, `btrfs_trans_release_chunk_metadata`, it unconditionally releases all the reserved space.
Now, where does `btrfs_remove_chunk` get called from? Is there a possibility that `trans->chunk_bytes_reserved` could be non-zero *before* the call to `check_system_chunk`? We know `btrfs_remove_chunk` is called from `btrfs_relocate_chunk` and `btrfs_delete_unused_bgs`.
So, the critical question is: Do the callers of `btrfs_remove_chunk` ever have existing chunk reservations? If they *do*, and `check_system_chunk` fails, the existing reservations will be prematurely released, leading to potential issues with subsequent operations that rely on those reservations. I need to investigate the callers to verify.
|
| 322/4 |
2026/08/08 04:11 |
tool |
codesearch-find-references |
0m
Args: map[ContextFile:fs/btrfs/volumes.c IncludeSnippetLines:0 Name:btrfs_remove_chunk SourceTreePrefix:]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_delete_unused_bgs SourceFile:fs/btrfs/block-group.c SourceLine:1818] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_relocate_chunk_finish SourceFile:fs/btrfs/volumes.c SourceLine:3614]] TruncatedOutput:false]
|
| 323/4 |
2026/08/08 04:11 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 37885
output: 43
thoughts: 289
Thoughts: **Chunk Removal Reservations: A Potential Problem?**
Okay, so I'm looking at these calls to `btrfs_remove_chunk` in `btrfs_delete_unused_bgs` and `btrfs_relocate_chunk_finish`. It strikes me that in both instances, there's no obvious reservation of chunk bytes *before* the removal happens. That feels… a little off, doesn't it? Like, we're just blindly removing chunks without accounting for potential resource implications?
And then I'm digging through this old patch and see they were checking `trans->chunk_bytes_reserved`. They were doing something like: storing the current value of `trans->chunk_bytes_reserved`, running some `check_system_chunk` stuff, and then comparing the before and after values of `trans->chunk_bytes_reserved`. The implication here is that `trans->chunk_bytes_reserved` *could* be non-zero at this point, or maybe it was just being coded defensively.
Now, hold on a second… if `trans->chunk_bytes_reserved` *was* non-zero, and we then call `btrfs_trans_release_chunk_metadata(trans)`, wouldn't that inadvertently release the *prior* reservations as well? That's a major concern. It suggests there's a risk of releasing resources we *think* are still allocated.
But wait, this feels like I might be missing something. Let me see, what could call that release function? Ah, yes, `btrfs_remove_dev_extents`. And it can trigger *nested* chunk allocations which then unconditionally call `btrfs_trans_release_chunk_metadata(trans)`! Okay, I need to check how that function works, and make sure my concern is unfounded.
|
| 324/4 |
2026/08/08 04:11 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/volumes.c Name:btrfs_remove_dev_extents]
Results: map[SourceCode:3404: int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
3405: {
3406: struct btrfs_fs_info *fs_info = trans->fs_info;
3407: struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
3408: u64 dev_extent_len = 0;
3409: int i, ret = 0;
3410:
3411: /*
3412: * First delete the device extent items from the devices btree.
3413: * We take the device_list_mutex to avoid racing with the finishing phase
3414: * of a device replace operation. See the comment below before acquiring
3415: * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
3416: * because that can result in a deadlock when deleting the device extent
3417: * items from the devices btree - COWing an extent buffer from the btree
3418: * may result in allocating a new metadata chunk, which would attempt to
3419: * lock again fs_info->chunk_mutex.
3420: */
3421: mutex_lock(&fs_devices->device_list_mutex);
3422: for (i = 0; i < map->num_stripes; i++) {
3423: struct btrfs_device *device = map->stripes[i].dev;
3424: ret = btrfs_free_dev_extent(trans, device,
3425: map->stripes[i].physical,
3426: &dev_extent_len);
3427: if (unlikely(ret)) {
3428: mutex_unlock(&fs_devices->device_list_mutex);
3429: btrfs_abort_transaction(trans, ret);
3430: return ret;
3431: }
3432:
3433: if (device->bytes_used > 0) {
3434: mutex_lock(&fs_info->chunk_mutex);
3435: btrfs_device_set_bytes_used(device,
3436: device->bytes_used - dev_extent_len);
3437: atomic64_add(dev_extent_len, &fs_info->free_chunk_space);
3438: btrfs_clear_space_info_full(fs_info);
3439:
3440: if (list_empty(&device->post_commit_list)) {
3441: list_add_tail(&device->post_commit_list,
3442: &trans->transaction->dev_update_list);
3443: }
3444:
3445: mutex_unlock(&fs_info->chunk_mutex);
3446: }
3447: }
3448: mutex_unlock(&fs_devices->device_list_mutex);
3449:
3450: return 0;
3451: }
SourceFile:fs/btrfs/volumes.c]
|
| 325/4 |
2026/08/08 04:11 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 39475
output: 32
thoughts: 3254
Thoughts: **Chunk Metadata Reservation Optimization: My Thought Process**
Alright, let's break this down. The key issue here revolves around reserving and releasing metadata chunk space during chunk removal in Btrfs, specifically in the `btrfs_remove_chunk` function. My analysis centered on the comment in `btrfs_remove_dev_extents`: "COWing an extent buffer from the btree may result in allocating a new metadata chunk, which would attempt to lock again `fs_info->chunk_mutex`." And, importantly, chunk allocations within that function lead to a release of `trans->chunk_bytes_reserved` anyway.
Given this, I realized we could safely and efficiently optimize the space reservation process. My approach is now to call `check_system_chunk` early *before* any device tree modifications happen using `btrfs_remove_dev_extents`. Crucially, I've modified `check_system_chunk` and the underlying `reserve_chunk_space` to return an `int` error code. This is a subtle but important change, as it enables the direct propagation and checking of failure conditions, aligning with the more idiomatic error handling in Btrfs. If `check_system_chunk` fails, we return `-ENOSPC` and fail gracefully *before* touching the trees.
If the initial `check_system_chunk` succeeds, we *immediately* release the block reserve with `btrfs_trans_release_chunk_metadata(trans)`. The reasoning is that, because `btrfs_remove_dev_extents` can *itself* trigger nested chunk allocations (which automatically release all chunk metadata reservations), holding this reservation is redundant. By releasing it now, we avoid potentially unnecessary and complex locking interactions. Finally, after `btrfs_remove_dev_extents` has run, we call `check_system_chunk` *again* to actually make the reservation for the `remove_chunk_item` operation. This ensures we have the necessary space at the appropriate time without holding the reservation unnecessarily for the intermediate steps.
I've double-checked that `check_system_chunk` is only called in `block-group.c` and `volumes.c`, and that in the former, it's return value is not checked in the current context.
Therefore, this is a clean and correct optimization.
|
| 326/4 |
2026/08/08 04:11 |
tool |
codeexpert |
0m
Args: map[Question:Is there any other place where check_system_chunk is called in the kernel tree?]
Results: map[Answer:Yes, there are exactly three places in the Linux kernel tree where `check_system_chunk` is called:
1. **`fs/btrfs/block-group.c`** inside the function **`btrfs_inc_block_group_ro`**:
```c
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, alloc_flags);
mutex_unlock(&fs_info->chunk_mutex);
}
```
2. **`fs/btrfs/block-group.c`** inside the function **`do_chunk_alloc`**:
```c
/*
* Check if we have enough space in the system space info because we
* will need to update device items in the chunk btree and insert a new
* chunk item in the chunk btree as well. This will allocate a new
* system block group if needed.
*/
check_system_chunk(trans, flags);
```
3. **`fs/btrfs/volumes.c`** inside the function **`btrfs_remove_chunk`**:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
```]
|
| 327/5 |
2026/08/08 04:11 |
agent |
codeexpert |
0m
Model:
Results: map[Answer:Yes, there are exactly three places in the Linux kernel tree where `check_system_chunk` is called:
1. **`fs/btrfs/block-group.c`** inside the function **`btrfs_inc_block_group_ro`**:
```c
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, alloc_flags);
mutex_unlock(&fs_info->chunk_mutex);
}
```
2. **`fs/btrfs/block-group.c`** inside the function **`do_chunk_alloc`**:
```c
/*
* Check if we have enough space in the system space info because we
* will need to update device items in the chunk btree and insert a new
* chunk item in the chunk btree as well. This will allocate a new
* system block group if needed.
*/
check_system_chunk(trans, flags);
```
3. **`fs/btrfs/volumes.c`** inside the function **`btrfs_remove_chunk`**:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
```]
Instruction:
You are a capable Linux kernel developer tasked with researching complex questions
about kernel source code. You will be given a concrete question, and need to provide
a concrete answer.
Use tools extensively while researching the question. Don't make assumptions,
or rely on your previous knowledge about the kernel source code, use available tools
to access the actual source code.
Use all available sources of information:
- kernel source code
- documentation in the Documentation dir in the source tree
- git commits descriptions, git blame
Do not guess file names or file paths and attempt to read them without
verifying their existence first using content search or directory listing tools.
If a file, symbol, or directory is not found via content search (grepper) or
directory listing (codesearch-dir-index), treat it as completely absent.
Do not attempt to guess alternative names, extensions, or directories.
Tool Selection Guidelines:
1. For C functions, structs, and variables: Prefer codesearch-definition-source
or codesearch-struct-layout FIRST to retrieve clean, exact definitions
without line-number guessing.
2. If symbol lookup fails (e.g., preprocessor macros, macro-generated code, or
disabled #ifdef branches), fall back to read-file or grepper.
3. For file headers, #include directives, preprocessor macro definitions, and
non-C files (Kconfig, Makefiles, docs): Use read-file or grepper directly.
Do NOT use git-log to search for the presence or existence of files in the
repository. git-log is only for tracing commit history of files that are
already present in the current checkout. If a file does not exist in the
current checkout, it cannot be used for reproduction.
Avoid running broad git-log queries (such as searches on the entire repo)
without a specific 'PathPrefix' to restrict the scope, as these are very
expensive and will time out.
If a git-log tool call times out, do not retry the query with the same broad
scope. You must specify a tighter, narrower 'PathPrefix' for subsequent queries.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Is there any other place where check_system_chunk is called in the kernel tree?
|
| 328/6 |
2026/08/08 04:11 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 3885
output: 23
thoughts: 69
Thoughts: **Investigating the Kernel's `check_system_chunk` Usage**
Okay, so I need to figure out the scope of `check_system_chunk` within the kernel. The user's question is essentially: "Besides where I already know, where else is this function called?" My initial instinct is to use the standard tools available to us for this kind of investigation. Let's start with `codesearch-find-references` or if that fails, try a grep search. Seems like the most efficient way to get a quick overview would be to start with the former. I'll launch that search and see what it turns up. Then I can dig deeper based on the results.
|
| 329/6 |
2026/08/08 04:11 |
tool |
codesearch-find-references |
0m
Args: map[Name:check_system_chunk]
Results: map[References:[map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_inc_block_group_ro SourceFile:fs/btrfs/block-group.c SourceLine:3234] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:btrfs_remove_chunk SourceFile:fs/btrfs/volumes.c SourceLine:3494] map[ReferenceKind:calls ReferencingEntityKind:function ReferencingEntityName:do_chunk_alloc SourceFile:fs/btrfs/block-group.c SourceLine:4141]] TruncatedOutput:false]
|
| 330/6 |
2026/08/08 04:11 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 4310
output: 19
thoughts: 38
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Ensuring Comprehensive Code Review**
Right, so I've gone through the main code files, but I need to be *absolutely* certain I haven't missed any instances of the `check_system_chunk` function. It's easy to overlook things, especially when dealing with a codebase of this size.
I'm thinking, a simple search through the primary files isn't enough. I need to be thorough. I need to grep the entire project, just to be certain. I need to catch every possible occurrence. I need to check for instances within comments, macro definitions, included files, and even in documentation files that might reference this function. This is critical. There could be indirect usages, or legacy implementations, I need to know about. A comprehensive search is the only way to guarantee a complete understanding of how this check is utilized within the entire system. Gotta be rigorous.
|
| 331/6 |
2026/08/08 04:11 |
tool |
grepper |
0m
Args: map[Expression:check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=1383=struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
--
fs/btrfs/block-group.c-1415- * more device items and remove one chunk item), but this is done at
fs/btrfs/block-group.c:1416: * btrfs_remove_chunk() through a call to check_system_chunk().
fs/btrfs/block-group.c-1417- */
--
fs/btrfs/block-group.c=3125=int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
--
fs/btrfs/block-group.c-3233- mutex_lock(&fs_info->chunk_mutex);
fs/btrfs/block-group.c:3234: check_system_chunk(trans, alloc_flags);
fs/btrfs/block-group.c-3235- mutex_unlock(&fs_info->chunk_mutex);
--
fs/btrfs/block-group.c=4128=static struct btrfs_block_group *do_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4140- */
fs/btrfs/block-group.c:4141: check_system_chunk(trans, flags);
fs/btrfs/block-group.c-4142-
--
fs/btrfs/block-group.c-4165- * single device, we are forced to allocate a new system chunk with a
fs/btrfs/block-group.c:4166: * SINGLE profile. Making check_system_chunk() iterate over all system
fs/btrfs/block-group.c-4167- * block groups and check if they have a usable profile and enough space
--
fs/btrfs/block-group.c-4173- * 2) We had enough free space info the system space_info, and one suitable
fs/btrfs/block-group.c:4174: * block group to allocate from when we called check_system_chunk()
fs/btrfs/block-group.c-4175- * above. However right after we called it, the only system block group
--
fs/btrfs/block-group.c-4181- * 3) We had one system block group with enough free space when we called
fs/btrfs/block-group.c:4182: * check_system_chunk(), but after that, right before we tried to
fs/btrfs/block-group.c-4183- * allocate the last extent buffer we needed, a discard operation came
--
fs/btrfs/block-group.c-4309- * needs to update the chunk btree (the only btree that uses system chunks), must
fs/btrfs/block-group.c:4310: * preallocate chunk space by calling either check_system_chunk() or
fs/btrfs/block-group.c-4311- * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
--
fs/btrfs/block-group.c-4316- *
fs/btrfs/block-group.c:4317: * The reservation of system space, done through check_system_chunk(), as well
fs/btrfs/block-group.c-4318- * as all the updates and insertions into the chunk btree must be done while
--
fs/btrfs/block-group.c=4341=int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4375- * before updating the chunk btree, by calling either
fs/btrfs/block-group.c:4376: * btrfs_reserve_chunk_metadata() or check_system_chunk().
fs/btrfs/block-group.c-4377- * It's possible that after a task reserves the space, it still ends up
--
fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
--
fs/btrfs/volumes.c=3453=int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
--
fs/btrfs/volumes.c-3493-
fs/btrfs/volumes.c:3494: check_system_chunk(trans, map->type);
fs/btrfs/volumes.c-3495-
--
fs/btrfs/volumes.c-3498- * Normally we should not get -ENOSPC since we reserved space before
fs/btrfs/volumes.c:3499: * through the call to check_system_chunk().
fs/btrfs/volumes.c-3500- *
--
fs/btrfs/volumes.c-3561- * We are done with chunk btree updates and deletions, so release the
fs/btrfs/volumes.c:3562: * system space we previously reserved (with check_system_chunk()).
fs/btrfs/volumes.c-3563- */
]
|
| 332/6 |
2026/08/08 04:11 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 6160
output: 40
thoughts: 47
Thoughts: Okay, let's dive into this.
**Understanding `check_system_chunk`**
My initial thought is to get a solid grasp on this `check_system_chunk` function. To truly understand its behavior and context, I need to know precisely what it *does* and where it's defined within the system. My go-to approach in these scenarios is to pinpoint the exact definition. This isn't just about reading the code; it's about building a mental map of how this function fits into the overall architecture. I'm going to leverage the `codesearch-definition-source` tool to track down where this function is declared. This should quickly give me the declaration location, and the source code itself to inspect. From there, I can analyze the function signature, and the code, in detail.
|
| 333/6 |
2026/08/08 04:12 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:fs/btrfs/block-group.c Name:check_system_chunk]
Results: map[SourceCode:4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
4580: /* num_devs device items to update and 1 chunk item to add or remove. */
4581: bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
4582: btrfs_calc_insert_metadata_size(fs_info, 1);
4583:
4584: reserve_chunk_space(trans, bytes, type);
4585: }
SourceFile:fs/btrfs/block-group.c]
|
| 334/6 |
2026/08/08 04:12 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 6741
output: 40
thoughts: 273
Thoughts: **Analyzing `check_system_chunk` Calls in the Btrfs Kernel Code**
Okay, let's take a closer look at these `check_system_chunk` calls within the Btrfs kernel code, just to make sure we haven't missed anything and that the context is exactly what we expect. The grep results gave us three locations to investigate.
First, there's a call in `fs/btrfs/block-group.c` at line 3234, inside `btrfs_inc_block_group_ro`. I want to make certain that's indeed the call we're interested in. It appears the code locks a `chunk_mutex` before the call, and unlocks it afterward. That is an important detail. Now let's look at the surrounding code.
|
| 335/6 |
2026/08/08 04:12 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:3225 LineCount:20]
Results: map[Contents:3225: goto out;
3226:
3227: ret = inc_block_group_ro(cache, false);
3228: if (ret == -ETXTBSY)
3229: goto unlock_out;
3230: out:
3231: if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
3232: alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
3233: mutex_lock(&fs_info->chunk_mutex);
3234: check_system_chunk(trans, alloc_flags);
3235: mutex_unlock(&fs_info->chunk_mutex);
3236: }
3237: unlock_out:
3238: mutex_unlock(&fs_info->ro_block_group_mutex);
3239:
3240: btrfs_end_transaction(trans);
3241: return ret;
3242: }
3243:
3244: void btrfs_dec_block_group_ro(struct btrfs_block_group *cache)
]
|
| 336/6 |
2026/08/08 04:12 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 7549
output: 40
thoughts: 56
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Diving into the Btrfs Block Group Increment Function**
Alright, so the code is calling `check_system_chunk` from within `btrfs_inc_block_group_ro`. That's interesting, and likely means there's some kind of consistency check happening within the block group increment operation. Given my experience, I'm guessing this might involve verifying some critical metadata or ensuring the read-only block group's integrity during the update. Now, let's get into the details: I need to refresh my memory on what's going on. I'll need to jump over to `fs/btrfs/block-group.c`, specifically around line 4141, to understand the context and the purpose of this call. Time to dig in and see what's being checked.
|
| 337/6 |
2026/08/08 04:12 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4130 LineCount:20]
Results: map[Contents:4130: u64 flags)
4131: {
4132: struct btrfs_block_group *bg;
4133: int ret;
4134:
4135: /*
4136: * Check if we have enough space in the system space info because we
4137: * will need to update device items in the chunk btree and insert a new
4138: * chunk item in the chunk btree as well. This will allocate a new
4139: * system block group if needed.
4140: */
4141: check_system_chunk(trans, flags);
4142:
4143: bg = btrfs_create_chunk(trans, space_info, flags);
4144: if (IS_ERR(bg)) {
4145: ret = PTR_ERR(bg);
4146: goto out;
4147: }
4148:
4149: ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
]
|
| 338/6 |
2026/08/08 04:12 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 8136
output: 38
thoughts: 48
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Diving into Btrfs Chunk Allocation**
Right, so `do_chunk_alloc` is calling `check_system_chunk`. That's the key starting point, as expected. We need to understand the logic flow, and likely any error conditions being handled there.
Now, let's head over to `fs/btrfs/volumes.c`, specifically around line 3494. That's usually the part where the low-level volume management and allocation decisions are made, so that's where I need to focus. I'll be looking for how the system chunk check interacts with the larger allocation process: error handling, resource availability, maybe some RAID-specific considerations. Got to dissect what's going on behind the scenes, and how it's all handled.
|
| 339/6 |
2026/08/08 04:12 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3485 LineCount:20]
Results: map[Contents:3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
3490: */
3491: trans->removing_chunk = true;
3492: mutex_lock(&fs_info->chunk_mutex);
3493:
3494: check_system_chunk(trans, map->type);
3495:
3496: ret = remove_chunk_item(trans, map, chunk_offset);
3497: /*
3498: * Normally we should not get -ENOSPC since we reserved space before
3499: * through the call to check_system_chunk().
3500: *
3501: * Despite our system space_info having enough free space, we may not
3502: * be able to allocate extents from its block groups, because all have
3503: * an incompatible profile, which will force us to allocate a new system
3504: * block group with the right profile, or right after we called
]
|
| 340/6 |
2026/08/08 04:12 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 8776
output: 360
thoughts: 1941
Thoughts: **Analyzing `check_system_chunk` Calls in the Btrfs Kernel Code**
Okay, so I see a direct call to `check_system_chunk` within `btrfs_remove_chunk`. That's the starting point. Now, to make absolutely sure, I need to understand the full scope of where this function is used. My initial thought was it was limited to the removal operation, but I know better than to assume.
I've got three direct call sites confirmed: first in `btrfs_inc_block_group_ro` within `fs/btrfs/block-group.c`, again in the same file within `do_chunk_alloc`, and finally, the original in `btrfs_remove_chunk` in `fs/btrfs/volumes.c`. But to be thorough, I ran a grep to make sure there weren't any other instances hiding somewhere, maybe indirect calls or within macros.
The grep output confirms the three direct calls. It's a key function in allocating and managing system chunks, so I'm not surprised to see its use scattered around the chunk allocation and removal processes within the `block-group.c` file. The documentation and comments within the grep results also clarify its purpose: reserving space for chunk metadata, particularly when data or metadata chunks are being updated. It appears to act as a central point for ensuring enough system space is available before performing operations that require chunk updates.
Looks like a pretty complete picture based on this data. This analysis provides a sound foundation for investigating the specific role of `check_system_chunk` in the context of `btrfs_remove_chunk`.
|
| 341/6 |
2026/08/08 04:12 |
tool |
set-results |
0m
Args: map[Answer:Yes, there are exactly three places in the Linux kernel tree where `check_system_chunk` is called:
1. **`fs/btrfs/block-group.c`** inside the function **`btrfs_inc_block_group_ro`**:
```c
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, alloc_flags);
mutex_unlock(&fs_info->chunk_mutex);
}
```
2. **`fs/btrfs/block-group.c`** inside the function **`do_chunk_alloc`**:
```c
/*
* Check if we have enough space in the system space info because we
* will need to update device items in the chunk btree and insert a new
* chunk item in the chunk btree as well. This will allocate a new
* system block group if needed.
*/
check_system_chunk(trans, flags);
```
3. **`fs/btrfs/volumes.c`** inside the function **`btrfs_remove_chunk`**:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
```]
Results: map[Answer:Yes, there are exactly three places in the Linux kernel tree where `check_system_chunk` is called:
1. **`fs/btrfs/block-group.c`** inside the function **`btrfs_inc_block_group_ro`**:
```c
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = btrfs_get_alloc_profile(fs_info, cache->flags);
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, alloc_flags);
mutex_unlock(&fs_info->chunk_mutex);
}
```
2. **`fs/btrfs/block-group.c`** inside the function **`do_chunk_alloc`**:
```c
/*
* Check if we have enough space in the system space info because we
* will need to update device items in the chunk btree and insert a new
* chunk item in the chunk btree as well. This will allocate a new
* system block group if needed.
*/
check_system_chunk(trans, flags);
```
3. **`fs/btrfs/volumes.c`** inside the function **`btrfs_remove_chunk`**:
```c
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
```]
|
| 342/4 |
2026/08/08 04:12 |
llm |
patch-generator |
0m
Model:
Tokens:
input: 43622
output: 607
thoughts: 264
Thoughts: **Finalizing the BTRFS Chunk Removal Fix**
Alright, I've got this. I've reviewed the existing code and understand the problem with the transaction abort. Time to formulate a complete summary of the changes I've made. First, I made some key signature adjustments, returning `int` instead of `void` from both `reserve_chunk_space` and `check_system_chunk`. This is crucial for proper error propagation, I'm making this explicit. I also updated the declaration of `check_system_chunk` in `fs/btrfs/block-group.h` to match. The major change lies within `btrfs_remove_chunk`. I've added an early call to `check_system_chunk` right at the beginning, *before* we get into `btrfs_remove_dev_extents`. This initial call is critical; it is the linchpin that will handle the problems found previously.
Now, here is the important part, I'm *checking the return value* of that early `check_system_chunk` call. If it fails, we fail *gracefully*, jumping directly to `out`. No more transaction abort! If it succeeds, and that's the intended outcome in most cases, then we immediately release the space reservation using `btrfs_trans_release_chunk_metadata` and unlock the mutex. This is important to ensure other operations can proceed. The remaining logic in `btrfs_remove_chunk` is left untouched, including the *second* call to `check_system_chunk`. This second call is now guaranteed to succeed since we've already ensured system space availability through the early check. I'm confident that this is a clean, idiomatic fix that aligns perfectly with established BTRFS error handling patterns and avoids the previous transaction abort issue. This is how it should be done!
|
| 343/3 |
2026/08/08 04:12 |
action |
test-patch |
23m
Results: map[PatchDiff:diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
TestError:]
|
| 344/1 |
2026/08/08 04:35 |
action |
apply-patch |
0m
Results:
|
| 345/1 |
2026/08/08 04:35 |
action |
init-style-items |
0m
Results:
|
| 346/1 |
2026/08/08 04:35 |
loop |
|
0m
|
| 347/2 |
2026/08/08 04:35 |
iteration |
0 |
0m
|
| 348/3 |
2026/08/08 04:35 |
agent |
patch-formatter |
0m
Model:
Instruction:
You are an expert Linux kernel developer tasked with formatting a kernel patch.
Your objective is purely formatting: you must ensure the patch complies with the kernel's coding style,
conforms to the surrounding code rules, and passes checkpatch.pl, while preserving the code logic exactly as it is.
You should stop once the requested formatting changes are done and checkpatch.pl is happy.
Do not question the requested changes unless they are obviously wrong.
If the code already conforms to the requested changes, surrounding code rules,
and checkpatch.pl is happy, you should just finish your task.
WARNING: The clang-format tool may break the formatting of the surrounding code (like manual alignment).
Use it with caution. We want to make the change fit into the existing formatting as much as possible.
Prefer calling several tools at the same time to save round-trips.
Prompt:
The current patch diff is:
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
Use the provided tools to format the patch.
Reply: I have thoroughly analyzed the patch and verified its compliance with the Linux kernel's coding style and surrounding code rules.
1. **Parameter Alignment**: The indentation of the parameters in `reserve_chunk_space` was correctly adjusted from 4 tabs (32 spaces) to 3 tabs and 7 spaces (31 spaces) to perfectly align with the open parenthesis after changing the return type from `void` to `int`.
2. **Line Lengths**: All added comments and code lines are well within the 80-character limit.
3. **Checkpatch Compliance**: Running `checkpatch.pl` on the patch yields:
```
total: 0 errors, 0 warnings, 68 lines checked
Your patch has no obvious style problems and is ready for submission.
```
Since the patch already conforms perfectly to the requested changes, surrounding code rules, and `checkpatch.pl` is completely happy, no further formatting changes are needed. The patch is ready as is!
|
| 349/4 |
2026/08/08 04:35 |
llm |
patch-formatter |
0m
Model:
Tokens:
input: 5588
output: 9
thoughts: 50
Thoughts: Okay, here's my take on those thoughts, framed as my own mental process:
**Patch Formatting Initial Assessment**
Alright, a new patch has landed. First order of business: get this formatted correctly. Before diving deep, let's fire up `checkpatch`. It's my go-to sanity check. But wait, `checkpatch` needs the *actual* patch data to function, so I'll need to figure out how to run it in this environment. Let's see if that's possible... I'll need to make sure I have the necessary tools and environment set up.
|
| 350/4 |
2026/08/08 04:35 |
tool |
checkpatch |
0m
Results: map[Output:total: 0 errors, 0 warnings, 68 lines checked
Your patch has no obvious style problems and is ready for submission.
]
|
| 351/4 |
2026/08/08 04:35 |
llm |
patch-formatter |
0m
Model:
Tokens:
input: 5817
output: 118
thoughts: 178
Thoughts: **Patch Review: A Quick Check for Btrfs Style**
Okay, so `checkpatch` is giving me the green light – "0 errors, 0 warnings, 68 lines checked." That's a good start, but I'm not going to blindly trust it. I need to make absolutely sure there aren't any subtle style issues that `checkpatch` might miss, or any subtle deviations from existing coding style. You know how it is; consistency is key in the kernel.
Therefore, let's dive into the code and give it a quick read-through to ensure formatting is correct and matches surrounding code. My plan is to zero in on a few key areas within this patch, specifically around these lines in the Btrfs filesystem:
* `fs/btrfs/block-group.c` around lines 4488, 4565, and 4581.
* `fs/btrfs/block-group.h` around line 378.
* `fs/btrfs/volumes.c` around line 3463.
I'll be looking for anything that jumps out – indentation, whitespace, naming conventions, and any general feel for the code. This quick read-through will either confirm the initial good news or allow me to correct minor formatting before submission.
|
| 352/4 |
2026/08/08 04:35 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.c FirstLine:4480 LineCount:115]
Results: map[Contents:4480: static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
4481: {
4482: u64 num_dev;
4483:
4484: num_dev = btrfs_raid_array[btrfs_bg_flags_to_raid_index(type)].devs_max;
4485: if (!num_dev)
4486: num_dev = fs_info->fs_devices->rw_devices;
4487:
4488: return num_dev;
4489: }
4490:
4491: static void reserve_chunk_space(struct btrfs_trans_handle *trans,
4492: u64 bytes,
4493: u64 type)
4494: {
4495: struct btrfs_fs_info *fs_info = trans->fs_info;
4496: struct btrfs_space_info *info;
4497: u64 left;
4498: int ret = 0;
4499:
4500: /*
4501: * Needed because we can end up allocating a system chunk and for an
4502: * atomic and race free space reservation in the chunk block reserve.
4503: */
4504: lockdep_assert_held(&fs_info->chunk_mutex);
4505:
4506: info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
4507: spin_lock(&info->lock);
4508: left = info->total_bytes - btrfs_space_info_used(info, true);
4509: spin_unlock(&info->lock);
4510:
4511: if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
4512: btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
4513: left, bytes, type);
4514: btrfs_dump_space_info(info, 0, false);
4515: }
4516:
4517: if (left < bytes) {
4518: u64 flags = btrfs_system_alloc_profile(fs_info);
4519: struct btrfs_block_group *bg;
4520: struct btrfs_space_info *space_info;
4521:
4522: space_info = btrfs_find_space_info(fs_info, flags);
4523: ASSERT(space_info);
4524:
4525: /*
4526: * Ignore failure to create system chunk. We might end up not
4527: * needing it, as we might not need to COW all nodes/leafs from
4528: * the paths we visit in the chunk tree (they were already COWed
4529: * or created in the current transaction for example).
4530: */
4531: bg = btrfs_create_chunk(trans, space_info, flags);
4532: if (IS_ERR(bg)) {
4533: ret = PTR_ERR(bg);
4534: } else {
4535: int activate_ret;
4536:
4537: /*
4538: * We have a new chunk. We also need to activate it for
4539: * zoned filesystem.
4540: */
4541: activate_ret = btrfs_zoned_activate_one_bg(info, true);
4542: if (activate_ret < 0) {
4543: ret = activate_ret;
4544: } else {
4545: /*
4546: * If we fail to add the chunk item here, we end
4547: * up trying again at phase 2 of chunk allocation,
4548: * at btrfs_create_pending_block_groups(). So
4549: * ignore any error here. An ENOSPC here could
4550: * happen, due to the cases described at
4551: * do_chunk_alloc() - the system block group we
4552: * just created was just turned into RO mode by a
4553: * scrub for example, or a running discard
4554: * temporarily removed its free space entries, etc.
4555: */
4556: btrfs_chunk_alloc_add_chunk_item(trans, bg);
4557: }
4558: }
4559: }
4560:
4561: if (!ret) {
4562: ret = btrfs_block_rsv_add(fs_info,
4563: &fs_info->chunk_block_rsv,
4564: bytes, BTRFS_RESERVE_NO_FLUSH);
4565: if (!ret)
4566: trans->chunk_bytes_reserved += bytes;
4567: }
4568: }
4569:
4570: /*
4571: * Reserve space in the system space for allocating or removing a chunk.
4572: * The caller must be holding fs_info->chunk_mutex.
4573: */
4574: void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
4575: {
4576: struct btrfs_fs_info *fs_info = trans->fs_info;
4577: const u64 num_devs = get_profile_num_devs(fs_info, type);
4578: u64 bytes;
4579:
]
|
| 353/4 |
2026/08/08 04:35 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/block-group.h FirstLine:370 LineCount:20]
Results: map[Contents: 370: int btrfs_update_block_group(struct btrfs_trans_handle *trans,
371: u64 bytenr, u64 num_bytes, bool alloc);
372: int btrfs_add_reserved_bytes(struct btrfs_block_group *cache,
373: u64 ram_bytes, u64 num_bytes, bool delalloc,
374: bool force_wrong_size_class);
375: void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, u64 num_bytes,
376: bool is_delalloc);
377: int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
378: struct btrfs_space_info *space_info, u64 flags,
379: enum btrfs_chunk_alloc_enum force);
380: int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
381: void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
382: void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
383: bool is_item_insertion);
384: u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
385: void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
386: int btrfs_free_block_groups(struct btrfs_fs_info *info);
387: int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
388: u64 physical, u64 **logical, int *naddrs, int *stripe_len);
389:
]
|
| 354/4 |
2026/08/08 04:35 |
tool |
read-file |
0m
Args: map[File:fs/btrfs/volumes.c FirstLine:3450 LineCount:40]
Results: map[Contents:3450: return 0;
3451: }
3452:
3453: int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
3454: {
3455: struct btrfs_fs_info *fs_info = trans->fs_info;
3456: struct btrfs_chunk_map *map;
3457: int ret;
3458:
3459: map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
3460: if (IS_ERR(map)) {
3461: DEBUG_WARN("errr %ld reading chunk map at offset %llu",
3462: PTR_ERR(map), chunk_offset);
3463: return PTR_ERR(map);
3464: }
3465:
3466: ret = btrfs_remove_dev_extents(trans, map);
3467: if (ret)
3468: goto out;
3469:
3470: /*
3471: * We acquire fs_info->chunk_mutex for 2 reasons:
3472: *
3473: * 1) Just like with the first phase of the chunk allocation, we must
3474: * reserve system space, do all chunk btree updates and deletions, and
3475: * update the system chunk array in the superblock while holding this
3476: * mutex. This is for similar reasons as explained on the comment at
3477: * the top of btrfs_chunk_alloc();
3478: *
3479: * 2) Prevent races with the final phase of a device replace operation
3480: * that replaces the device object associated with the map's stripes,
3481: * because the device object's id can change at any time during that
3482: * final phase of the device replace operation
3483: * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
3484: * replaced device and then see it with an ID of
3485: * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
3486: * the device item, which does not exists on the chunk btree.
3487: * The finishing phase of device replace acquires both the
3488: * device_list_mutex and the chunk_mutex, in that order, so we are
3489: * safe by just acquiring the chunk_mutex.
]
|
| 355/4 |
2026/08/08 04:35 |
llm |
patch-formatter |
0m
Model:
Tokens:
input: 9258
output: 10
thoughts: 1360
Thoughts: **Analyzing the Btrfs Chunk Space Reservation Patch**
Okay, let's dive into this patch diff. My immediate focus is on how it modifies the chunk space reservation logic within the Btrfs filesystem. The primary changes seem to be centered around `reserve_chunk_space` and `check_system_chunk`, and they are likely aimed at improving error handling or resource management during chunk allocation and removal, specifically concerning metadata space.
Firstly, the signature of `reserve_chunk_space` has changed; it now returns an `int` rather than being `void`. This indicates a shift from a fire-and-forget approach to one where the caller expects and *needs* to know the outcome of the reservation attempt. This return value is going to be crucial to determining what happens if the allocation fails, and I need to investigate the error codes the function can return and where they are used.
Secondly, the change from `void check_system_chunk` to `int check_system_chunk` mirrors this. Previously, `check_system_chunk` appears to have simply *attempted* to reserve chunk space, while the new version explicitly reports success or failure. This is very important.
The `volumes.c` changes are particularly interesting. The `btrfs_remove_chunk` function now calls `check_system_chunk` *before* modifying the device tree. This is a very good defensive programming practice. By performing the chunk space check *before* modifying the filesystem structure, the patch avoids potential inconsistencies if the space reservation fails. If `check_system_chunk` returns an error, the code gracefully exits, ensuring that the filesystem is not left in a partially modified state. There's a call to `btrfs_trans_release_chunk_metadata` which is called only when the space check is successful and before the chunk is removed. This likely optimizes metadata allocation by releasing it prior to starting the process.
This is all good. These changes look like they are designed to improve the robustness of Btrfs, and, based on the context of the diff, I can infer that the goal is likely to prevent issues when removing chunks. Now I will inspect the surrounding code, and look into where the return value of `reserve_chunk_space` and `check_system_chunk` are handled. I need to understand the implications of a failed reservation and how the system responds. I'll need to check if the error is handled appropriately or if there's any potential for cascading failures, and whether this reservation is happening within a transaction, as is expected. If these changes are coupled with changes to other routines, I'll need to carefully inspect the logic around those changes to make sure I understand the complete impact of this change.
|
| 356/4 |
2026/08/08 04:35 |
tool |
patch-diff |
0m
Error:
|
| 357/4 |
2026/08/08 04:35 |
llm |
patch-formatter |
0m
Model:
Tokens:
input: 11218
output: 15
thoughts: 71
Thoughts: **Navigating the `patch-diff` Tool**
Okay, so I'm looking at this `patch-diff` tool, and the documentation is a little… ambiguous, isn't it? It *says* the `File` argument is optional, that it will show all changes if left empty. But then the schema, the *schema* explicitly states that `File` is *required*! This is a classic case of documentation and schema contradicting each other, which always throws a wrench into things.
So, let me get this straight: if I want all the changes, I can't just *omit* the `File` argument as the documentation seems to imply. Instead, I need to provide an empty string, `""`, as the argument. That's a little unintuitive, but I've dealt with this before. It's the only way to satisfy the schema's requirements while achieving the desired result. Alright, I'll go with `""`. Hopefully, that’s going to work as intended.
|
| 358/4 |
2026/08/08 04:35 |
tool |
patch-diff |
0m
Args:
Results: map[Output:diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4481,122 +4481,124 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
{
u64 num_dev;
num_dev = btrfs_raid_array[btrfs_bg_flags_to_raid_index(type)].devs_max;
if (!num_dev)
num_dev = fs_info->fs_devices->rw_devices;
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
int ret = 0;
/*
* Needed because we can end up allocating a system chunk and for an
* atomic and race free space reservation in the chunk block reserve.
*/
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
left, bytes, type);
btrfs_dump_space_info(info, 0, false);
}
if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
*/
bg = btrfs_create_chunk(trans, space_info, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
} else {
int activate_ret;
/*
* We have a new chunk. We also need to activate it for
* zoned filesystem.
*/
activate_ret = btrfs_zoned_activate_one_bg(info, true);
if (activate_ret < 0) {
ret = activate_ret;
} else {
/*
* If we fail to add the chunk item here, we end
* up trying again at phase 2 of chunk allocation,
* at btrfs_create_pending_block_groups(). So
* ignore any error here. An ENOSPC here could
* happen, due to the cases described at
* do_chunk_alloc() - the system block group we
* just created was just turned into RO mode by a
* scrub for example, or a running discard
* temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
}
}
if (!ret) {
ret = btrfs_block_rsv_add(fs_info,
&fs_info->chunk_block_rsv,
bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
u64 bytes;
/* num_devs device items to update and 1 chunk item to add or remove. */
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
* Reserve space in the system space, if needed, for doing a modification to the
* chunk btree.
*
* @trans: A transaction handle.
* @is_item_insertion: Indicate if the modification is for inserting a new item
* in the chunk btree or if it's for the deletion or update
* of an existing item.
*
* This is used in a context where we need to update the chunk btree outside
* block group allocation and removal, to avoid a deadlock with a concurrent
* task that is allocating a metadata or data block group and therefore needs to
* update the chunk btree while holding the chunk mutex. After the update to the
* chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
*
*/
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -371,21 +371,21 @@ int btrfs_update_block_group(struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes, bool alloc);
int btrfs_add_reserved_bytes(struct btrfs_block_group *cache,
u64 ram_bytes, u64 num_bytes, bool delalloc,
bool force_wrong_size_class);
void btrfs_free_reserved_bytes(struct btrfs_block_group *cache, u64 num_bytes,
bool is_delalloc);
int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
u64 physical, u64 **logical, int *naddrs, int *stripe_len);
static inline u64 btrfs_data_alloc_profile(struct btrfs_fs_info *fs_info)
{
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3453,127 +3453,145 @@ int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chun
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_chunk_map *map;
int ret;
map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
if (IS_ERR(map)) {
DEBUG_WARN("errr %ld reading chunk map at offset %llu",
PTR_ERR(map), chunk_offset);
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
*
* 1) Just like with the first phase of the chunk allocation, we must
* reserve system space, do all chunk btree updates and deletions, and
* update the system chunk array in the superblock while holding this
* mutex. This is for similar reasons as explained on the comment at
* the top of btrfs_chunk_alloc();
*
* 2) Prevent races with the final phase of a device replace operation
* that replaces the device object associated with the map's stripes,
* because the device object's id can change at any time during that
* final phase of the device replace operation
* (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
* replaced device and then see it with an ID of
* BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
* the device item, which does not exists on the chunk btree.
* The finishing phase of device replace acquires both the
* device_list_mutex and the chunk_mutex, in that order, so we are
* safe by just acquiring the chunk_mutex.
*/
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
/*
* Normally we should not get -ENOSPC since we reserved space before
* through the call to check_system_chunk().
*
* Despite our system space_info having enough free space, we may not
* be able to allocate extents from its block groups, because all have
* an incompatible profile, which will force us to allocate a new system
* block group with the right profile, or right after we called
* check_system_space() above, a scrub turned the only system block group
* with enough free space into RO mode.
* This is explained with more detail at do_chunk_alloc().
*
* So if we get -ENOSPC, allocate a new system chunk and retry once.
*/
if (ret == -ENOSPC) {
const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *sys_bg;
struct btrfs_space_info *space_info;
space_info = btrfs_find_space_info(fs_info, sys_flags);
if (unlikely(!space_info)) {
ret = -EINVAL;
btrfs_abort_transaction(trans, ret);
goto out;
}
sys_bg = btrfs_create_chunk(trans, space_info, sys_flags);
if (IS_ERR(sys_bg)) {
ret = PTR_ERR(sys_bg);
btrfs_abort_transaction(trans, ret);
goto out;
}
ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
ret = remove_chunk_item(trans, map, chunk_offset);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
} else if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
trace_btrfs_chunk_free(fs_info, map, chunk_offset, map->chunk_len);
if (map->type & BTRFS_BLOCK_GROUP_SYSTEM) {
ret = btrfs_del_sys_chunk(fs_info, chunk_offset);
if (unlikely(ret)) {
btrfs_abort_transaction(trans, ret);
goto out;
}
}
btrfs_update_per_profile_avail(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
/*
* We are done with chunk btree updates and deletions, so release the
* system space we previously reserved (with check_system_chunk()).
*/
btrfs_trans_release_chunk_metadata(trans);
/* On error, btrfs_remove_block_group() aborts the transaction. */
ret = btrfs_remove_block_group(trans, map);
if (unlikely(ret))
ASSERT(BTRFS_FS_ERROR(fs_info) != 0);
out:
if (trans->removing_chunk) {
mutex_unlock(&fs_info->chunk_mutex);
trans->removing_chunk = false;
}
/* once for us */
btrfs_free_chunk_map(map);
return ret;
}
]
|
| 359/4 |
2026/08/08 04:35 |
llm |
patch-formatter |
0m
Model:
Tokens:
input: 15624
output: 211
thoughts: 2932
Thoughts: **Patch Analysis and Formatting Compliance**
Alright, let's break this down. My initial scan of the provided code changes, specifically the modification of `check_system_chunk`'s return type from `void` to `int` and adjustments in `fs/btrfs/volumes.c` and `fs/btrfs/block-group.c` and `fs/btrfs/block-group.h`, triggered a series of checks. First, I verified if the change introduces any logical inconsistencies, which isn't a factor here, given the explicit formatting focus.
The core concern was the preservation of coding style, especially alignment. I meticulously assessed the parameter alignment in `reserve_chunk_space` and other affected functions, ensuring that the introduction of a different return type hasn't disrupted the existing alignment which perfectly adheres to kernel coding guidelines (aligning parameters vertically with the opening parenthesis). The indentation and formatting are all correct.
I ran `checkpatch.pl` and carefully reviewed its output. It came back clean, reporting no errors or warnings, and confirming that the patch "has no obvious style problems and is ready for submission." No other tool suggests any formatting changes and none are needed.
Therefore, since the code aligns perfectly with the formatting specified in the kernel's coding standards and the surrounding code, and since `checkpatch.pl` returns no errors or warnings, I conclude that no further formatting adjustments are necessary. The patch is ready as is.
|
| 360/3 |
2026/08/08 04:36 |
action |
test-patch-inplace |
0m
Results: map[PatchDiff:diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
TestError:]
|
| 361/3 |
2026/08/08 04:36 |
action |
run-checkpatch |
0m
Results: map[CheckpatchOutput:total: 0 errors, 0 warnings, 68 lines checked
Your patch has no obvious style problems and is ready for submission.
NeedRefinement:false]
|
| 362/1 |
2026/08/08 04:36 |
agent |
fixes-finder |
13m
Model:
Results: map[FixesHash:79bd37120b149532af5b21953643ed74af69654f]
Instruction:
You are an experienced Linux kernel developer tasked with identifying the commit
that introduced the bug being fixed. Identifying the correct buggy commit is crucial
for proper kernel maintenance (backporting to stable trees, etc.).
Your investigation strategy:
1. Examine the patch that fixes the bug. Use git tools (like git-log or git-blame)
to trace the history of the lines or functions modified by the patch.
2. Analyze the stack trace in the crash report. Identify the key files and functions
involved in the crash and investigate their history to see when the problematic
logic was introduced.
3. Compare the bug explanation with the commit history to find the point where
the described logic error first appeared.
A bug is typically introduced when a piece of code is first written, or when
a refactoring changed its logic in a way that introduced the bug.
Trace the history of relevant symbols or find when specific code patterns were introduced/removed.
You must provide exactly one bug-introducing commit hash.
If you are unable to confidently determine the bug-introducing commit after investigation,
return an empty string rather than guessing.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt:
The crash is:
BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
The explanation of the root cause is:
An analysis of the crash reveals that it is caused by a transaction abort (`-ENOSPC`) in `btrfs_remove_chunk` during a chunk relocation/balance operation. The abort happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
### Root Cause
1. **Order of Operations**: During chunk removal, `btrfs_remove_chunk` first removes the device extents from the device tree by calling `btrfs_remove_dev_extents`.
2. **System Space Reservation**: It then calls `check_system_chunk` to reserve system space for removing the chunk item from the chunk tree. If the system space is full, `check_system_chunk` attempts to allocate a new system chunk.
3. **Commit Root Search**: To allocate a new chunk, `find_free_dev_extent` is called. However, `find_free_dev_extent` searches the *commit root* of the device tree to prevent reusing space freed in the current transaction (which could lead to corruption if the transaction aborts).
4. **Invisible Freed Space**: Because the device extents were just freed in the *current* transaction by `btrfs_remove_dev_extents`, `find_free_dev_extent` does not see this freed space. If the device is otherwise full, it returns `-ENOSPC`.
5. **Ignored Failure**: `check_system_chunk` ignores the failure to allocate a system chunk and does not reserve any space in `trans->chunk_bytes_reserved`.
6. **Transaction Abort**: `remove_chunk_item` is then called, which fails with `-ENOSPC` because it needs to COW a node in the chunk tree but no system space was reserved. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully (returning `-ENOSPC` would leave the filesystem in an inconsistent state with the chunk item present but device extents missing). Thus, it is forced to abort the transaction.
### The Fix
We must ensure that we have enough system space *before* we modify the device tree. However, we cannot simply move the `check_system_chunk` call before `btrfs_remove_dev_extents` and keep the reservation, because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations (`btrfs_trans_release_chunk_metadata`).
Instead, we can call `check_system_chunk` early to *ensure* the system `space_info` has enough free space (allocating a system chunk if necessary), and then immediately release the `chunk_block_rsv` reservation. If it fails to allocate a system chunk, we can fail gracefully by returning `-ENOSPC` before any trees are modified. Then, after `btrfs_remove_dev_extents`, we call `check_system_chunk` again to make the actual reservation for `remove_chunk_item`.
Here is the proposed fix for `fs/btrfs/volumes.c` in `btrfs_remove_chunk`:
```c
map = em->map_lookup;
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
/*
* Take the device list mutex to prevent races with the final phase of
* a device replace operation...
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
// ... existing btrfs_remove_dev_extents logic ...
}
mutex_unlock(&fs_devices->device_list_mutex);
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
### Why this works:
1. **Graceful Failure**: By checking `trans->chunk_bytes_reserved`, we can detect if `check_system_chunk` failed to allocate a system chunk. If it failed, we return `-ENOSPC` *before* any trees are modified, allowing the balance operation to fail cleanly instead of crashing the kernel.
2. **No Metadata Leaks**: We immediately release the early reservation using `btrfs_trans_release_chunk_metadata(trans)`.
3. **Safe from Nested Allocations**: Because we release the reservation, we don't care if `btrfs_remove_dev_extents` triggers a nested chunk allocation that clears `trans->chunk_bytes_reserved`.
4. **Guaranteed Space**: The early `check_system_chunk` ensures the system `space_info` has enough free space (by allocating a 32MB system chunk if needed). When we call `check_system_chunk` the second time, it will see the free space in the `space_info` and successfully reserve the few KB needed for `remove_chunk_item`.
The patch that fixes the bug is:
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
Search for the commit(s) that introduced this bug.
|
| 363/2 |
2026/08/08 04:36 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 8352
output: 38
thoughts: 195
Thoughts: **Tracing the Bug in btrfs_remove_chunk**
Okay, so we've got a tricky one here. I need to pinpoint the commit that broke `btrfs_remove_chunk`. Specifically, this bug causes the transaction to abort due to system space exhaustion. The proposed fix, which seems reasonable, changes `check_system_chunk` to return an integer, rather than `void`. This allows `btrfs_remove_chunk` to check available system space *early*, before attempting the chunk removal. If there's insufficient space, it bails out, immediately freeing the metadata reservation, and returning an error, preventing the transaction abortion.
My strategy is clear: I need to dive into the history of both `check_system_chunk` and `btrfs_remove_chunk`. I have to identify the point where `check_system_chunk` started ignoring allocation failures, as that's where the root cause lies. It used to be `void`, silently swallowing errors. My assumption is that's the culprit.
I'll lean heavily on `git-log` to do the heavy lifting here. I'll trace the evolution of both functions, examining commit messages and diffs to identify the relevant changes. This will enable me to pinpoint the precise commit that introduced this potentially fatal oversight.
|
| 364/2 |
2026/08/08 04:36 |
tool |
git-log |
0m
Args: map[SourcePath:fs/btrfs/block-group.c SymbolName:check_system_chunk]
Results: map[Output:2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
]
|
| 365/2 |
2026/08/08 04:36 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 8899
output: 20
thoughts: 34
Thoughts: Here's my thought process, summarized as you requested:
**Potential Code Location Uncertainty**
Okay, hold on a second. I have a hunch that this `check_system_chunk` function might have been refactored or moved around recently. I don't recall seeing it explicitly defined in the current file. To be absolutely sure and avoid a wild goose chase, I need to verify its location. I need to conduct a thorough search. Let's fire up a tree-wide search for the exact identifier, `check_system_chunk`, to determine its definitive location within the project. This will let me confirm if it's where I expect or if I need to update import paths or understand where its definition now resides. A comprehensive search is the only way to avoid wasted time.
|
| 366/2 |
2026/08/08 04:36 |
tool |
grepper |
0m
Args: map[Expression:void check_system_chunk]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
]
|
| 367/2 |
2026/08/08 04:36 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 9372
output: 23
thoughts: 47
Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue:
**Tracing the Lineage of `check_system_chunk`**
Alright, time to get to the bottom of this. I need to understand the evolution of that `check_system_chunk` function. My gut tells me it's central to how Btrfs handles its block groups, and potentially how it interacts with the extent tree, but I need to confirm and understand the changes. Let me start by digging into its past. I'm going to track down its history specifically within `fs/btrfs/block-group.c` and `fs/btrfs/extent-tree.c`. It's pretty likely it has been moved, refactored, or maybe even integrated into something else over the course of Btrfs's development. This is where the real Btrfs expertise comes into play: I will need to understand where it *used* to be to understand its current behavior, as it may be renamed or integrated in another function. I anticipate it may have moved from one place to another, so I'll need to go to the git history to see the path it took. Now, let's see where that takes me...
|
| 368/2 |
2026/08/08 04:36 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:void check_system_chunk]
Results:
|
| 369/2 |
2026/08/08 04:46 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 9742
output: 35
thoughts: 20
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience and written in the first person:
**Diving into `check_system_chunk` in Btrfs**
Okay, so I want to understand how the system chunk is being checked within the Btrfs filesystem. Let's start by tracing the function `check_system_chunk`. My immediate goal is to understand the control flow, the data structures it interacts with, and what conditions trigger its execution. I'll be looking for potential error paths, any optimizations implemented, and how it handles different scenarios within a Btrfs filesystem. I'm assuming a certain level of familiarity with Btrfs internals, so I'll be focusing on the specifics of this chunk-checking function rather than going back to first principles. I expect to find interactions with the block group map, extent trees, and potentially checksumming or redundancy mechanisms. I also anticipate it could check metadata integrity, handle device failures (if relevant to system chunks), and handle error correction.
|
| 370/2 |
2026/08/08 04:46 |
tool |
git-log |
0m
Args: map[CodeRegexp:void check_system_chunk PathPrefix:fs/btrfs/]
Results: map[Output:2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
07730d87ac78 btrfs: migrate the chunk allocation code
451a2c130342 btrfs: Remove fs_info from check_system_chunk
39c2d7faccc5 Btrfs: fix -ENOSPC on block group removal
15d1ff8111aa Btrfs: fix deadlock during allocating chunks
]
|
| 371/2 |
2026/08/08 04:46 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 10139
output: 24
thoughts: 51
Thoughts: Okay, let's dive into these two Btrfs commits. I want to understand how they address specific issues, particularly from a performance and reliability standpoint, given my understanding of the filesystem's internals.
**Initial Assessment of Btrfs Commit Fixes**
First up, `39c2d7faccc5 Btrfs: fix -ENOSPC on block group removal`. My initial reaction is that an `-ENOSPC` error during block group removal is a critical failure. This usually points to a significant flaw in the space management or accounting, something that I need to quickly identify as this can trigger a chain of performance issues and data integrity concerns. I immediately start wondering about the specific scenario that triggers this error: is it related to fragmentation, metadata overhead, or some race condition during block group unlinking? The details of the patch will be key here.
Then, there's `15d1ff8111aa Btrfs: fix deadlock during allocating chunks`. Deadlocks during chunk allocation are equally dangerous. They can completely halt filesystem operations, leading to data loss and system instability. I want to know precisely which code paths are involved in this deadlock. Understanding the locking primitives used, the resource contention, and the execution order will be essential to grasping the root cause. My instinct is that this likely involves some interaction between metadata updates, allocation bitmap manipulation, and possibly transaction management. I'm keen to see if this involves the transaction layer, or lower-level block allocation.
|
| 372/2 |
2026/08/08 04:47 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 39c2d7faccc5ca5a1be682b01c0db5fafa8adeda
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed May 20 14:01:55 2015 +0100
Btrfs: fix -ENOSPC on block group removal
Unlike when attempting to allocate a new block group, where we check
that we have enough space in the system space_info to update the device
items and insert a new chunk item in the chunk tree, we were not checking
if the system space_info had enough space for updating the device items
and deleting the chunk item in the chunk tree. This often lead to -ENOSPC
error when attempting to allocate blocks for the chunk tree (during btree
node/leaf COW operations) while updating the device items or deleting the
chunk item, which resulted in the current transaction being aborted and
turning the filesystem into read-only mode.
While running fstests generic/038, which stresses allocation of block
groups and removal of unused block groups, with a large scratch device
(750Gb) this happened often, despite more than enough unallocated space,
and resulted in the following trace:
[68663.586604] WARNING: CPU: 3 PID: 1521 at fs/btrfs/super.c:260 __btrfs_abort_transaction+0x52/0x114 [btrfs]()
[68663.600407] BTRFS: Transaction aborted (error -28)
(...)
[68663.730829] Call Trace:
[68663.732585] [<ffffffff8142fa46>] dump_stack+0x4f/0x7b
[68663.734334] [<ffffffff8108b6a2>] ? console_unlock+0x361/0x3ad
[68663.739980] [<ffffffff81045ea5>] warn_slowpath_common+0xa1/0xbb
[68663.757153] [<ffffffffa036ca6d>] ? __btrfs_abort_transaction+0x52/0x114 [btrfs]
[68663.760925] [<ffffffff81045f05>] warn_slowpath_fmt+0x46/0x48
[68663.762854] [<ffffffffa03b159d>] ? btrfs_update_device+0x15a/0x16c [btrfs]
[68663.764073] [<ffffffffa036ca6d>] __btrfs_abort_transaction+0x52/0x114 [btrfs]
[68663.765130] [<ffffffffa03b3638>] btrfs_remove_chunk+0x597/0x5ee [btrfs]
[68663.765998] [<ffffffffa0384663>] ? btrfs_delete_unused_bgs+0x245/0x296 [btrfs]
[68663.767068] [<ffffffffa0384676>] btrfs_delete_unused_bgs+0x258/0x296 [btrfs]
[68663.768227] [<ffffffff8143527f>] ? _raw_spin_unlock_irq+0x2d/0x4c
[68663.769081] [<ffffffffa038b109>] cleaner_kthread+0x13d/0x16c [btrfs]
[68663.799485] [<ffffffffa038afcc>] ? btrfs_alloc_root+0x28/0x28 [btrfs]
[68663.809208] [<ffffffff8105f367>] kthread+0xef/0xf7
[68663.828795] [<ffffffff810e603f>] ? time_hardirqs_on+0x15/0x28
[68663.844942] [<ffffffff8105f278>] ? __kthread_parkme+0xad/0xad
[68663.846486] [<ffffffff81435a88>] ret_from_fork+0x58/0x90
[68663.847760] [<ffffffff8105f278>] ? __kthread_parkme+0xad/0xad
[68663.849503] ---[ end trace 798477c6d6dbaad6 ]---
[68663.850525] BTRFS: error (device sdc) in btrfs_remove_chunk:2652: errno=-28 No space left
So fix this by verifying that enough space exists in system space_info,
and reserving the space in the chunk block reserve, before attempting to
delete the block group and allocate a new system chunk if we don't have
enough space to perform the necessary updates and delete in the chunk
tree. Like for the block group creation case, we don't error our if we
fail to allocate a new system chunk, since we might end up not needing
it (no node/leaf splits happen during the COW operations and/or we end
up not needing to COW any btree nodes or leafs because they were already
COWed in the current transaction and their writeback didn't start yet).
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Chris Mason <clm@fb.com>
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 8ee5645ef9e1..92e908394403 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -3516,6 +3516,10 @@ int btrfs_delayed_refs_qgroup_accounting(struct btrfs_trans_handle *trans,
int __get_raid_index(u64 flags);
int btrfs_start_write_no_snapshoting(struct btrfs_root *root);
void btrfs_end_write_no_snapshoting(struct btrfs_root *root);
+void check_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ const u64 type,
+ const bool is_allocation);
/* ctree.c */
int btrfs_bin_search(struct extent_buffer *eb, struct btrfs_key *key,
int level, int *slot);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 4e08e47ace30..e78ab29f8f1b 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4092,7 +4092,7 @@ static int should_alloc_chunk(struct btrfs_root *root,
return 1;
}
-static u64 get_system_chunk_thresh(struct btrfs_root *root, u64 type)
+static u64 get_profile_num_devs(struct btrfs_root *root, u64 type)
{
u64 num_dev;
@@ -4106,17 +4106,24 @@ static u64 get_system_chunk_thresh(struct btrfs_root *root, u64 type)
else
num_dev = 1; /* DUP or single */
- /* metadata for updaing devices and chunk tree */
- return btrfs_calc_trans_metadata_size(root, num_dev + 1);
+ return num_dev;
}
-static void check_system_chunk(struct btrfs_trans_handle *trans,
- struct btrfs_root *root, u64 type)
+/*
+ * If @is_allocation is true, reserve space in the system space info necessary
+ * for allocating a chunk, otherwise if it's false, reserve space necessary for
+ * removing a chunk.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 type,
+ const bool is_allocation)
{
struct btrfs_space_info *info;
u64 left;
u64 thresh;
int ret = 0;
+ u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -4131,7 +4138,15 @@ static void check_system_chunk(struct btrfs_trans_handle *trans,
info->bytes_may_use;
spin_unlock(&info->lock);
- thresh = get_system_chunk_thresh(root, type);
+ num_devs = get_profile_num_devs(root, type);
+
+ /* num_devs device items to update and 1 chunk item to add or remove */
+ if (is_allocation)
+ thresh = btrfs_calc_trans_metadata_size(root, num_devs + 1);
+ else
+ thresh = btrfs_calc_trans_metadata_size(root, num_devs) +
+ btrfs_calc_trunc_metadata_size(root, 1);
+
if (left < thresh && btrfs_test_opt(root, ENOSPC_DEBUG)) {
btrfs_info(root->fs_info, "left=%llu, need=%llu, flags=%llu",
left, thresh, type);
@@ -4243,7 +4258,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans,
* Check if we have enough space in SYSTEM chunk because we may need
* to update devices.
*/
- check_system_chunk(trans, extent_root, flags);
+ check_system_chunk(trans, extent_root, flags, true);
ret = btrfs_alloc_chunk(trans, extent_root, flags);
trans->allocating_chunk = false;
@@ -8905,7 +8920,7 @@ int btrfs_set_block_group_ro(struct btrfs_root *root,
if (cache->flags & BTRFS_BLOCK_GROUP_SYSTEM) {
alloc_flags = update_block_group_flags(root, cache->flags);
lock_chunks(root->fs_info->chunk_root);
- check_system_chunk(trans, root, alloc_flags);
+ check_system_chunk(trans, root, alloc_flags, true);
unlock_chunks(root->fs_info->chunk_root);
}
mutex_unlock(&root->fs_info->ro_block_group_mutex);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 534be440dd56..d7668756b9d0 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2625,6 +2625,9 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans,
return -EINVAL;
}
map = (struct map_lookup *)em->bdev;
+ lock_chunks(root->fs_info->chunk_root);
+ check_system_chunk(trans, extent_root, map->type, false);
+ unlock_chunks(root->fs_info->chunk_root);
for (i = 0; i < map->num_stripes; i++) {
struct btrfs_device *device = map->stripes[i].dev;
]
|
| 373/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 13785
output: 25
thoughts: 278
Thoughts: **Analyzing a Btrfs Chunk Removal Bug**
Okay, so `39c2d7faccc5` is where `check_system_chunk` was brought into `btrfs_remove_chunk`. And the bug report… right, it's hitting a snag because `btrfs_remove_dev_extents` just freed device extents within the *current* transaction. So, `find_free_dev_extent` doesn't see that space anymore, and returns `-ENOSPC` if the device is otherwise full. The key issue is that `check_system_chunk` *ignores* this allocation failure and doesn't reserve any space in `trans->chunk_bytes_reserved`.
That's the critical bit. Then, `remove_chunk_item` gets called, and it needs to COW a node in the chunk tree. But, since no system space was reserved, it also fails with `-ENOSPC`. And here's where things go sideways: `btrfs_remove_dev_extents` has already modified the device tree. So, `btrfs_remove_chunk` can't gracefully handle the `-ENOSPC` because that would leave the filesystem in a fundamentally inconsistent state. We’d have the chunk item present, but the device extents would be missing. The only option left is to abort the transaction. It seems related to the deadlock I was seeing in `15d1ff8111aa`, Btrfs: fix deadlock during allocating chunks.
|
| 374/2 |
2026/08/08 04:47 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 15d1ff8111aad85d8b40ee396758990d17a2caac
Author: Liu Bo <liubo2009@cn.fujitsu.com>
Date: Thu Mar 29 09:57:44 2012 -0400
Btrfs: fix deadlock during allocating chunks
This deadlock comes from xfstests 251.
We'll hold the chunk_mutex throughout the whole of a chunk allocation.
But if we find that we've used up system chunk space, we need to allocate a
new system chunk, but this will lead to a recursion of chunk allocation and end
up with a deadlock on chunk_mutex.
So instead we need to allocate the system chunk first if we find we're in ENOSPC.
Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 7c233407beee..a84420491c11 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3445,6 +3445,50 @@ static int should_alloc_chunk(struct btrfs_root *root,
return 1;
}
+static u64 get_system_chunk_thresh(struct btrfs_root *root, u64 type)
+{
+ u64 num_dev;
+
+ if (type & BTRFS_BLOCK_GROUP_RAID10 ||
+ type & BTRFS_BLOCK_GROUP_RAID0)
+ num_dev = root->fs_info->fs_devices->rw_devices;
+ else if (type & BTRFS_BLOCK_GROUP_RAID1)
+ num_dev = 2;
+ else
+ num_dev = 1; /* DUP or single */
+
+ /* metadata for updaing devices and chunk tree */
+ return btrfs_calc_trans_metadata_size(root, num_dev + 1);
+}
+
+static void check_system_chunk(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root, u64 type)
+{
+ struct btrfs_space_info *info;
+ u64 left;
+ u64 thresh;
+
+ info = __find_space_info(root->fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
+ spin_lock(&info->lock);
+ left = info->total_bytes - info->bytes_used - info->bytes_pinned -
+ info->bytes_reserved - info->bytes_readonly;
+ spin_unlock(&info->lock);
+
+ thresh = get_system_chunk_thresh(root, type);
+ if (left < thresh && btrfs_test_opt(root, ENOSPC_DEBUG)) {
+ printk(KERN_INFO "left=%llu, need=%llu, flags=%llu\n",
+ left, thresh, type);
+ dump_space_info(info, 0, 0);
+ }
+
+ if (left < thresh) {
+ u64 flags;
+
+ flags = btrfs_get_alloc_profile(root->fs_info->chunk_root, 0);
+ btrfs_alloc_chunk(trans, root, flags);
+ }
+}
+
static int do_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_root *extent_root, u64 alloc_bytes,
u64 flags, int force)
@@ -3515,6 +3559,12 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans,
force_metadata_allocation(fs_info);
}
+ /*
+ * Check if we have enough space in SYSTEM chunk because we may need
+ * to update devices.
+ */
+ check_system_chunk(trans, extent_root, flags);
+
ret = btrfs_alloc_chunk(trans, extent_root, flags);
if (ret < 0 && ret != -ENOSPC)
goto out;
]
|
| 375/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 15561
output: 37
thoughts: 406
Thoughts: **Analyzing the Placement of `check_system_chunk` in `btrfs_remove_chunk`**
Okay, let's break this down. My initial thought is, "Wait, `check_system_chunk` was introduced for chunk allocation in commit `15d1ff8111aa`, but then it was added to `btrfs_remove_chunk` in `39c2d7faccc5`." Now, I need to understand *how* it was added. Let's delve back into `39c2d7faccc5`.
Right, looking at the diff, `check_system_chunk` was indeed added to `btrfs_remove_chunk` like this:
```c
@@ -2625,6 +2625,9 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans,
return -EINVAL;
}
map = (struct map_lookup *)em->bdev;
+ lock_chunks(root->fs_info->chunk_root);
+ check_system_chunk(trans, extent_root, map->type, false);
+ unlock_chunks(root->fs_info->chunk_root);
for (i = 0; i < map->num_stripes; i++) {
struct btrfs_device *device = map->stripes[i].dev;
```
Interesting. Now, a key observation: In this original commit, the `check_system_chunk` call is *before* the loop that iterates through the stripes and calls `btrfs_remove_dev_extents`. This is crucial.
The problem lies elsewhere. The buggy code calls `check_system_chunk` *after* calling `btrfs_remove_dev_extents` for a chunk, which should be done before, not after.
The next thing I must look at is when `check_system_chunk` was moved to the *after* position, and why. That's the key to finding this bug.
|
| 376/2 |
2026/08/08 04:47 |
tool |
git-log |
0m
Args: map[CodeRegexp:check_system_chunk PathPrefix:fs/btrfs/volumes.c]
Results: map[Output:79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
451a2c130342 btrfs: Remove fs_info from check_system_chunk
2ff7e61e0d30 btrfs: take an fs_info directly when the root is not used otherwise
4617ea3a52cf Btrfs: fix necessary chunk tree space calculation when allocating a chunk
39c2d7faccc5 Btrfs: fix -ENOSPC on block group removal
]
|
| 377/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 16612
output: 26
thoughts: 33
Thoughts: Here's a summary of my thoughts, as though I'm processing that commit message:
**Analyzing Btrfs Chunk Allocation Rework**
Okay, so we're diving into `79bd37120b14 btrfs: rework chunk allocation to avoid exhaustion of the system chunk array`. This immediately flags some concerns. The "exhaustion of the system chunk array" part is the key issue here. I know Btrfs is already complex, and chunk allocation is a critical component of its on-disk layout and performance. Exhausting this array implies a fundamental limitation – a potential point of failure that's probably impacting scalability or, worse, leading to data corruption scenarios if the array is truly exhausted.
My immediate questions are: What's triggering the exhaustion? Is it a predictable workload pattern, an unexpected corner case, or simply a flawed allocation algorithm? What's the size of this "system chunk array"? Is it a fixed size, or dynamic? If fixed, that *screams* scalability bottleneck. If dynamic, how does it grow? What's the cost of growth?
The fact that they're *reworking* the allocation implies that the original implementation had fundamental flaws, not just simple tuning issues. This suggests a potentially invasive change, and the potential for regression is *high*. I'll need to carefully examine the patch to understand the previous algorithm and how this new rework improves the situation.
I'll be looking for things like: how they've changed the allocation strategy. Are they spreading chunks more evenly across devices, reducing fragmentation? Did they change the chunk size in order to resolve the issue? Are they introducing any new data structures or metadata to track the chunk availability?
I'm expecting to see a combination of algorithmic changes and possibly some tweaks to the data structures involved, perhaps a different use of bitmaps, or even more complex free space tracking. The commit message itself needs to be precise and provide enough context about the problem *and* the solution, I'm expecting detailed justification for this change. I'll need to look at performance implications, especially during heavy I/O operations and recovery scenarios, to ensure that this rework doesn't introduce any new performance bottlenecks or, worse, data corruption vulnerabilities. Overall, this is something I'll need to pay close attention to.
|
| 378/2 |
2026/08/08 04:47 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1281 lines.
[commit 79bd37120b149532af5b21953643ed74af69654f
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:06 2021 +0100
btrfs: rework chunk allocation to avoid exhaustion of the system chunk array
Commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations") fixed a problem that resulted in
exhausting the system chunk array in the superblock when there are many
tasks allocating chunks in parallel. Basically too many tasks enter the
first phase of chunk allocation without previous tasks having finished
their second phase of allocation, resulting in too many system chunks
being allocated. That was originally observed when running the fallocate
tests of stress-ng on a PowerPC machine, using a node size of 64K.
However that commit also introduced a deadlock where a task in phase 1 of
the chunk allocation waited for another task that had allocated a system
chunk to finish its phase 2, but that other task was waiting on an extent
buffer lock held by the first task, therefore resulting in both tasks not
making any progress. That change was later reverted by a patch with the
subject "btrfs: fix deadlock with concurrent chunk allocations involving
system chunks", since there is no simple and short solution to address it
and the deadlock is relatively easy to trigger on zoned filesystems, while
the system chunk array exhaustion is not so common.
This change reworks the chunk allocation to avoid the system chunk array
exhaustion. It accomplishes that by making the first phase of chunk
allocation do the updates of the device items in the chunk btree and the
insertion of the new chunk item in the chunk btree. This is done while
under the protection of the chunk mutex (fs_info->chunk_mutex), in the
same critical section that checks for available system space, allocates
a new system chunk if needed and reserves system chunk space. This way
we do not have chunk space reserved until the second phase completes.
The same logic is applied to chunk removal as well, since it keeps
reserved system space long after it is done updating the chunk btree.
For direct allocation of system chunks, the previous behaviour remains,
because otherwise we would deadlock on extent buffers of the chunk btree.
Changes to the chunk btree are by large done by chunk allocation and chunk
removal, which first reserve chunk system space and then later do changes
to the chunk btree. The other remaining cases are uncommon and correspond
to adding a device, removing a device and resizing a device. All these
other cases do not pre-reserve system space, they modify the chunk btree
right away, so they don't hold reserved space for a long period like chunk
allocation and chunk removal do.
The diff of this change is huge, but more than half of it is just addition
of comments describing both how things work regarding chunk allocation and
removal, including both the new behavior and the parts of the old behavior
that did not change.
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a26209f98279..c557327b4545 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2207,6 +2207,13 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
return ret;
}
+/*
+ * This function, insert_block_group_item(), belongs to the phase 2 of chunk
+ * allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
static int insert_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_block_group *block_group)
{
@@ -2229,15 +2236,19 @@ static int insert_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_insert_item(trans, root, &key, &bgi, sizeof(bgi));
}
+/*
+ * This function, btrfs_create_pending_block_groups(), belongs to the phase 2 of
+ * chunk allocation.
+ *
+ * See the comment at btrfs_chunk_alloc() for details about the chunk allocation
+ * phases.
+ */
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *block_group;
int ret = 0;
- if (!trans->can_flush_pending_bgs)
- return;
-
while (!list_empty(&trans->new_bgs)) {
int index;
@@ -2252,6 +2263,13 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
ret = insert_block_group_item(trans, block_group);
if (ret)
btrfs_abort_transaction(trans, ret);
+ if (!block_group->chunk_item_inserted) {
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, block_group);
+ mutex_unlock(&fs_info->chunk_mutex);
+ if (ret)
+ btrfs_abort_transaction(trans, ret);
+ }
ret = btrfs_finish_chunk_alloc(trans, block_group->start,
block_group->length);
if (ret)
@@ -2275,8 +2293,9 @@ void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans)
btrfs_trans_release_chunk_metadata(trans);
}
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size)
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_block_group *cache;
@@ -2286,7 +2305,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
cache = btrfs_create_block_group_cache(fs_info, chunk_offset);
if (!cache)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
cache->length = size;
set_free_space_tree_thresholds(cache);
@@ -2300,7 +2319,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
ret = btrfs_load_block_group_zone_info(cache, true);
if (ret) {
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
ret = exclude_super_stripes(cache);
@@ -2308,7 +2327,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
/* We may have excluded something, so call this just in case */
btrfs_free_excluded_extents(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
add_new_free_space(cache, chunk_offset, chunk_offset + size);
@@ -2335,7 +2354,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
if (ret) {
btrfs_remove_free_space_cache(cache);
btrfs_put_block_group(cache);
- return ret;
+ return ERR_PTR(ret);
}
/*
@@ -2354,7 +2373,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
btrfs_update_delayed_refs_rsv(trans);
set_avail_alloc_bits(fs_info, type);
- return 0;
+ return cache;
}
/*
@@ -3232,11 +3251,203 @@ int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type)
return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
+static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ /*
+ * Check if we have enough space in the system space info because we
+ * will need to update device items in the chunk btree and insert a new
+ * chunk item in the chunk btree as well. This will allocate a new
+ * system block group if needed.
+ */
+ check_system_chunk(trans, flags);
+
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ goto out;
+ }
+
+ /*
+ * If this is a system chunk allocation then stop right here and do not
+ * add the chunk item to the chunk btree. This is to prevent a deadlock
+ * because this system chunk allocation can be triggered while COWing
+ * some extent buffer of the chunk btree and while holding a lock on a
+ * parent extent buffer, in which case attempting to insert the chunk
+ * item (or update the device item) would result in a deadlock on that
+ * parent extent buffer. In this case defer the chunk btree updates to
+ * the second phase of chunk allocation and keep our reservation until
+ * the second phase completes.
+ *
+ * This is a rare case and can only be triggered by the very few cases
+ * we have where we need to touch the chunk btree outside chunk allocation
+ * and chunk removal. These cases are basically adding a device, removing
+ * a device or resizing a device.
+ */
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
+ return 0;
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ /*
+ * Normally we are not expected to fail with -ENOSPC here, since we have
+ * previously reserved space in the system space_info and allocated one
+ * new system chunk if necessary. However there are two exceptions:
+ *
+ * 1) We may have enough free space in the system space_info but all the
+ * existing system block groups have a profile which can not be used
+ * for extent allocation.
+ *
+ * This happens when mounting in degraded mode. For example we have a
+ * RAID1 filesystem with 2 devices, lose one device and mount the fs
+ * using the other device in degraded mode. If we then allocate a chunk,
+ * we may have enough free space in the existing system space_info, but
+ * none of the block groups can be used for extent allocation since they
+ * have a RAID1 profile, and because we are in degraded mode with a
+ * single device, we are forced to allocate a new system chunk with a
+ * SINGLE profile. Making check_system_chunk() iterate over all system
+ * block groups and check if they have a usable profile and enough space
+ * can be slow on very large filesystems, so we tolerate the -ENOSPC and
+ * try again after forcing allocation of a new system chunk. Like this
+ * we avoid paying the cost of that search in normal circumstances, when
+ * we were not mounted in degraded mode;
+ *
+ * 2) We had enough free space info the system space_info, and one suitable
+ * block group to allocate from when we called check_system_chunk()
+ * above. However right after we called it, the only system block group
+ * with enough free space got turned into RO mode by a running scrub,
+ * and in this case we have to allocate a new one and retry. We only
+ * need do this allocate and retry once, since we have a transaction
+ * handle and scrub uses the commit root to search for block groups.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(trans->fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+out:
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
+}
+
/*
- * If force is CHUNK_ALLOC_FORCE:
+ * Chunk allocation is done in 2 phases:
+ *
+ * 1) Phase 1 - through btrfs_chunk_alloc() we allocate device extents for
+ * the chunk, the chunk mapping, create its block group and add the items
+ * that belong in the chunk btree to it - more specifically, we need to
+ * update device items in the chunk btree and add a new chunk item to it.
+ *
+ * 2) Phase 2 - through btrfs_create_pending_block_groups(), we add the block
+ * group item to the extent btree and the device extent items to the devices
+ * btree.
+ *
+ * This is done to prevent deadlocks. For example when COWing a node from the
+ * extent btree we are holding a write lock on the node's parent and if we
+ * trigger chunk allocation and attempted to insert the new block group item
+ * in the extent btree right way, we could deadlock because the path for the
+ * insertion can include that parent node. At first glance it seems impossible
+ * to trigger chunk allocation after starting a transaction since tasks should
+ * reserve enough transaction units (metadata space), however while that is true
+ * most of the time, chunk allocation may still be triggered for several reasons:
+ *
+ * 1) When reserving metadata, we check if there is enough free space in the
+ * metadata space_info and therefore don't trigger allocation of a new chunk.
+ * However later when the task actually tries to COW an extent buffer from
+ * the extent btree or from the device btree for example, it is forced to
+ * allocate a new block group (chunk) because the only one that had enough
+ * free space was just turned to RO mode by a running scrub for example (or
+ * device replace, block group reclaim thread, etc), so we can not use it
+ * for allocating an extent and end up being forced to allocate a new one;
+ *
+ * 2) Because we only check that the metadata space_info has enough free bytes,
+ * we end up not allocating a new metadata chunk in that case. However if
+ * the filesystem was mounted in degraded mode, none of the existing block
+ * groups might be suitable for extent allocation due to their incompatible
+ * profile (for e.g. mounting a 2 devices filesystem, where all block groups
+ * use a RAID1 profile, in degraded mode using a single device). In this case
+ * when the task attempts to COW some extent buffer of the extent btree for
+ * example, it will trigger allocation of a new metadata block group with a
+ * suitable profile (SINGLE profile in the example of the degraded mount of
+ * the RAID1 filesystem);
+ *
+ * 3) The task has reserved enough transaction units / metadata space, but when
+ * it attempts to COW an extent buffer from the extent or device btree for
+ * example, it does not find any free extent in any metadata block group,
+ * therefore forced to try to allocate a new metadata block group.
+ * This is because some other task allocated all available extents in the
+ * meanwhile - this typically happens with tasks that don't reserve space
+ * properly, either intentionally or as a bug. One example where this is
+ * done intentionally is fsync, as it does not reserve any transaction units
+ * and ends up allocating a variable number of metadata extents for log
+ * tree extent buffers.
+ *
+ * We also need this 2 phases setup when adding a device to a filesystem with
+ * a seed device - we must create new metadata and system chunks without adding
+ * any of the block group items to the chunk, extent and device btrees. If we
+ * did not do it this way, we would get ENOSPC when attempting to update those
+ * btrees, since all the chunks from the seed device are read-only.
+ *
+ * Phase 1 does the updates and insertions to the chunk btree because if we had
+ * it done in phase 2 and have a thundering herd of tasks allocating chunks in
+ * parallel, we risk having too many system chunks allocated by many tasks if
+ * many tasks reach phase 1 without the previous ones completing phase 2. In the
+ * extreme case this leads to exhaustion of the system chunk array in the
+ * superblock. This is easier to trigger if using a btree node/leaf size of 64K
+ * and with RAID filesystems (so we have more device items in the chunk btree).
+ * This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
+ * the system chunk array due to concurrent allocations") provides more details.
+ *
+ * For allocation of system chunks, we defer the updates and insertions into the
+ * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
+ * if the chunk allocation is triggered while COWing an extent buffer of the
+ * chunk btree, we are holding a lock on the parent of that extent buffer and
+ * doing the chunk btree updates and insertions can require locking that parent.
+ * This is for the very few and rare cases where we update the chunk btree that
+ * are not chunk allocation or chunk removal: adding a device, removing a device
+ * or resizing a device.
+ *
+ * The reservation of system space, done through check_system_chunk(), as well
+ * as all the updates and insertions into the chunk btree must be done while
+ * holding fs_info->chunk_mutex. This is important to guarantee that while COWing
+ * an extent buffer from the chunks btree we never trigger allocation of a new
+ * system chunk, which would result in a deadlock (trying to lock twice an
+ * extent buffer of the chunk btree, first time before triggering the chunk
+ * allocation and the second time during chunk allocation while attempting to
+ * update the chunks btree). The system chunk array is also updated while holding
+ * that mutex. The same logic applies to removing chunks - we must reserve system
+ * space, update the chunk btree and the system chunk array in the superblock
+ * while holding fs_info->chunk_mutex.
+ *
+ * This function, btrfs_chunk_alloc(), belongs to phase 1.
+ *
+ * If @force is CHUNK_ALLOC_FORCE:
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
+ * If @force is NOT CHUNK_ALLOC_FORCE:
* - return 0 if it doesn't need to allocate a new chunk,
* - return 1 if it successfully allocates a chunk,
* - return errors including -ENOSPC otherwise.
@@ -3253,6 +3464,13 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
/* Don't re-enter if we're already allocating a chunk */
if (trans->allocating_chunk)
return -ENOSPC;
+ /*
+ * If we are removing a chunk, don't re-enter or we would deadlock.
+ * System space reservation and system chunk allocation is done by the
+ * chunk remove operation (btrfs_remove_chunk()).
+ */
+ if (trans->removing_chunk)
+ return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
@@ -3316,13 +3534,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
force_metadata_allocation(fs_info);
}
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
+ ret = do_chunk_alloc(trans, flags);
trans->allocating_chunk = false;
spin_lock(&space_info->lock);
@@ -3341,22 +3553,6 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
space_info->chunk_alloc = 0;
spin_unlock(&space_info->lock);
mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
return ret;
}
@@ -3409,14 +3605,31 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *bg;
/*
* Ignore failure to create system chunk. We might end up not
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
+ *
+ * Also, if our caller is allocating a system chunk, do not
+ * attempt to insert the chunk item in the chunk btree, as we
+ * could deadlock on an extent buffer since our caller may be
+ * COWing an extent buffer from the chunk btree.
*/
- ret = btrfs_alloc_chunk(trans, flags);
+ bg = btrfs_alloc_chunk(trans, flags);
+ if (IS_ERR(bg)) {
+ ret = PTR_ERR(bg);
+ } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ /*
+ * If we fail to add the chunk item here, we end up
+ * trying again at phase 2 of chunk allocation, at
+ * btrfs_create_pending_block_groups(). So ignore
+ * any error here.
+ */
+ btrfs_chunk_alloc_add_chunk_item(trans, bg);
+ }
}
if (!ret) {
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 7b927425dc71..c72a71efcb18 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -97,6 +97,7 @@ struct btrfs_block_group {
unsigned int removed:1;
unsigned int to_copy:1;
unsigned int relocating_repair:1;
+ unsigned int chunk_item_inserted:1;
int disk_cache_state;
@@ -268,8 +269,9 @@ void btrfs_reclaim_bgs_work(struct work_struct *work);
void btrfs_reclaim_bgs(struct btrfs_fs_info *fs_info);
void btrfs_mark_bg_to_reclaim(struct btrfs_block_group *bg);
int btrfs_read_block_groups(struct btrfs_fs_info *info);
-int btrfs_make_block_group(struct btrfs_trans_handle *trans, u64 bytes_used,
- u64 type, u64 chunk_offset, u64 size);
+struct btrfs_block_group *btrfs_make_block_group(struct btrfs_trans_handle *trans,
+ u64 bytes_used, u64 type,
+ u64 chunk_offset, u64 size);
void btrfs_create_pending_block_groups(struct btrfs_trans_handle *trans);
int btrfs_inc_block_group_ro(struct btrfs_block_group *cache,
bool do_chunk_alloc);
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 4bc3ca2cbd7d..c5c08c87e130 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -364,49 +364,6 @@ static noinline int update_ref_for_cow(struct btrfs_trans_handle *trans,
return 0;
}
-static struct extent_buffer *alloc_tree_block_no_bg_flush(
- struct btrfs_trans_handle *trans,
- struct btrfs_root *root,
- u64 parent_start,
- const struct btrfs_disk_key *disk_key,
- int level,
- u64 hint,
- u64 empty_size,
- enum btrfs_lock_nesting nest)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct extent_buffer *ret;
-
- /*
- * If we are COWing a node/leaf from the extent, chunk, device or free
- * space trees, make sure that we do not finish block group creation of
- * pending block groups. We do this to avoid a deadlock.
- * COWing can result in allocation of a new chunk, and flushing pending
- * block groups (btrfs_create_pending_block_groups()) can be triggered
- * when finishing allocation of a new chunk. Creation of a pending block
- * group modifies the extent, chunk, device and free space trees,
- * therefore we could deadlock with ourselves since we are holding a
- * lock on an extent buffer that btrfs_create_pending_block_groups() may
- * try to COW later.
- * For similar reasons, we also need to delay flushing pending block
- * groups when splitting a leaf or node, from one of those trees, since
- * we are holding a write lock on it and its parent or when inserting a
- * new root node for one of those trees.
- */
- if (root == fs_info->extent_root ||
- root == fs_info->chunk_root ||
- root == fs_info->dev_root ||
- root == fs_info->free_space_root)
- trans->can_flush_pending_bgs = false;
-
- ret = btrfs_alloc_tree_block(trans, root, parent_start,
- root->root_key.objectid, disk_key, level,
- hint, empty_size, nest);
- trans->can_flush_pending_bgs = true;
-
- return ret;
-}
-
/*
* does the dirty work in cow of a single block. The parent block (if
* supplied) is updated to point to the new cow copy. The new buffer is marked
@@ -455,8 +412,9 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
if ((root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) && parent)
parent_start = parent->start;
- cow = alloc_tree_block_no_bg_flush(trans, root, parent_start, &disk_key,
- level, search_start, empty_size, nest);
+ cow = btrfs_alloc_tree_block(trans, root, parent_start,
+ root->root_key.objectid, &disk_key, level,
+ search_start, empty_size, nest);
if (IS_ERR(cow))
return PTR_ERR(cow);
@@ -2458,9 +2416,9 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
else
btrfs_node_key(lower, &lower_key, 0);
- c = alloc_tree_block_no_bg_flush(trans, root, 0, &lower_key, level,
- root->node->start, 0,
- BTRFS_NESTING_NEW_ROOT);
+ c = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &lower_key, level, root->node->start, 0,
+ BTRFS_NESTING_NEW_ROOT);
if (IS_ERR(c))
return PTR_ERR(c);
@@ -2589,8 +2547,9 @@ static noinline int split_node(struct btrfs_trans_handle *trans,
mid = (c_nritems + 1) / 2;
btrfs_node_key(c, &disk_key, mid);
- split = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, level,
- c->start, 0, BTRFS_NESTING_SPLIT);
+ split = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, level, c->start, 0,
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(split))
return PTR_ERR(split);
@@ -3381,10 +3340,10 @@ static noinline int split_leaf(struct btrfs_trans_handle *trans,
* BTRFS_NESTING_SPLIT_THE_SPLITTENING if we need to, but for now just
* use BTRFS_NESTING_NEW_ROOT.
*/
- right = alloc_tree_block_no_bg_flush(trans, root, 0, &disk_key, 0,
- l->start, 0, num_doubles ?
- BTRFS_NESTING_NEW_ROOT :
- BTRFS_NESTING_SPLIT);
+ right = btrfs_alloc_tree_block(trans, root, 0, root->root_key.objectid,
+ &disk_key, 0, l->start, 0,
+ num_doubles ? BTRFS_NESTING_NEW_ROOT :
+ BTRFS_NESTING_SPLIT);
if (IS_ERR(right))
return PTR_ERR(right);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 443c348bc6f3..14b9fdc8aaa9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -254,8 +254,11 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
}
/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
+ * To be called after doing the chunk btree updates right after allocating a new
+ * chunk (after btrfs_chunk_alloc_add_chunk_item() is called), when removing a
+ * chunk after all chunk btree updates and after finishing the second phase of
+ * chunk allocation (btrfs_create_pending_block_groups()) in case some block
+ * group had its chunk item insertion delayed to the second phase.
*/
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
@@ -264,8 +267,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
if (!trans->chunk_bytes_reserved)
return;
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
trans->chunk_bytes_reserved = 0;
@@ -696,7 +697,6 @@ start_transaction(struct btrfs_root *root, unsigned int num_items,
h->fs_info = root->fs_info;
h->type = type;
- h->can_flush_pending_bgs = true;
INIT_LIST_HEAD(&h->new_bgs);
smp_mb();
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index a18d67796b54..ba45065f9451 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -132,7 +132,7 @@ struct btrfs_trans_handle {
short aborted;
bool adding_csums;
bool allocating_chunk;
- bool can_flush_pending_bgs;
+ bool removing_chunk;
bool reloc_reserved;
bool in_fsync;
struct btrfs_root *root;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 782e16795bc4..c6c14315b1c9 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1745,19 +1745,14 @@ static int btrfs_free_dev_extent(struct btrfs_trans_handle *trans,
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_handle_fs_error(fs_info, ret, "Slot search failed");
goto out;
}
*dev_extent_len = btrfs_dev_extent_length(leaf, extent);
ret = btrfs_del_item(trans, root, path);
- if (ret) {
- btrfs_handle_fs_error(fs_info, ret,
- "Failed to remove dev extent item");
- } else {
+ if (ret == 0)
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
- }
out:
btrfs_free_path(path);
return ret;
@@ -2942,7 +2937,7 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
u32 cur;
struct btrfs_key key;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
array_size = btrfs_super_sys_array_size(super_copy);
ptr = super_copy->sys_chunk_array;
@@ -2972,7 +2967,6 @@ static int btrfs_del_sys_chunk(struct btrfs_fs_info *fs_info, u64 chunk_offset)
cur += len;
}
}
- mutex_unlock(&fs_info->chunk_mutex);
return ret;
}
@@ -3012,6 +3006,29 @@ struct extent_map *btrfs_get_chunk_map(struct btrfs_fs_info *fs_info,
return em;
}
+static int remove_chunk_item(struct btrfs_trans_handle *trans,
+ struct map_lookup *map, u64 chunk_offset)
+{
+ int i;
+
+ /*
+ * Removing chunk items and updating the device items in the chunks btree
+ * requires holding the chunk_mutex.
+ * See the comment at btrfs_chunk_alloc() for the details.
+ */
+ lockdep_assert_held(&trans->fs_info->chunk_mutex);
+
+ for (i = 0; i < map->num_stripes; i++) {
+ int ret;
+
+ ret = btrfs_update_device(trans, map->stripes[i].dev);
+ if (ret)
+ return ret;
+ }
+
+ return btrfs_free_chunk(trans, chunk_offset);
+}
+
int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3032,14 +3049,16 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(em);
}
map = em->map_lookup;
- mutex_lock(&fs_info->chunk_mutex);
- check_system_chunk(trans, map->type);
- mutex_unlock(&fs_info->chunk_mutex);
/*
- * Take the device list mutex to prevent races with the final phase of
- * a device replace operation that replaces the device object associated
- * with map stripes (dev-replace.c:btrfs_dev_replace_finishing()).
+ * First delete the device extent items from the devices btree.
+ * We take the device_list_mutex to avoid racing with the finishing phase
+ * of a device replace operation. See the comment below before acquiring
+ * fs_info->chunk_mutex. Note that here we do not acquire the chunk_mutex
+ * because that can result in a deadlock when deleting the device extent
+ * items from the devices btree - COWing an extent buffer from the btree
+ * may result in allocating a new metadata chunk, which would attempt to
+ * lock again fs_info->chunk_mutex.
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
@@ -3061,18 +3080,73 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
btrfs_clear_space_info_full(fs_info);
mutex_unlock(&fs_info->chunk_mutex);
}
+ }
+ mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_update_device(trans, device);
+ /*
+ * We acquire fs_info->chunk_mutex for 2 reasons:
+ *
+ * 1) Just like with the first phase of the chunk allocation, we must
+ * reserve system space, do all chunk btree updates and deletions, and
+ * update the system chunk array in the superblock while holding this
+ * mutex. This is for similar reasons as explained on the comment at
+ * the top of btrfs_chunk_alloc();
+ *
+ * 2) Prevent races with the final phase of a device replace operation
+ * that replaces the device object associated with the map's stripes,
+ * because the device object's id can change at any time during that
+ * final phase of the device replace operation
+ * (dev-replace.c:btrfs_dev_replace_finishing()), so we could grab the
+ * replaced device and then see it with an ID of
+ * BTRFS_DEV_REPLACE_DEVID, which would cause a failure when updating
+ * the device item, which does not exists on the chunk btree.
+ * The finishing phase of device replace acquires both the
+ * device_list_mutex and the chunk_mutex, in that order, so we are
+ * safe by just acquiring the chunk_mutex.
+ */
+ trans->removing_chunk = true;
+ mutex_lock(&fs_info->chunk_mutex);
+
+ check_system_chunk(trans, map->type);
+
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ /*
+ * Normally we should not get -ENOSPC since we reserved space before
+ * through the call to check_system_chunk().
+ *
+ * Despite our system space_info having enough free space, we may not
+ * be able to allocate extents from its block groups, because all have
+ * an incompatible profile, which will force us to allocate a new system
+ * block group with the right profile, or right after we called
+ * check_system_space() above, a scrub turned the only system block group
+ * with enough free space into RO mode.
+ * This is explained with more detail at do_chunk_alloc().
+ *
+ * So if we get -ENOSPC, allocate a new system chunk and retry once.
+ */
+ if (ret == -ENOSPC) {
+ const u64 sys_flags = btrfs_system_alloc_profile(fs_info);
+ struct btrfs_block_group *sys_bg;
+
+ sys_bg = btrfs_alloc_chunk(trans, sys_flags);
+ if (IS_ERR(sys_bg)) {
+ ret = PTR_ERR(sys_bg);
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+
+ ret = btrfs_chunk_alloc_add_chunk_item(trans, sys_bg);
if (ret) {
- mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
goto out;
}
- }
- mutex_unlock(&fs_devices->device_list_mutex);
- ret = btrfs_free_chunk(trans, chunk_offset);
- if (ret) {
+ ret = remove_chunk_item(trans, map, chunk_offset);
+ if (ret) {
+ btrfs_abort_transaction(trans, ret);
+ goto out;
+ }
+ } else if (ret) {
btrfs_abort_transaction(trans, ret);
goto out;
}
@@ -3087,6 +3161,15 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
}
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+
+ /*
+ * We are done with chunk btree updates and deletions, so release the
+ * system space we previously reserved (with check_system_chunk()).
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+
ret = btrfs_remove_block_group(trans, chunk_offset, em);
if (ret) {
btrfs_abort_transaction(trans, ret);
@@ -3094,6 +3177,10 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
out:
+ if (trans->removing_chunk) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ trans->removing_chunk = false;
+ }
/* once for us */
free_extent_map(em);
return ret;
@@ -4860,13 +4947,12 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
u32 array_size;
u8 *ptr;
- mutex_lock(&fs_info->chunk_mutex);
+ lockdep_assert_held(&fs_info->chunk_mutex);
+
array_size = btrfs_super_sys_array_size(super_copy);
if (array_size + item_size + sizeof(disk_key)
- > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE) {
- mutex_unlock(&fs_info->chunk_mutex);
+ > BTRFS_SYSTEM_CHUNK_ARRAY_SIZE)
return -EFBIG;
- }
ptr = super_copy->sys_chunk_array + array_size;
btrfs_cpu_key_to_disk(&disk_key, key);
@@ -4875,7 +4961,6 @@ static int btrfs_add_system_chunk(struct btrfs_fs_info *fs_info,
memcpy(ptr, chunk, item_size);
item_size += sizeof(disk_key);
btrfs_set_super_sys_array_size(super_copy, array_size + item_size);
- mutex_unlock(&fs_info->chunk_mutex);
return 0;
}
@@ -5225,13 +5310,14 @@ static int decide_stripe_size(struct btrfs_fs_devices *fs_devices,
}
}
-static int create_chunk(struct btrfs_trans_handle *trans,
+static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
{
struct btrfs_fs_info *info = trans->fs_info;
struct map_lookup *map = NULL;
struct extent_map_tree *em_tree;
+ struct btrfs_block_group *block_group;
struct extent_map *em;
u64 start = ctl->start;
u64 type = ctl->type;
@@ -5241,7 +5327,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
map = kmalloc(map_lookup_size(ctl->num_stripes), GFP_NOFS);
if (!map)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
map->num_stripes = ctl->num_stripes;
for (i = 0; i < ctl->ndevs; ++i) {
@@ -5263,7 +5349,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
em = alloc_extent_map();
if (!em) {
kfree(map);
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
set_bit(EXTENT_FLAG_FS_MAPPING, &em->flags);
em->map_lookup = map;
@@ -5279,12 +5365,12 @@ static int create_chunk(struct btrfs_trans_handle *trans,
if (ret) {
write_unlock(&em_tree->lock);
free_extent_map(em);
- return ret;
+ return ERR_PTR(ret);
}
write_unlock(&em_tree->lock);
- ret = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
- if (ret)
+ block_group = btrfs_make_block_group(trans, 0, type, start, ctl->chunk_size);
+ if (IS_ERR(block_group))
goto error_del_extent;
for (i = 0; i < map->num_stripes; i++) {
@@ -5304,7 +5390,7 @@ static int create_chunk(struct btrfs_trans_handle *trans,
check_raid56_incompat_flag(info, type);
check_raid1c34_incompat_flag(info, type);
- return 0;
+ return block_group;
error_del_extent:
write_lock(&em_tree->lock);
@@ -5316,34 +5402,36 @@ static int create_chunk(struct btrfs_trans_handle *trans,
/* One for the tree reference */
free_extent_map(em);
- return ret;
+ return block_group;
}
-int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
+struct btrfs_block_group *btrfs_alloc_chunk(struct btrfs_trans_handle *trans,
+ u64 type)
{
struct btrfs_fs_info *info = trans->fs_info;
struct btrfs_fs_devices *fs_devices = info->fs_devices;
struct btrfs_device_info *devices_info = NULL;
struct alloc_chunk_ctl ctl;
+ struct btrfs_block_group *block_group;
int ret;
lockdep_assert_held(&info->chunk_mutex);
if (!alloc_profile_is_valid(type, 0)) {
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (list_empty(&fs_devices->alloc_list)) {
if (btrfs_test_opt(info, ENOSPC_DEBUG))
btrfs_debug(info, "%s: no writable device", __func__);
- return -ENOSPC;
+ return ERR_PTR(-ENOSPC);
}
if (!(type & BTRFS_BLOCK_GROUP_TYPE_MASK)) {
btrfs_err(info, "invalid chunk type 0x%llx requested", type);
ASSERT(0);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
ctl.start = find_next_chunk(info);
@@ -5353,46 +5441,43 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type)
devices_info = kcalloc(fs_devices->rw_devices, sizeof(*devices_info),
GFP_NOFS);
if (!devices_info)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = gather_device_info(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
ret = decide_stripe_size(fs_devices, &ctl, devices_info);
- if (ret < 0)
+ if (ret < 0) {
+ block_group = ERR_PTR(ret);
goto out;
+ }
- ret = create_chunk(trans, &ctl, devices_info);
+ block_group = create_chunk(trans, &ctl, devices_info);
out:
kfree(devices_info);
- return ret;
+ return block_group;
}
/*
- * Chunk allocation falls into two parts. The first part does work
]
]
|
| 379/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 31192
output: 42
thoughts: 1466
Thoughts: **Analyzing the Change in `btrfs_remove_chunk`**
Okay, so looking at this commit `79bd37120b14`, it seems like `check_system_chunk` has been moved to a new position within the `btrfs_remove_chunk` function. Specifically, it's now called *after* the `btrfs_remove_dev_extents` operation, which, as I recall, handles deleting device extent items. That doesn't seem right to me.
I remember reading a bug report about this and I seem to recall that this change is likely the root cause of an issue. The problem, as I understand it, is that by calling `check_system_chunk` *after* deleting the device extents, the system might not be aware of available space on the device because those extents are only freed within the current transaction. This can lead to `-ENOSPC` errors down the line if the device is otherwise full.
The report also pointed out that `check_system_chunk` would ignore any failure to allocate a system chunk and wouldn't reserve the needed space. And that this is problematic because if the `remove_chunk_item` function then fails because it needs to COW a node in the chunk tree and there's no space, the transaction aborts, and the filesystem will be left in an inconsistent state. Since the device extents were already removed before this point, the chunk item would still be present. This seems like a potentially nasty data corruption bug.
Let me just quickly confirm that `btrfs_remove_dev_extents` *wasn't* extracted from the loop, and that `btrfs_free_dev_extent` does the work in that loop.
Right, yes, the loop calls `btrfs_free_dev_extent` which does the actual deletion work.
Okay, that confirms my initial assessment. Moving `check_system_chunk` after the device extent deletion introduced this subtle but critical error. I need to keep an eye on these types of changes in the future!
|
| 380/2 |
2026/08/08 04:47 |
tool |
git-log |
0m
Args: map[CodeRegexp:btrfs_remove_dev_extents PathPrefix:fs/btrfs/volumes.c]
Results: map[Output:979e1dc3d69e btrfs: handle deletions from remapped block group
]
|
| 381/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 33181
output: 26
thoughts: 17
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm the expert and considering that string of characters.
**Analyzing the Hash**
Alright, let's break down this gibberish: `979e1dc3d69e`. My initial thought is it's highly likely this is a hexadecimal representation. Given its length, it's probably either a short hash (potentially MD5 if I'm being optimistic) or a portion of a larger hash, perhaps SHA-1 or even something more robust truncated for some reason. The "e" at the end makes me lean towards a hexadecimal representation used in crypto or system data. Now, the context is important here. Without more information, I'm just guessing. Is this related to data, a file, a commit, a password, or perhaps a unique identifier in some system I'm familiar with? I need to know the origin to analyze it effectively. My next step would be to try and determine its algorithm/purpose: I'll use tools to help decipher the hash, assuming that I have some context to go on.
|
| 382/2 |
2026/08/08 04:47 |
tool |
git-show |
0m
Args:
Results: map[Output:
Full output is too long, showing 1000 out of 1029 lines.
[commit 979e1dc3d69e4c825eec05d05d9567b251f6ec23
Author: Mark Harmstone <mark@harmstone.com>
Date: Wed Jan 7 14:09:10 2026 +0000
btrfs: handle deletions from remapped block group
Handle the case where we free an extent from a block group that has the
REMAPPED flag set. Because the remap tree is orthogonal to the extent
tree, for data this may be within any number of identity remaps or
actual remaps. If we're freeing a metadata node, this will be wholly
inside one or the other.
btrfs_remove_extent_from_remap_tree() searches the remap tree for the
remaps that cover the range in question, then calls
remove_range_from_remap_tree() for each one, to punch a hole in the
remap and adjust the free-space tree.
For an identity remap, remove_range_from_remap_tree() will adjust the
block group's `identity_remap_count` if this changes. If it reaches
zero we mark the block group as fully remapped.
For an identity remap, remove_range_from_remap_tree() will adjust the
block group's `identity_remap_count` if this changes. If it reaches
zero we mark the block group as fully remapped.
Fully remapped block groups have their chunk stripes removed and their
device extents freed, which makes the disk space available again to the
chunk allocator. This happens asynchronously: in the cleaner thread for
sync discard and nodiscard, and (in a later patch) in the discard worker
for async discard.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a1ab513fa8ea..3b8a750d8519 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -1067,6 +1067,24 @@ static int remove_block_group_item(struct btrfs_trans_handle *trans,
return btrfs_del_item(trans, root, path);
}
+void btrfs_remove_bg_from_sinfo(struct btrfs_block_group *bg)
+{
+ int factor = btrfs_bg_type_to_factor(bg->flags);
+
+ spin_lock(&bg->space_info->lock);
+ if (btrfs_test_opt(bg->fs_info, ENOSPC_DEBUG)) {
+ WARN_ON(bg->space_info->total_bytes < bg->length);
+ WARN_ON(bg->space_info->bytes_readonly < bg->length - bg->zone_unusable);
+ WARN_ON(bg->space_info->bytes_zone_unusable < bg->zone_unusable);
+ WARN_ON(bg->space_info->disk_total < bg->length * factor);
+ }
+ bg->space_info->total_bytes -= bg->length;
+ bg->space_info->bytes_readonly -= (bg->length - bg->zone_unusable);
+ btrfs_space_info_update_bytes_zone_unusable(bg->space_info, -bg->zone_unusable);
+ bg->space_info->disk_total -= bg->length * factor;
+ spin_unlock(&bg->space_info->lock);
+}
+
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct btrfs_chunk_map *map)
{
@@ -1078,7 +1096,6 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct kobject *kobj = NULL;
int ret;
int index;
- int factor;
struct btrfs_caching_control *caching_ctl = NULL;
bool remove_map;
bool remove_rsv = false;
@@ -1087,7 +1104,7 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
if (!block_group)
return -ENOENT;
- BUG_ON(!block_group->ro);
+ BUG_ON(!block_group->ro && !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED));
trace_btrfs_remove_block_group(block_group);
/*
@@ -1099,7 +1116,6 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
block_group->length);
index = btrfs_bg_flags_to_raid_index(block_group->flags);
- factor = btrfs_bg_type_to_factor(block_group->flags);
/* make sure this block group isn't part of an allocation cluster */
cluster = &fs_info->data_alloc_cluster;
@@ -1223,26 +1239,11 @@ int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
spin_lock(&block_group->space_info->lock);
list_del_init(&block_group->ro_list);
-
- if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
- WARN_ON(block_group->space_info->total_bytes
- < block_group->length);
- WARN_ON(block_group->space_info->bytes_readonly
- < block_group->length - block_group->zone_unusable);
- WARN_ON(block_group->space_info->bytes_zone_unusable
- < block_group->zone_unusable);
- WARN_ON(block_group->space_info->disk_total
- < block_group->length * factor);
- }
- block_group->space_info->total_bytes -= block_group->length;
- block_group->space_info->bytes_readonly -=
- (block_group->length - block_group->zone_unusable);
- btrfs_space_info_update_bytes_zone_unusable(block_group->space_info,
- -block_group->zone_unusable);
- block_group->space_info->disk_total -= block_group->length * factor;
-
spin_unlock(&block_group->space_info->lock);
+ if (!(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED))
+ btrfs_remove_bg_from_sinfo(block_group);
+
/*
* Remove the free space for the block group from the free space tree
* and the block group's item from the extent tree before marking the
@@ -1575,8 +1576,10 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
spin_lock(&space_info->lock);
spin_lock(&block_group->lock);
- if (btrfs_is_block_group_used(block_group) || block_group->ro ||
- list_is_singular(&block_group->list)) {
+ if (btrfs_is_block_group_used(block_group) ||
+ (block_group->ro && !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED)) ||
+ list_is_singular(&block_group->list) ||
+ test_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &block_group->runtime_flags)) {
/*
* We want to bail if we made new allocations or have
* outstanding allocations in this block group. We do
@@ -1617,9 +1620,10 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info)
* needing to allocate extents from the block group.
*/
used = btrfs_space_info_used(space_info, true);
- if ((space_info->total_bytes - block_group->length < used &&
- block_group->zone_unusable < block_group->length) ||
- has_unwritten_metadata(block_group)) {
+ if (((space_info->total_bytes - block_group->length < used &&
+ block_group->zone_unusable < block_group->length) ||
+ has_unwritten_metadata(block_group)) &&
+ !(block_group->flags & BTRFS_BLOCK_GROUP_REMAPPED)) {
/*
* Add a reference for the list, compensate for the ref
* drop under the "next" label for the
@@ -1784,6 +1788,9 @@ void btrfs_mark_bg_unused(struct btrfs_block_group *bg)
btrfs_get_block_group(bg);
trace_btrfs_add_unused_block_group(bg);
list_add_tail(&bg->bg_list, &fs_info->unused_bgs);
+ } else if (bg->flags & BTRFS_BLOCK_GROUP_REMAPPED &&
+ bg->identity_remap_count == 0) {
+ /* Leave fully remapped block groups on the fully_remapped_bgs list. */
} else if (!test_bit(BLOCK_GROUP_FLAG_NEW, &bg->runtime_flags)) {
/* Pull out the block group from the reclaim_bgs list. */
trace_btrfs_add_unused_block_group(bg);
@@ -4581,6 +4588,13 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
list_del_init(&block_group->bg_list);
btrfs_put_block_group(block_group);
}
+
+ while (!list_empty(&info->fully_remapped_bgs)) {
+ block_group = list_first_entry(&info->fully_remapped_bgs,
+ struct btrfs_block_group, bg_list);
+ list_del_init(&block_group->bg_list);
+ btrfs_put_block_group(block_group);
+ }
spin_unlock(&info->unused_bgs_lock);
spin_lock(&info->zone_active_bgs_lock);
@@ -4768,3 +4782,24 @@ bool btrfs_block_group_should_use_size_class(const struct btrfs_block_group *bg)
return false;
return true;
}
+
+void btrfs_mark_bg_fully_remapped(struct btrfs_block_group *bg,
+ struct btrfs_trans_handle *trans)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ /*
+ * The block group might already be on the unused_bgs list, remove it
+ * if it is. It'll get readded after the async discard worker finishes,
+ * or in btrfs_handle_fully_remapped_bgs() if we're not using async
+ * discard.
+ */
+ if (!list_empty(&bg->bg_list))
+ list_del(&bg->bg_list);
+ else
+ btrfs_get_block_group(bg);
+
+ list_add_tail(&bg->bg_list, &fs_info->fully_remapped_bgs);
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index ecabb1a9fc0e..f5c15c7f6cc7 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -92,6 +92,7 @@ enum btrfs_block_group_flags {
* transaction.
*/
BLOCK_GROUP_FLAG_NEW,
+ BLOCK_GROUP_FLAG_FULLY_REMAPPED,
};
enum btrfs_caching_type {
@@ -340,6 +341,7 @@ int btrfs_add_new_free_space(struct btrfs_block_group *block_group,
struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
struct btrfs_fs_info *fs_info,
const u64 chunk_offset);
+void btrfs_remove_bg_from_sinfo(struct btrfs_block_group *bg);
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
struct btrfs_chunk_map *map);
void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info);
@@ -411,5 +413,7 @@ int btrfs_use_block_group_size_class(struct btrfs_block_group *bg,
enum btrfs_block_group_size_class size_class,
bool force_wrong_size_class);
bool btrfs_block_group_should_use_size_class(const struct btrfs_block_group *bg);
+void btrfs_mark_bg_fully_remapped(struct btrfs_block_group *bg,
+ struct btrfs_trans_handle *trans);
#endif /* BTRFS_BLOCK_GROUP_H */
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index c69734c74c26..627282613eee 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1473,6 +1473,10 @@ static int cleaner_kthread(void *arg)
*/
btrfs_run_defrag_inodes(fs_info);
+ if (btrfs_fs_incompat(fs_info, REMAP_TREE) &&
+ !btrfs_test_opt(fs_info, DISCARD_ASYNC))
+ btrfs_handle_fully_remapped_bgs(fs_info);
+
/*
* Acquires fs_info->reclaim_bgs_lock to avoid racing
* with relocation (btrfs_relocate_chunk) and relocation
@@ -2807,6 +2811,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
INIT_LIST_HEAD(&fs_info->tree_mod_seq_list);
INIT_LIST_HEAD(&fs_info->unused_bgs);
INIT_LIST_HEAD(&fs_info->reclaim_bgs);
+ INIT_LIST_HEAD(&fs_info->fully_remapped_bgs);
INIT_LIST_HEAD(&fs_info->zone_active_bgs);
#ifdef CONFIG_BTRFS_DEBUG
INIT_LIST_HEAD(&fs_info->allocated_roots);
@@ -2862,6 +2867,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
mutex_init(&fs_info->chunk_mutex);
mutex_init(&fs_info->transaction_kthread_mutex);
mutex_init(&fs_info->cleaner_mutex);
+ mutex_init(&fs_info->remap_mutex);
mutex_init(&fs_info->ro_block_group_mutex);
init_rwsem(&fs_info->commit_root_sem);
init_rwsem(&fs_info->cleanup_work_sem);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index ce4bda1f37ad..5e3e9f18b263 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -41,6 +41,7 @@
#include "tree-checker.h"
#include "raid-stripe-tree.h"
#include "delayed-inode.h"
+#include "relocation.h"
#undef SCRAMBLE_DELAYED_REFS
@@ -2848,6 +2849,73 @@ static int unpin_extent_range(struct btrfs_fs_info *fs_info,
return 0;
}
+/*
+ * Complete the remapping of a block group by removing its chunk stripes and
+ * device extents, and adding it to the unused list if there's no longer any
+ * extents nominally within it.
+ */
+int btrfs_complete_bg_remapping(struct btrfs_block_group *bg)
+{
+ struct btrfs_fs_info *fs_info = bg->fs_info;
+ struct btrfs_chunk_map *map;
+ int ret;
+
+ map = btrfs_get_chunk_map(fs_info, bg->start, 1);
+ if (IS_ERR(map))
+ return PTR_ERR(map);
+
+ ret = btrfs_last_identity_remap_gone(map, bg);
+ if (ret) {
+ btrfs_free_chunk_map(map);
+ return ret;
+ }
+
+ /*
+ * Set num_stripes to 0, so that btrfs_remove_dev_extents() won't run a
+ * second time.
+ */
+ map->num_stripes = 0;
+
+ btrfs_free_chunk_map(map);
+
+ if (bg->used == 0) {
+ spin_lock(&fs_info->unused_bgs_lock);
+ if (!list_empty(&bg->bg_list)) {
+ list_del_init(&bg->bg_list);
+ btrfs_put_block_group(bg);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+
+ btrfs_mark_bg_unused(bg);
+ }
+
+ return 0;
+}
+
+void btrfs_handle_fully_remapped_bgs(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_block_group *bg;
+ int ret;
+
+ spin_lock(&fs_info->unused_bgs_lock);
+ while (!list_empty(&fs_info->fully_remapped_bgs)) {
+ bg = list_first_entry(&fs_info->fully_remapped_bgs,
+ struct btrfs_block_group, bg_list);
+ list_del_init(&bg->bg_list);
+ spin_unlock(&fs_info->unused_bgs_lock);
+
+ ret = btrfs_complete_bg_remapping(bg);
+ if (ret) {
+ btrfs_put_block_group(bg);
+ return;
+ }
+
+ btrfs_put_block_group(bg);
+ spin_lock(&fs_info->unused_bgs_lock);
+ }
+ spin_unlock(&fs_info->unused_bgs_lock);
+}
+
int btrfs_finish_extent_commit(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
@@ -3000,11 +3068,22 @@ u64 btrfs_get_extent_owner_root(struct btrfs_fs_info *fs_info,
}
static int do_free_extent_accounting(struct btrfs_trans_handle *trans,
- u64 bytenr, struct btrfs_squota_delta *delta)
+ u64 bytenr, struct btrfs_squota_delta *delta,
+ struct btrfs_path *path)
{
int ret;
+ bool remapped = false;
u64 num_bytes = delta->num_bytes;
+ /* Returns 1 on success and 0 on no-op. */
+ ret = btrfs_remove_extent_from_remap_tree(trans, path, bytenr, num_bytes);
+ if (unlikely(ret < 0)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ } else if (ret == 1) {
+ remapped = true;
+ }
+
if (delta->is_data) {
struct btrfs_root *csum_root;
@@ -3028,10 +3107,13 @@ static int do_free_extent_accounting(struct btrfs_trans_handle *trans,
return ret;
}
- ret = btrfs_add_to_free_space_tree(trans, bytenr, num_bytes);
- if (unlikely(ret)) {
- btrfs_abort_transaction(trans, ret);
- return ret;
+ /* If remapped, FST has already been taken care of in remove_range_from_remap_tree(). */
+ if (!remapped) {
+ ret = btrfs_add_to_free_space_tree(trans, bytenr, num_bytes);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
}
ret = btrfs_update_block_group(trans, bytenr, num_bytes, false);
@@ -3390,7 +3472,7 @@ static int __btrfs_free_extent(struct btrfs_trans_handle *trans,
}
btrfs_release_path(path);
- ret = do_free_extent_accounting(trans, bytenr, &delta);
+ ret = do_free_extent_accounting(trans, bytenr, &delta, path);
}
btrfs_release_path(path);
diff --git a/fs/btrfs/extent-tree.h b/fs/btrfs/extent-tree.h
index 71bb8109c969..d7b6aeb63656 100644
--- a/fs/btrfs/extent-tree.h
+++ b/fs/btrfs/extent-tree.h
@@ -163,5 +163,7 @@ void btrfs_error_unpin_extent_range(struct btrfs_fs_info *fs_info, u64 start, u6
int btrfs_discard_extent(struct btrfs_fs_info *fs_info, u64 bytenr,
u64 num_bytes, u64 *actual_bytes);
int btrfs_trim_fs(struct btrfs_fs_info *fs_info, struct fstrim_range *range);
+void btrfs_handle_fully_remapped_bgs(struct btrfs_fs_info *fs_info);
+int btrfs_complete_bg_remapping(struct btrfs_block_group *bg);
#endif
diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h
index 13b0aa0b9da9..5bbc993a66e1 100644
--- a/fs/btrfs/fs.h
+++ b/fs/btrfs/fs.h
@@ -587,6 +587,7 @@ struct btrfs_fs_info {
struct mutex transaction_kthread_mutex;
struct mutex cleaner_mutex;
struct mutex chunk_mutex;
+ struct mutex remap_mutex;
/*
* This is taken to make sure we don't set block groups ro after the
@@ -840,10 +841,11 @@ struct btrfs_fs_info {
struct list_head reclaim_bgs;
int bg_reclaim_threshold;
- /* Protects the lists unused_bgs and reclaim_bgs. */
+ /* Protects the lists unused_bgs, reclaim_bgs, and fully_remapped_bgs. */
spinlock_t unused_bgs_lock;
/* Protected by unused_bgs_lock. */
struct list_head unused_bgs;
+ struct list_head fully_remapped_bgs;
struct mutex unused_bg_unpin_mutex;
/* Protect block groups that are going to be deleted */
struct mutex reclaim_bgs_lock;
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 6de508323dbd..e0558b2cd0b4 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -37,6 +37,7 @@
#include "super.h"
#include "tree-checker.h"
#include "raid-stripe-tree.h"
+#include "free-space-tree.h"
/*
* Relocation overview
@@ -3859,6 +3860,177 @@ static const char *stage_to_string(enum reloc_stage stage)
return "unknown";
}
+static void adjust_block_group_remap_bytes(struct btrfs_trans_handle *trans,
+ struct btrfs_block_group *bg, s64 diff)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ bool bg_already_dirty = true;
+ bool mark_unused = false;
+
+ spin_lock(&bg->lock);
+ bg->remap_bytes += diff;
+ if (bg->used == 0 && bg->remap_bytes == 0)
+ mark_unused = true;
+ spin_unlock(&bg->lock);
+
+ if (mark_unused)
+ btrfs_mark_bg_unused(bg);
+
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ if (list_empty(&bg->dirty_list)) {
+ list_add_tail(&bg->dirty_list, &trans->transaction->dirty_bgs);
+ bg_already_dirty = false;
+ btrfs_get_block_group(bg);
+ }
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+
+ /* Modified block groups are accounted for in the delayed_refs_rsv. */
+ if (!bg_already_dirty)
+ btrfs_inc_delayed_refs_rsv_bg_updates(fs_info);
+}
+
+static int remove_chunk_stripes(struct btrfs_trans_handle *trans,
+ struct btrfs_chunk_map *chunk_map,
+ struct btrfs_path *path)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key;
+ struct extent_buffer *leaf;
+ struct btrfs_chunk *chunk;
+ int ret;
+
+ key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
+ key.type = BTRFS_CHUNK_ITEM_KEY;
+ key.offset = chunk_map->start;
+
+ btrfs_reserve_chunk_metadata(trans, false);
+
+ ret = btrfs_search_slot(trans, fs_info->chunk_root, &key, path, 0, 1);
+ if (ret) {
+ if (ret == 1) {
+ btrfs_release_path(path);
+ ret = -ENOENT;
+ }
+ btrfs_trans_release_chunk_metadata(trans);
+ return ret;
+ }
+
+ leaf = path->nodes[0];
+
+ chunk = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_chunk);
+ btrfs_set_chunk_num_stripes(leaf, chunk, 0);
+ btrfs_set_chunk_sub_stripes(leaf, chunk, 0);
+
+ btrfs_truncate_item(trans, path, offsetof(struct btrfs_chunk, stripe), 1);
+
+ btrfs_mark_buffer_dirty(trans, leaf);
+
+ btrfs_release_path(path);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return 0;
+}
+
+int btrfs_last_identity_remap_gone(struct btrfs_chunk_map *chunk_map,
+ struct btrfs_block_group *bg)
+{
+ struct btrfs_fs_info *fs_info = bg->fs_info;
+ struct btrfs_trans_handle *trans;
+ int ret;
+ unsigned int num_items;
+ BTRFS_PATH_AUTO_FREE(path);
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ /*
+ * One item for each entry we're removing in the dev extents tree, and
+ * another for each device. DUP chunks are all on one device,
+ * everything else has one device per stripe.
+ */
+ if (bg->flags & BTRFS_BLOCK_GROUP_DUP)
+ num_items = chunk_map->num_stripes + 1;
+ else
+ num_items = 2 * chunk_map->num_stripes;
+
+ trans = btrfs_start_transaction_fallback_global_rsv(fs_info->tree_root, num_items);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+
+ ret = btrfs_remove_dev_extents(trans, chunk_map);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+
+ mutex_lock(&trans->fs_info->chunk_mutex);
+ for (unsigned int i = 0; i < chunk_map->num_stripes; i++) {
+ ret = btrfs_update_device(trans, chunk_map->stripes[i].dev);
+ if (unlikely(ret)) {
+ mutex_unlock(&trans->fs_info->chunk_mutex);
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+ }
+ mutex_unlock(&trans->fs_info->chunk_mutex);
+
+ write_lock(&trans->fs_info->mapping_tree_lock);
+ btrfs_chunk_map_device_clear_bits(chunk_map, CHUNK_ALLOCATED);
+ write_unlock(&trans->fs_info->mapping_tree_lock);
+
+ btrfs_remove_bg_from_sinfo(bg);
+
+ ret = remove_chunk_stripes(trans, chunk_map, path);
+ if (unlikely(ret)) {
+ btrfs_abort_transaction(trans, ret);
+ return ret;
+ }
+
+ ret = btrfs_commit_transaction(trans);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void adjust_identity_remap_count(struct btrfs_trans_handle *trans,
+ struct btrfs_block_group *bg, int delta)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ bool bg_already_dirty = true;
+ bool mark_fully_remapped = false;
+
+ WARN_ON(delta < 0 && -delta > bg->identity_remap_count);
+
+ spin_lock(&bg->lock);
+
+ bg->identity_remap_count += delta;
+
+ if (bg->identity_remap_count == 0 &&
+ !test_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &bg->runtime_flags)) {
+ set_bit(BLOCK_GROUP_FLAG_FULLY_REMAPPED, &bg->runtime_flags);
+ mark_fully_remapped = true;
+ }
+
+ spin_unlock(&bg->lock);
+
+ spin_lock(&trans->transaction->dirty_bgs_lock);
+ if (list_empty(&bg->dirty_list)) {
+ list_add_tail(&bg->dirty_list, &trans->transaction->dirty_bgs);
+ bg_already_dirty = false;
+ btrfs_get_block_group(bg);
+ }
+ spin_unlock(&trans->transaction->dirty_bgs_lock);
+
+ /* Modified block groups are accounted for in the delayed_refs_rsv. */
+ if (!bg_already_dirty)
+ btrfs_inc_delayed_refs_rsv_bg_updates(fs_info);
+
+ if (mark_fully_remapped)
+ btrfs_mark_bg_fully_remapped(bg, trans);
+}
+
int btrfs_translate_remap(struct btrfs_fs_info *fs_info, u64 *logical, u64 *length)
{
int ret;
@@ -4463,3 +4635,260 @@ u64 btrfs_get_reloc_bg_bytenr(const struct btrfs_fs_info *fs_info)
logical = fs_info->reloc_ctl->block_group->start;
return logical;
}
+
+static int insert_remap_item(struct btrfs_trans_handle *trans, struct btrfs_path *path,
+ u64 old_addr, u64 length, u64 new_addr)
+{
+ int ret;
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key;
+ struct btrfs_remap_item remap = { 0 };
+
+ if (old_addr == new_addr) {
+ /* Add new identity remap item. */
+ key.objectid = old_addr;
+ key.type = BTRFS_IDENTITY_REMAP_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root, path,
+ &key, 0);
+ if (ret)
+ return ret;
+ } else {
+ /* Add new remap item. */
+ key.objectid = old_addr;
+ key.type = BTRFS_REMAP_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root,
+ path, &key, sizeof(struct btrfs_remap_item));
+ if (ret)
+ return ret;
+
+ btrfs_set_stack_remap_address(&remap, new_addr);
+
+ write_extent_buffer(path->nodes[0], &remap,
+ btrfs_item_ptr_offset(path->nodes[0], path->slots[0]),
+ sizeof(struct btrfs_remap_item));
+
+ btrfs_release_path(path);
+
+ /* Add new backref item. */
+ key.objectid = new_addr;
+ key.type = BTRFS_REMAP_BACKREF_KEY;
+ key.offset = length;
+
+ ret = btrfs_insert_empty_item(trans, fs_info->remap_root,
+ path, &key,
+ sizeof(struct btrfs_remap_item));
+ if (ret)
+ return ret;
+
+ btrfs_set_stack_remap_address(&remap, old_addr);
+
+ write_extent_buffer(path->nodes[0], &remap,
+ btrfs_item_ptr_offset(path->nodes[0], path->slots[0]),
+ sizeof(struct btrfs_remap_item));
+ }
+
+ btrfs_release_path(path);
+
+ return 0;
+}
+
+/*
+ * Punch a hole in the remap item or identity remap item pointed to by path,
+ * for the range [hole_start, hole_start + hole_length).
+ */
+static int remove_range_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ struct btrfs_block_group *bg,
+ u64 hole_start, u64 hole_length)
+{
+ int ret;
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct extent_buffer *leaf = path->nodes[0];
+ struct btrfs_key key;
+ u64 hole_end, new_addr, remap_start, remap_length, remap_end;
+ u64 overlap_length;
+ bool is_identity_remap;
+ int identity_count_delta = 0;
+
+ hole_end = hole_start + hole_length;
+
+ btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
+
+ is_identity_remap = (key.type == BTRFS_IDENTITY_REMAP_KEY);
+
+ remap_start = key.objectid;
+ remap_length = key.offset;
+ remap_end = remap_start + remap_length;
+
+ if (is_identity_remap) {
+ new_addr = remap_start;
+ } else {
+ struct btrfs_remap_item *remap_ptr;
+
+ remap_ptr = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_remap_item);
+ new_addr = btrfs_remap_address(leaf, remap_ptr);
+ }
+
+ /* Delete old item. */
+ ret = btrfs_del_item(trans, fs_info->remap_root, path);
+ btrfs_release_path(path);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap) {
+ identity_count_delta = -1;
+ } else {
+ /* Remove backref. */
+ key.objectid = new_addr;
+ key.type = BTRFS_REMAP_BACKREF_KEY;
+ key.offset = remap_length;
+
+ ret = btrfs_search_slot(trans, fs_info->remap_root, &key, path, -1, 1);
+ if (ret) {
+ if (ret == 1) {
+ btrfs_release_path(path);
+ ret = -ENOENT;
+ }
+ return ret;
+ }
+
+ ret = btrfs_del_item(trans, fs_info->remap_root, path);
+
+ btrfs_release_path(path);
+
+ if (ret)
+ return ret;
+ }
+
+ /* If hole_start > remap_start, re-add the start of the remap item. */
+ if (hole_start > remap_start) {
+ ret = insert_remap_item(trans, path, remap_start,
+ hole_start - remap_start, new_addr);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap)
+ identity_count_delta++;
+ }
+
+ /* If hole_end < remap_end, re-add the end of the remap item. */
+ if (hole_end < remap_end) {
+ ret = insert_remap_item(trans, path, hole_end,
+ remap_end - hole_end,
+ hole_end - remap_start + new_addr);
+ if (ret)
+ return ret;
+
+ if (is_identity_remap)
+ identity_count_delta++;
+ }
+
+ if (identity_count_delta != 0)
+ adjust_identity_remap_count(trans, bg, identity_count_delta);
+
+ overlap_length = min_t(u64, hole_end, remap_end) -
+ max_t(u64, hole_start, remap_start);
+
+ if (!is_identity_remap) {
+ struct btrfs_block_group *dest_bg;
+
+ dest_bg = btrfs_lookup_block_group(fs_info, new_addr);
+ adjust_block_group_remap_bytes(trans, dest_bg, -overlap_length);
+ btrfs_put_block_group(dest_bg);
+ ret = btrfs_add_to_free_space_tree(trans,
+ hole_start - remap_start + new_addr,
+ overlap_length);
+ if (ret)
+ return ret;
+ }
+
+ ret = overlap_length;
+
+ return ret;
+}
+
+/*
+ * Return 1 if remove_range_from_remap_tree() has been called successfully,
+ * 0 if block group wasn't remapped, and a negative number on error.
+ */
+int btrfs_remove_extent_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ u64 bytenr, u64 num_bytes)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_key key, found_key;
+ struct extent_buffer *leaf;
+ struct btrfs_block_group *bg;
+ int ret, length;
+
+ if (!(btrfs_super_incompat_flags(fs_info->super_copy) &
+ BTRFS_FEATURE_INCOMPAT_REMAP_TREE))
+ return 0;
+
+ bg = btrfs_lookup_block_group(fs_info, bytenr);
+ if (!bg)
+ return 0;
+
+ mutex_lock(&fs_info->remap_mutex);
+
+ if (!(bg->flags & BTRFS_BLOCK_GROUP_REMAPPED)) {
+ mutex_unlock(&fs_info->remap_mutex);
+ btrfs_put_block_group(bg);
+ return 0;
+ }
+
+ do {
+ key.objectid = bytenr;
+ key.type = (u8)-1;
+ key.offset = (u64)-1;
+
+ ret = btrfs_search_slot(trans, fs_info->remap_root, &key, path, -1, 1);
+ if (ret < 0)
+ goto end;
+
+ leaf = path->nodes[0];
+ if (path->slots[0] == 0) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ path->slots[0]--;
+
+ btrfs_item_key_to_cpu(leaf, &found_key, path->slots[0]);
+
+ if (found_key.type != BTRFS_IDENTITY_REMAP_KEY &&
+ found_key.type != BTRFS_REMAP_KEY) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ if (bytenr < found_key.objectid ||
+ bytenr >= found_key.objectid + found_key.offset) {
+ ret = -ENOENT;
+ goto end;
+ }
+
+ length = remove_range_from_remap_tree(trans, path, bg, bytenr, num_bytes);
+ if (length < 0) {
+ ret = length;
+ goto end;
+ }
+
+ bytenr += length;
+ num_bytes -= length;
+ } while (num_bytes > 0);
+
+ ret = 1;
+
+end:
+ mutex_unlock(&fs_info->remap_mutex);
+
+ btrfs_put_block_group(bg);
+ btrfs_release_path(path);
+
+ return ret;
+}
diff --git a/fs/btrfs/relocation.h b/fs/btrfs/relocation.h
index c0ee26004fc1..3afb6f85b722 100644
--- a/fs/btrfs/relocation.h
+++ b/fs/btrfs/relocation.h
@@ -32,5 +32,10 @@ struct btrfs_root *find_reloc_root(struct btrfs_fs_info *fs_info, u64 bytenr);
bool btrfs_should_ignore_reloc_root(const struct btrfs_root *root);
u64 btrfs_get_reloc_bg_bytenr(const struct btrfs_fs_info *fs_info);
int btrfs_translate_remap(struct btrfs_fs_info *fs_info, u64 *logical, u64 *length);
+int btrfs_remove_extent_from_remap_tree(struct btrfs_trans_handle *trans,
+ struct btrfs_path *path,
+ u64 bytenr, u64 num_bytes);
+int btrfs_last_identity_remap_gone(struct btrfs_chunk_map *chunk_map,
+ struct btrfs_block_group *bg);
#endif
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 2a4bda452d85..4884c7b62c61 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2925,8 +2925,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path
return ret;
}
-static noinline int btrfs_update_device(struct btrfs_trans_handle *trans,
- struct btrfs_device *device)
+int btrfs_update_device(struct btrfs_trans_handle *trans, struct btrfs_device *device)
{
int ret;
BTRFS_PATH_AUTO_FREE(path);
@@ -3224,25 +3223,12 @@ static int remove_chunk_item(struct btrfs_trans_handle *trans,
return btrfs_free_chunk(trans, chunk_offset);
}
-int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
+int btrfs_remove_dev_extents(struct btrfs_trans_handle *trans, struct btrfs_chunk_map *map)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_chunk_map *map;
+ struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
u64 dev_extent_len = 0;
int i, ret = 0;
- struct btrfs_fs_devices *fs_devices = fs_info->fs_devices;
-
- map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
- if (IS_ERR(map)) {
- /*
- * This is a logic error, but we don't want to just rely on the
- * user having built with ASSERT enabled, so if ASSERT doesn't
- * do anything we still error out.
- */
- DEBUG_WARN("errr %ld reading chunk map at offset %llu",
- PTR_ERR(map), chunk_offset);
- return PTR_ERR(map);
- }
/*
* First delete the device extent items from the devices btree.
@@ -3263,7 +3249,7 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
if (unlikely(ret)) {
mutex_unlock(&fs_devices->device_list_mutex);
btrfs_abort_transaction(trans, ret);
- goto out;
+ return ret;
}
if (device->bytes_used > 0) {
@@ -3283,6 +3269,26 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
}
mutex_unlock(&fs_devices->device_list_mutex);
+ return 0;
+}
+
+int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_chunk_map *map;
+ int ret;
+
+ map = btrfs_get_chunk_map(fs_info, chunk_offset, 1);
+ if (IS_ERR(map)) {
+ DEBUG_WARN("errr %ld reading chunk map at offset %llu",
+ PTR_ERR(map), chunk_offset);
+ return PTR_ERR(map);
+ }
+
+ ret = btrfs_remove_dev_extents(trans, map);
+ if (ret)
+ goto out;
+
/*
* We acquire fs_info->chunk_mutex for 2 reasons:
*
@@ -5419,7 +5425,7 @@ static void chunk_map_device_set_bits(struct btrfs_chunk_map *map, unsigned int
}
}
-static void chunk_map_device_clear_bits(struct btrfs_chunk_map *map, unsigned int bits)
+void btrfs_chunk_map_device_clear_bits(struct btrfs_chunk_map *map, unsigned int bits)
{
for (int i = 0; i < map->num_stripes; i++) {
struct btrfs_io_stripe *stripe = &map->stripes[i];
@@ -5436,7 +5442,7 @@ void btrfs_remove_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_ma
write_lock(&fs_info->mapping_tree_lock);
rb_erase_cached(&map->rb_node, &fs_info->mapping_tree);
RB_CLEAR_NODE(&map->rb_node);
- chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
+ btrfs_chunk_map_device_clear_bits(map, CHUNK_ALLOCATED);
write_unlock(&fs_info->mapping_tree_lock);
/* Once for the tree reference. */
@@ -5472,7 +5478,7 @@ int btrfs_add_chunk_map(struct btrfs_fs_info *fs_info, struct btrfs_chunk_map *m
return -EEXIST;
}
chunk_map_device_set_bits(map, CHUNK_ALLOCATED);
- chunk_map_device_clear_bits(map, CHUNK_TRIMMED);
+ btrfs_chunk_map_device_clear_bits(map, CHUNK_TRIMMED);
write_unlock(&fs_info->mapping_tree_lock);
return 0;
@@ -5828,7 +5834,7 @@ void btrfs_mapping_tree_free(struct btrfs_fs_info *fs_info)
]
]
|
| 383/2 |
2026/08/08 04:47 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 47131
output: 65
thoughts: 2235
Thoughts: **Analyzing the Btrfs Chunk Removal Process**
Okay, so I'm looking at how `btrfs_remove_chunk` works. It seems like commit `979e1dc3d69e` extracted `btrfs_remove_dev_extents`, which is key. The real interesting change, though, happened in `79bd37120b149532af5b21953643ed74af69654f`. That's where they changed the order of operations, specifically when `check_system_chunk` is called. It used to be called *before* removing device extents, but now it's *after*.
Let me confirm that. Yeah, `79bd37120b149532af5b21953643ed74af69654f` moved the `check_system_chunk` call to after the device extents are removed. The bug report is helpful, because it says that before, `check_system_chunk` ignores the failure to allocate a system chunk, but that now *after* `btrfs_remove_dev_extents`, the `find_free_dev_extent` call can't see the newly freed space in the current transaction. This leads to `-ENOSPC`, which, because we have already changed the device tree, forces an abort.
Wait. The thing is, this "ignoring the failure" behavior of `check_system_chunk` was already present *before* the patch. Looking at `79bd37120b149532af5b21953643ed74af69654f`, it was modified with a comment that basically says, "we ignore it." It also has this comment that explains why it ignores an error from `btrfs_alloc_chunk`, which is so if we deadlock on allocating an extent buffer from the chunk tree. So really it isn't so much a *new* problem as an *exposed* problem.
So, the key is the timing. If `check_system_chunk` is *after* the extents are removed, and it fails to allocate, the `remove_chunk_item` call will also fail because it needs to COW but it hasn't reserved space because `check_system_chunk` ignores allocation failures!
Let's think. If `check_system_chunk` was *before* removal, the device is full, it fails, but ignores the failure. `btrfs_remove_dev_extents` runs, freeing the device extents. Then `remove_chunk_item` runs. The `remove_chunk_item` will fail because, even though the space is technically available, the *transaction* has no record of the space, so it can't reserve space.
But now, what if `check_system_chunk` *did* reserve space even if it failed? Let me see... I need to look at the code before `79bd37120b149532af5b21953643ed74af69654f`.
So, the original sequence was: `mutex_lock`, `check_system_chunk`, `mutex_unlock`, `mutex_lock`, remove device extents, `mutex_unlock`, `btrfs_free_chunk`. It seems like before `79bd37120b149532af5b21953643ed74af69654f`, when the device was full, `check_system_chunk` would still ignore the failure to allocate a chunk, but *wouldn't* reserve space either? I need to check the code before `79bd37120b149532af5b21953643ed74af69654f` to be certain. I'll need to confirm that `check_system_chunk` didn't reserve any bytes when it failed. If it didn't reserve, then `remove_chunk_item` would also fail, *regardless* of when the extents are removed. No, hold on. The *order* is still the problem. The *location* of the call to `check_system_chunk` is the critical thing.
|
| 384/2 |
2026/08/08 04:48 |
tool |
git-show |
0m
Args: map[Commit:79bd37120b149532af5b21953643ed74af69654f~1:fs/btrfs/block-group.c]
Results: map[Output:
Full output is too long, showing 1000 out of 3623 lines.
[// SPDX-License-Identifier: GPL-2.0
#include "misc.h"
#include "ctree.h"
#include "block-group.h"
#include "space-info.h"
#include "disk-io.h"
#include "free-space-cache.h"
#include "free-space-tree.h"
#include "volumes.h"
#include "transaction.h"
#include "ref-verify.h"
#include "sysfs.h"
#include "tree-log.h"
#include "delalloc-space.h"
#include "discard.h"
#include "raid56.h"
#include "zoned.h"
/*
* Return target flags in extended format or 0 if restripe for this chunk_type
* is not in progress
*
* Should be called with balance_lock held
*/
static u64 get_restripe_target(struct btrfs_fs_info *fs_info, u64 flags)
{
struct btrfs_balance_control *bctl = fs_info->balance_ctl;
u64 target = 0;
if (!bctl)
return 0;
if (flags & BTRFS_BLOCK_GROUP_DATA &&
bctl->data.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_DATA | bctl->data.target;
} else if (flags & BTRFS_BLOCK_GROUP_SYSTEM &&
bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_SYSTEM | bctl->sys.target;
} else if (flags & BTRFS_BLOCK_GROUP_METADATA &&
bctl->meta.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_METADATA | bctl->meta.target;
}
return target;
}
/*
* @flags: available profiles in extended format (see ctree.h)
*
* Return reduced profile in chunk format. If profile changing is in progress
* (either running or paused) picks the target profile (if it's already
* available), otherwise falls back to plain reducing.
*/
static u64 btrfs_reduce_alloc_profile(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 num_devices = fs_info->fs_devices->rw_devices;
u64 target;
u64 raid_type;
u64 allowed = 0;
/*
* See if restripe for this chunk_type is in progress, if so try to
* reduce to the target profile
*/
spin_lock(&fs_info->balance_lock);
target = get_restripe_target(fs_info, flags);
if (target) {
spin_unlock(&fs_info->balance_lock);
return extended_to_chunk(target);
}
spin_unlock(&fs_info->balance_lock);
/* First, mask out the RAID levels which aren't possible */
for (raid_type = 0; raid_type < BTRFS_NR_RAID_TYPES; raid_type++) {
if (num_devices >= btrfs_raid_array[raid_type].devs_min)
allowed |= btrfs_raid_array[raid_type].bg_flag;
}
allowed &= flags;
if (allowed & BTRFS_BLOCK_GROUP_RAID6)
allowed = BTRFS_BLOCK_GROUP_RAID6;
else if (allowed & BTRFS_BLOCK_GROUP_RAID5)
allowed = BTRFS_BLOCK_GROUP_RAID5;
else if (allowed & BTRFS_BLOCK_GROUP_RAID10)
allowed = BTRFS_BLOCK_GROUP_RAID10;
else if (allowed & BTRFS_BLOCK_GROUP_RAID1)
allowed = BTRFS_BLOCK_GROUP_RAID1;
else if (allowed & BTRFS_BLOCK_GROUP_RAID0)
allowed = BTRFS_BLOCK_GROUP_RAID0;
flags &= ~BTRFS_BLOCK_GROUP_PROFILE_MASK;
return extended_to_chunk(flags | allowed);
}
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags)
{
unsigned seq;
u64 flags;
do {
flags = orig_flags;
seq = read_seqbegin(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
flags |= fs_info->avail_data_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
flags |= fs_info->avail_system_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_METADATA)
flags |= fs_info->avail_metadata_alloc_bits;
} while (read_seqretry(&fs_info->profiles_lock, seq));
return btrfs_reduce_alloc_profile(fs_info, flags);
}
void btrfs_get_block_group(struct btrfs_block_group *cache)
{
refcount_inc(&cache->refs);
}
void btrfs_put_block_group(struct btrfs_block_group *cache)
{
if (refcount_dec_and_test(&cache->refs)) {
WARN_ON(cache->pinned > 0);
WARN_ON(cache->reserved > 0);
/*
* A block_group shouldn't be on the discard_list anymore.
* Remove the block_group from the discard_list to prevent us
* from causing a panic due to NULL pointer dereference.
*/
if (WARN_ON(!list_empty(&cache->discard_list)))
btrfs_discard_cancel_work(&cache->fs_info->discard_ctl,
cache);
/*
* If not empty, someone is still holding mutex of
* full_stripe_lock, which can only be released by caller.
* And it will definitely cause use-after-free when caller
* tries to release full stripe lock.
*
* No better way to resolve, but only to warn.
*/
WARN_ON(!RB_EMPTY_ROOT(&cache->full_stripe_locks_root.root));
kfree(cache->free_space_ctl);
kfree(cache);
}
}
/*
* This adds the block group to the fs_info rb tree for the block group cache
*/
static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
struct btrfs_block_group *block_group)
{
struct rb_node **p;
struct rb_node *parent = NULL;
struct btrfs_block_group *cache;
ASSERT(block_group->length != 0);
spin_lock(&info->block_group_cache_lock);
p = &info->block_group_cache_tree.rb_node;
while (*p) {
parent = *p;
cache = rb_entry(parent, struct btrfs_block_group, cache_node);
if (block_group->start < cache->start) {
p = &(*p)->rb_left;
} else if (block_group->start > cache->start) {
p = &(*p)->rb_right;
} else {
spin_unlock(&info->block_group_cache_lock);
return -EEXIST;
}
}
rb_link_node(&block_group->cache_node, parent, p);
rb_insert_color(&block_group->cache_node,
&info->block_group_cache_tree);
if (info->first_logical_byte > block_group->start)
info->first_logical_byte = block_group->start;
spin_unlock(&info->block_group_cache_lock);
return 0;
}
/*
* This will return the block group at or after bytenr if contains is 0, else
* it will return the block group that contains the bytenr
*/
static struct btrfs_block_group *block_group_cache_tree_search(
struct btrfs_fs_info *info, u64 bytenr, int contains)
{
struct btrfs_block_group *cache, *ret = NULL;
struct rb_node *n;
u64 end, start;
spin_lock(&info->block_group_cache_lock);
n = info->block_group_cache_tree.rb_node;
while (n) {
cache = rb_entry(n, struct btrfs_block_group, cache_node);
end = cache->start + cache->length - 1;
start = cache->start;
if (bytenr < start) {
if (!contains && (!ret || start < ret->start))
ret = cache;
n = n->rb_left;
} else if (bytenr > start) {
if (contains && bytenr <= end) {
ret = cache;
break;
}
n = n->rb_right;
} else {
ret = cache;
break;
}
}
if (ret) {
btrfs_get_block_group(ret);
if (bytenr == 0 && info->first_logical_byte > ret->start)
info->first_logical_byte = ret->start;
}
spin_unlock(&info->block_group_cache_lock);
return ret;
}
/*
* Return the block group that starts at or after bytenr
*/
struct btrfs_block_group *btrfs_lookup_first_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 0);
}
/*
* Return the block group that contains the given bytenr
*/
struct btrfs_block_group *btrfs_lookup_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 1);
}
struct btrfs_block_group *btrfs_next_block_group(
struct btrfs_block_group *cache)
{
struct btrfs_fs_info *fs_info = cache->fs_info;
struct rb_node *node;
spin_lock(&fs_info->block_group_cache_lock);
/* If our block group was removed, we need a full search. */
if (RB_EMPTY_NODE(&cache->cache_node)) {
const u64 next_bytenr = cache->start + cache->length;
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_put_block_group(cache);
cache = btrfs_lookup_first_block_group(fs_info, next_bytenr); return cache;
}
node = rb_next(&cache->cache_node);
btrfs_put_block_group(cache);
if (node) {
cache = rb_entry(node, struct btrfs_block_group, cache_node);
btrfs_get_block_group(cache);
} else
cache = NULL;
spin_unlock(&fs_info->block_group_cache_lock);
return cache;
}
bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bool ret = true;
bg = btrfs_lookup_block_group(fs_info, bytenr);
if (!bg)
return false;
spin_lock(&bg->lock);
if (bg->ro)
ret = false;
else
atomic_inc(&bg->nocow_writers);
spin_unlock(&bg->lock);
/* No put on block group, done by btrfs_dec_nocow_writers */
if (!ret)
btrfs_put_block_group(bg);
return ret;
}
void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, bytenr);
ASSERT(bg);
if (atomic_dec_and_test(&bg->nocow_writers))
wake_up_var(&bg->nocow_writers);
/*
* Once for our lookup and once for the lookup done by a previous call
* to btrfs_inc_nocow_writers()
*/
btrfs_put_block_group(bg);
btrfs_put_block_group(bg);
}
void btrfs_wait_nocow_writers(struct btrfs_block_group *bg)
{
wait_var_event(&bg->nocow_writers, !atomic_read(&bg->nocow_writers));
}
void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
const u64 start)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, start);
ASSERT(bg);
if (atomic_dec_and_test(&bg->reservations))
wake_up_var(&bg->reservations);
btrfs_put_block_group(bg);
}
void btrfs_wait_block_group_reservations(struct btrfs_block_group *bg)
{
struct btrfs_space_info *space_info = bg->space_info;
ASSERT(bg->ro);
if (!(bg->flags & BTRFS_BLOCK_GROUP_DATA))
return;
/*
* Our block group is read only but before we set it to read only,
* some task might have had allocated an extent from it already, but it
* has not yet created a respective ordered extent (and added it to a
* root's list of ordered extents).
* Therefore wait for any task currently allocating extents, since the
* block group's reservations counter is incremented while a read lock
* on the groups' semaphore is held and decremented after releasing
* the read access on that semaphore and creating the ordered extent.
*/
down_write(&space_info->groups_sem);
up_write(&space_info->groups_sem);
wait_var_event(&bg->reservations, !atomic_read(&bg->reservations));
}
struct btrfs_caching_control *btrfs_get_caching_control(
struct btrfs_block_group *cache)
{
struct btrfs_caching_control *ctl;
spin_lock(&cache->lock);
if (!cache->caching_ctl) {
spin_unlock(&cache->lock);
return NULL;
}
ctl = cache->caching_ctl;
refcount_inc(&ctl->count);
spin_unlock(&cache->lock);
return ctl;
}
void btrfs_put_caching_control(struct btrfs_caching_control *ctl)
{
if (refcount_dec_and_test(&ctl->count))
kfree(ctl);
}
/*
* When we wait for progress in the block group caching, its because our
* allocation attempt failed at least once. So, we must sleep and let some
* progress happen before we try again.
*
* This function will sleep at least once waiting for new free space to show
* up, and then it will check the block group free space numbers for our min
* num_bytes. Another option is to have it go ahead and look in the rbtree for
* a free extent of a given size, but this is a good start.
*
* Callers of this must check if cache->cached == BTRFS_CACHE_ERROR before using
* any of the information in this block group.
*/
void btrfs_wait_block_group_cache_progress(struct btrfs_block_group *cache,
u64 num_bytes)
{
struct btrfs_caching_control *caching_ctl;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache) ||
(cache->free_space_ctl->free_space >= num_bytes));
btrfs_put_caching_control(caching_ctl);
}
int btrfs_wait_block_group_cache_done(struct btrfs_block_group *cache)
{
struct btrfs_caching_control *caching_ctl;
int ret = 0;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return (cache->cached == BTRFS_CACHE_ERROR) ? -EIO : 0;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache));
if (cache->cached == BTRFS_CACHE_ERROR)
ret = -EIO;
btrfs_put_caching_control(caching_ctl);
return ret;
}
static bool space_cache_v1_done(struct btrfs_block_group *cache)
{
bool ret;
spin_lock(&cache->lock);
ret = cache->cached != BTRFS_CACHE_FAST;
spin_unlock(&cache->lock);
return ret;
}
void btrfs_wait_space_cache_v1_finished(struct btrfs_block_group *cache,
struct btrfs_caching_control *caching_ctl)
{
wait_event(caching_ctl->wait, space_cache_v1_done(cache));
}
#ifdef CONFIG_BTRFS_DEBUG
static void fragment_free_space(struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = block_group->fs_info;
u64 start = block_group->start;
u64 len = block_group->length;
u64 chunk = block_group->flags & BTRFS_BLOCK_GROUP_METADATA ?
fs_info->nodesize : fs_info->sectorsize;
u64 step = chunk << 1;
while (len > chunk) {
btrfs_remove_free_space(block_group, start, chunk);
start += step;
if (len < step)
len = 0;
else
len -= step;
}
}
#endif
/*
* This is only called by btrfs_cache_block_group, since we could have freed
* extents we need to check the pinned_extents for any extents that can't be
* used yet since their free space will be released as soon as the transaction
* commits.
*/
u64 add_new_free_space(struct btrfs_block_group *block_group, u64 start, u64 end)
{
struct btrfs_fs_info *info = block_group->fs_info;
u64 extent_start, extent_end, size, total_added = 0;
int ret;
while (start < end) {
ret = find_first_extent_bit(&info->excluded_extents, start,
&extent_start, &extent_end,
EXTENT_DIRTY | EXTENT_UPTODATE,
NULL);
if (ret)
break;
if (extent_start <= start) {
start = extent_end + 1;
} else if (extent_start > start && extent_start < end) {
size = extent_start - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group,
start, size);
BUG_ON(ret); /* -ENOMEM or logic error */
start = extent_end + 1;
} else {
break;
}
}
if (start < end) {
size = end - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group, start,
size);
BUG_ON(ret); /* -ENOMEM or logic error */
}
return total_added;
}
static int load_extent_tree_free(struct btrfs_caching_control *caching_ctl)
{
struct btrfs_block_group *block_group = caching_ctl->block_group;
struct btrfs_fs_info *fs_info = block_group->fs_info;
struct btrfs_root *extent_root = fs_info->extent_root;
struct btrfs_path *path;
struct extent_buffer *leaf;
struct btrfs_key key;
u64 total_found = 0;
u64 last = 0;
u32 nritems;
int ret;
bool wakeup = true;
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
last = max_t(u64, block_group->start, BTRFS_SUPER_INFO_OFFSET);
#ifdef CONFIG_BTRFS_DEBUG
/*
* If we're fragmenting we don't want to make anybody think we can
* allocate from this block group until we've had a chance to fragment
* the free space.
*/
if (btrfs_should_fragment_free_space(block_group))
wakeup = false;
#endif
/*
* We don't want to deadlock with somebody trying to allocate a new
* extent for the extent root while also trying to search the extent
* root to add free space. So we skip locking and search the commit
* root, since its read-only
*/
path->skip_locking = 1;
path->search_commit_root = 1;
path->reada = READA_FORWARD;
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
next:
ret = btrfs_search_slot(NULL, extent_root, &key, path, 0, 0);
if (ret < 0)
goto out;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
while (1) {
if (btrfs_fs_closing(fs_info) > 1) {
last = (u64)-1;
break;
}
if (path->slots[0] < nritems) {
btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
} else {
ret = btrfs_find_next_key(extent_root, path, &key, 0, 0);
if (ret)
break;
if (need_resched() ||
rwsem_is_contended(&fs_info->commit_root_sem)) {
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
up_read(&fs_info->commit_root_sem);
mutex_unlock(&caching_ctl->mutex);
cond_resched();
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
goto next;
}
ret = btrfs_next_leaf(extent_root, path);
if (ret < 0)
goto out;
if (ret)
break;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
continue;
}
if (key.objectid < last) {
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
goto next;
}
if (key.objectid < block_group->start) {
path->slots[0]++;
continue;
}
if (key.objectid >= block_group->start + block_group->length)
break;
if (key.type == BTRFS_EXTENT_ITEM_KEY ||
key.type == BTRFS_METADATA_ITEM_KEY) {
total_found += add_new_free_space(block_group, last,
key.objectid);
if (key.type == BTRFS_METADATA_ITEM_KEY)
last = key.objectid +
fs_info->nodesize;
else
last = key.objectid + key.offset;
if (total_found > CACHING_CTL_WAKE_UP) {
total_found = 0;
if (wakeup)
wake_up(&caching_ctl->wait);
}
}
path->slots[0]++;
}
ret = 0;
total_found += add_new_free_space(block_group, last,
block_group->start + block_group->length);
caching_ctl->progress = (u64)-1;
out:
btrfs_free_path(path);
return ret;
}
static noinline void caching_thread(struct btrfs_work *work)
{
struct btrfs_block_group *block_group;
struct btrfs_fs_info *fs_info;
struct btrfs_caching_control *caching_ctl;
int ret;
caching_ctl = container_of(work, struct btrfs_caching_control, work);
block_group = caching_ctl->block_group;
fs_info = block_group->fs_info;
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
if (btrfs_test_opt(fs_info, SPACE_CACHE)) {
ret = load_free_space_cache(block_group);
if (ret == 1) {
ret = 0;
goto done;
}
/*
* We failed to load the space cache, set ourselves to
* CACHE_STARTED and carry on.
*/
spin_lock(&block_group->lock);
block_group->cached = BTRFS_CACHE_STARTED;
spin_unlock(&block_group->lock);
wake_up(&caching_ctl->wait);
}
/*
* If we are in the transaction that populated the free space tree we
* can't actually cache from the free space tree as our commit root and
* real root are the same, so we could change the contents of the blocks
* while caching. Instead do the slow caching in this case, and after
* the transaction has committed we will be safe.
*/
if (btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE) &&
!(test_bit(BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, &fs_info->flags)))
ret = load_free_space_tree(caching_ctl);
else
ret = load_extent_tree_free(caching_ctl);
done:
spin_lock(&block_group->lock);
block_group->caching_ctl = NULL;
block_group->cached = ret ? BTRFS_CACHE_ERROR : BTRFS_CACHE_FINISHED;
spin_unlock(&block_group->lock);
#ifdef CONFIG_BTRFS_DEBUG
if (btrfs_should_fragment_free_space(block_group)) {
u64 bytes_used;
spin_lock(&block_group->space_info->lock);
spin_lock(&block_group->lock);
bytes_used = block_group->length - block_group->used;
block_group->space_info->bytes_used += bytes_used >> 1;
spin_unlock(&block_group->lock);
spin_unlock(&block_group->space_info->lock);
fragment_free_space(block_group);
}
#endif
caching_ctl->progress = (u64)-1;
up_read(&fs_info->commit_root_sem);
btrfs_free_excluded_extents(block_group);
mutex_unlock(&caching_ctl->mutex);
wake_up(&caching_ctl->wait);
btrfs_put_caching_control(caching_ctl);
btrfs_put_block_group(block_group);
}
int btrfs_cache_block_group(struct btrfs_block_group *cache, int load_cache_only)
{
DEFINE_WAIT(wait);
struct btrfs_fs_info *fs_info = cache->fs_info;
struct btrfs_caching_control *caching_ctl = NULL;
int ret = 0;
/* Allocator for zoned filesystems does not use the cache at all */
if (btrfs_is_zoned(fs_info))
return 0;
caching_ctl = kzalloc(sizeof(*caching_ctl), GFP_NOFS);
if (!caching_ctl)
return -ENOMEM;
INIT_LIST_HEAD(&caching_ctl->list);
mutex_init(&caching_ctl->mutex);
init_waitqueue_head(&caching_ctl->wait);
caching_ctl->block_group = cache;
caching_ctl->progress = cache->start;
refcount_set(&caching_ctl->count, 2);
btrfs_init_work(&caching_ctl->work, caching_thread, NULL, NULL);
spin_lock(&cache->lock);
if (cache->cached != BTRFS_CACHE_NO) {
kfree(caching_ctl);
caching_ctl = cache->caching_ctl;
if (caching_ctl)
refcount_inc(&caching_ctl->count);
spin_unlock(&cache->lock);
goto out;
}
WARN_ON(cache->caching_ctl);
cache->caching_ctl = caching_ctl;
if (btrfs_test_opt(fs_info, SPACE_CACHE))
cache->cached = BTRFS_CACHE_FAST;
else
cache->cached = BTRFS_CACHE_STARTED;
cache->has_caching_ctl = 1;
spin_unlock(&cache->lock);
spin_lock(&fs_info->block_group_cache_lock);
refcount_inc(&caching_ctl->count);
list_add_tail(&caching_ctl->list, &fs_info->caching_block_groups);
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_get_block_group(cache);
btrfs_queue_work(fs_info->caching_workers, &caching_ctl->work);
out:
if (load_cache_only && caching_ctl)
btrfs_wait_space_cache_v1_finished(cache, caching_ctl);
if (caching_ctl)
btrfs_put_caching_control(caching_ctl);
return ret;
}
static void clear_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 extra_flags = chunk_to_extended(flags) &
BTRFS_EXTENDED_PROFILE_MASK;
write_seqlock(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
fs_info->avail_data_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_METADATA)
fs_info->avail_metadata_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
fs_info->avail_system_alloc_bits &= ~extra_flags;
write_sequnlock(&fs_info->profiles_lock);
}
/*
* Clear incompat bits for the following feature(s):
*
* - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
* in the whole filesystem
*
* - RAID1C34 - same as above for RAID1C3 and RAID1C4 block groups
*/
static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
bool found_raid56 = false;
bool found_raid1c34 = false;
if ((flags & BTRFS_BLOCK_GROUP_RAID56_MASK) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C3) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C4)) {
struct list_head *head = &fs_info->space_info;
struct btrfs_space_info *sinfo;
list_for_each_entry_rcu(sinfo, head, list) {
down_read(&sinfo->groups_sem);
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C3]))
found_raid1c34 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C4]))
found_raid1c34 = true;
up_read(&sinfo->groups_sem);
}
if (!found_raid56)
btrfs_clear_fs_incompat(fs_info, RAID56);
if (!found_raid1c34)
btrfs_clear_fs_incompat(fs_info, RAID1C34);
}
}
static int remove_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_path *path,
struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_root *root;
struct btrfs_key key;
int ret;
root = fs_info->extent_root;
key.objectid = block_group->start;
key.type = BTRFS_BLOCK_GROUP_ITEM_KEY;
key.offset = block_group->length;
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
if (ret > 0)
ret = -ENOENT;
if (ret < 0)
return ret;
ret = btrfs_del_item(trans, root, path);
return ret;
}
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
u64 group_start, struct extent_map *em)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_path *path;
struct btrfs_block_group *block_group;
struct btrfs_free_cluster *cluster;
struct inode *inode;
struct kobject *kobj = NULL;
int ret;
int index;
int factor;
struct btrfs_caching_control *caching_ctl = NULL;
bool remove_em;
bool remove_rsv = false;
block_group = btrfs_lookup_block_group(fs_info, group_start);
BUG_ON(!block_group);
BUG_ON(!block_group->ro);
trace_btrfs_remove_block_group(block_group);
/*
* Free the reserved super bytes from this block group before
* remove it.
*/
btrfs_free_excluded_extents(block_group);
btrfs_free_ref_tree_range(fs_info, block_group->start,
block_group->length);
index = btrfs_bg_flags_to_raid_index(block_group->flags);
factor = btrfs_bg_type_to_factor(block_group->flags);
/* make sure this block group isn't part of an allocation cluster */
cluster = &fs_info->data_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
/*
* make sure this block group isn't part of a metadata
* allocation cluster
*/
cluster = &fs_info->meta_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
btrfs_clear_treelog_bg(block_group);
path = btrfs_alloc_path();
if (!path) {
ret = -ENOMEM;
goto out;
}
/*
* get the inode first so any iput calls done for the io_list
* aren't the final iput (no unlinks allowed now)
*/
inode = lookup_free_space_inode(block_group, path);
mutex_lock(&trans->transaction->cache_write_mutex);
/*
* Make sure our free space cache IO is done before removing the
* free space inode
*/
spin_lock(&trans->transaction->dirty_bgs_lock);
if (!list_empty(&block_group->io_list)) {
list_del_init(&block_group->io_list);
WARN_ON(!IS_ERR(inode) && inode != block_group->io_ctl.inode);
spin_unlock(&trans->transaction->dirty_bgs_lock);
btrfs_wait_cache_io(trans, block_group, path);
btrfs_put_block_group(block_group);
spin_lock(&trans->transaction->dirty_bgs_lock);
}
if (!list_empty(&block_group->dirty_list)) {
list_del_init(&block_group->dirty_list);
remove_rsv = true;
btrfs_put_block_group(block_group);
}
spin_unlock(&trans->transaction->dirty_bgs_lock);
mutex_unlock(&trans->transaction->cache_write_mutex);
ret = btrfs_remove_free_space_inode(trans, inode, block_group);
if (ret)
goto out;
spin_lock(&fs_info->block_group_cache_lock);
rb_erase(&block_group->cache_node,
&fs_info->block_group_cache_tree);
RB_CLEAR_NODE(&block_group->cache_node);
/* Once for the block groups rbtree */
btrfs_put_block_group(block_group);
if (fs_info->first_logical_byte == block_group->start)
fs_info->first_logical_byte = (u64)-1;
spin_unlock(&fs_info->block_group_cache_lock);
down_write(&block_group->space_info->groups_sem);
/*
* we must use list_del_init so people can check to see if they
* are still on the list after taking the semaphore
*/
list_del_init(&block_group->list);
if (list_empty(&block_group->space_info->block_groups[index])) {
kobj = block_group->space_info->block_group_kobjs[index];
block_group->space_info->block_group_kobjs[index] = NULL;
clear_avail_alloc_bits(fs_info, block_group->flags);
}
up_write(&block_group->space_info->groups_sem);
clear_incompat_bg_bits(fs_info, block_group->flags);
if (kobj) {
kobject_del(kobj);
kobject_put(kobj);
}
if (block_group->has_caching_ctl)
caching_ctl = btrfs_get_caching_control(block_group);
if (block_group->cached == BTRFS_CACHE_STARTED)
btrfs_wait_block_group_cache_done(block_group);
if (block_group->has_caching_ctl) {
spin_lock(&fs_info->block_group_cache_lock);
if (!caching_ctl) {
struct btrfs_caching_control *ctl;
list_for_each_entry(ctl,
&fs_info->caching_block_groups, list)
if (ctl->block_group == block_group) {
caching_ctl = ctl;
refcount_inc(&caching_ctl->count);
break;
}
}
if (caching_ctl)
list_del_init(&caching_ctl->list);
spin_unlock(&fs_info->block_group_cache_lock);
if (caching_ctl) {
/* Once for the caching bgs list and once for us. */
btrfs_put_caching_control(caching_ctl);
btrfs_put_caching_control(caching_ctl);
]
]
|
| 385/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 61926
output: 65
thoughts: 92
Thoughts: Okay, here's my take on those thoughts, framed for an expert like myself:
**Diving into Btrfs Block Group Internals**
Alright, so `check_system_chunk` has been moved, and it's now residing in `fs/btrfs/block-group.c`. My immediate thought is, "Okay, good to know the location shift." This likely reflects a refactoring or organization improvement in the codebase. Now, before I even start to truly dig into the function itself, my focus is the version of the code before `79bd37120b149532af5b21953643ed74af69654f`. I want to understand *why* the move happened, and what kind of potential changes I might see around that version. Grasping the prior implementation will provide a good baseline for comparison and will help me understand the context behind this code relocation. I will want to quickly understand the differences, as knowing this will provide crucial insight into whatever problem the patch was trying to resolve.
|
| 386/2 |
2026/08/08 04:48 |
tool |
git-show |
0m
Args: map[Commit:79bd37120b149532af5b21953643ed74af69654f~1:fs/btrfs/block-group.c]
Results: map[Output:
Full output is too long, showing 1000 out of 3623 lines.
[// SPDX-License-Identifier: GPL-2.0
#include "misc.h"
#include "ctree.h"
#include "block-group.h"
#include "space-info.h"
#include "disk-io.h"
#include "free-space-cache.h"
#include "free-space-tree.h"
#include "volumes.h"
#include "transaction.h"
#include "ref-verify.h"
#include "sysfs.h"
#include "tree-log.h"
#include "delalloc-space.h"
#include "discard.h"
#include "raid56.h"
#include "zoned.h"
/*
* Return target flags in extended format or 0 if restripe for this chunk_type
* is not in progress
*
* Should be called with balance_lock held
*/
static u64 get_restripe_target(struct btrfs_fs_info *fs_info, u64 flags)
{
struct btrfs_balance_control *bctl = fs_info->balance_ctl;
u64 target = 0;
if (!bctl)
return 0;
if (flags & BTRFS_BLOCK_GROUP_DATA &&
bctl->data.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_DATA | bctl->data.target;
} else if (flags & BTRFS_BLOCK_GROUP_SYSTEM &&
bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_SYSTEM | bctl->sys.target;
} else if (flags & BTRFS_BLOCK_GROUP_METADATA &&
bctl->meta.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_METADATA | bctl->meta.target;
}
return target;
}
/*
* @flags: available profiles in extended format (see ctree.h)
*
* Return reduced profile in chunk format. If profile changing is in progress
* (either running or paused) picks the target profile (if it's already
* available), otherwise falls back to plain reducing.
*/
static u64 btrfs_reduce_alloc_profile(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 num_devices = fs_info->fs_devices->rw_devices;
u64 target;
u64 raid_type;
u64 allowed = 0;
/*
* See if restripe for this chunk_type is in progress, if so try to
* reduce to the target profile
*/
spin_lock(&fs_info->balance_lock);
target = get_restripe_target(fs_info, flags);
if (target) {
spin_unlock(&fs_info->balance_lock);
return extended_to_chunk(target);
}
spin_unlock(&fs_info->balance_lock);
/* First, mask out the RAID levels which aren't possible */
for (raid_type = 0; raid_type < BTRFS_NR_RAID_TYPES; raid_type++) {
if (num_devices >= btrfs_raid_array[raid_type].devs_min)
allowed |= btrfs_raid_array[raid_type].bg_flag;
}
allowed &= flags;
if (allowed & BTRFS_BLOCK_GROUP_RAID6)
allowed = BTRFS_BLOCK_GROUP_RAID6;
else if (allowed & BTRFS_BLOCK_GROUP_RAID5)
allowed = BTRFS_BLOCK_GROUP_RAID5;
else if (allowed & BTRFS_BLOCK_GROUP_RAID10)
allowed = BTRFS_BLOCK_GROUP_RAID10;
else if (allowed & BTRFS_BLOCK_GROUP_RAID1)
allowed = BTRFS_BLOCK_GROUP_RAID1;
else if (allowed & BTRFS_BLOCK_GROUP_RAID0)
allowed = BTRFS_BLOCK_GROUP_RAID0;
flags &= ~BTRFS_BLOCK_GROUP_PROFILE_MASK;
return extended_to_chunk(flags | allowed);
}
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags)
{
unsigned seq;
u64 flags;
do {
flags = orig_flags;
seq = read_seqbegin(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
flags |= fs_info->avail_data_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
flags |= fs_info->avail_system_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_METADATA)
flags |= fs_info->avail_metadata_alloc_bits;
} while (read_seqretry(&fs_info->profiles_lock, seq));
return btrfs_reduce_alloc_profile(fs_info, flags);
}
void btrfs_get_block_group(struct btrfs_block_group *cache)
{
refcount_inc(&cache->refs);
}
void btrfs_put_block_group(struct btrfs_block_group *cache)
{
if (refcount_dec_and_test(&cache->refs)) {
WARN_ON(cache->pinned > 0);
WARN_ON(cache->reserved > 0);
/*
* A block_group shouldn't be on the discard_list anymore.
* Remove the block_group from the discard_list to prevent us
* from causing a panic due to NULL pointer dereference.
*/
if (WARN_ON(!list_empty(&cache->discard_list)))
btrfs_discard_cancel_work(&cache->fs_info->discard_ctl,
cache);
/*
* If not empty, someone is still holding mutex of
* full_stripe_lock, which can only be released by caller.
* And it will definitely cause use-after-free when caller
* tries to release full stripe lock.
*
* No better way to resolve, but only to warn.
*/
WARN_ON(!RB_EMPTY_ROOT(&cache->full_stripe_locks_root.root));
kfree(cache->free_space_ctl);
kfree(cache);
}
}
/*
* This adds the block group to the fs_info rb tree for the block group cache
*/
static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
struct btrfs_block_group *block_group)
{
struct rb_node **p;
struct rb_node *parent = NULL;
struct btrfs_block_group *cache;
ASSERT(block_group->length != 0);
spin_lock(&info->block_group_cache_lock);
p = &info->block_group_cache_tree.rb_node;
while (*p) {
parent = *p;
cache = rb_entry(parent, struct btrfs_block_group, cache_node);
if (block_group->start < cache->start) {
p = &(*p)->rb_left;
} else if (block_group->start > cache->start) {
p = &(*p)->rb_right;
} else {
spin_unlock(&info->block_group_cache_lock);
return -EEXIST;
}
}
rb_link_node(&block_group->cache_node, parent, p);
rb_insert_color(&block_group->cache_node,
&info->block_group_cache_tree);
if (info->first_logical_byte > block_group->start)
info->first_logical_byte = block_group->start;
spin_unlock(&info->block_group_cache_lock);
return 0;
}
/*
* This will return the block group at or after bytenr if contains is 0, else
* it will return the block group that contains the bytenr
*/
static struct btrfs_block_group *block_group_cache_tree_search(
struct btrfs_fs_info *info, u64 bytenr, int contains)
{
struct btrfs_block_group *cache, *ret = NULL;
struct rb_node *n;
u64 end, start;
spin_lock(&info->block_group_cache_lock);
n = info->block_group_cache_tree.rb_node;
while (n) {
cache = rb_entry(n, struct btrfs_block_group, cache_node);
end = cache->start + cache->length - 1;
start = cache->start;
if (bytenr < start) {
if (!contains && (!ret || start < ret->start))
ret = cache;
n = n->rb_left;
} else if (bytenr > start) {
if (contains && bytenr <= end) {
ret = cache;
break;
}
n = n->rb_right;
} else {
ret = cache;
break;
}
}
if (ret) {
btrfs_get_block_group(ret);
if (bytenr == 0 && info->first_logical_byte > ret->start)
info->first_logical_byte = ret->start;
}
spin_unlock(&info->block_group_cache_lock);
return ret;
}
/*
* Return the block group that starts at or after bytenr
*/
struct btrfs_block_group *btrfs_lookup_first_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 0);
}
/*
* Return the block group that contains the given bytenr
*/
struct btrfs_block_group *btrfs_lookup_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 1);
}
struct btrfs_block_group *btrfs_next_block_group(
struct btrfs_block_group *cache)
{
struct btrfs_fs_info *fs_info = cache->fs_info;
struct rb_node *node;
spin_lock(&fs_info->block_group_cache_lock);
/* If our block group was removed, we need a full search. */
if (RB_EMPTY_NODE(&cache->cache_node)) {
const u64 next_bytenr = cache->start + cache->length;
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_put_block_group(cache);
cache = btrfs_lookup_first_block_group(fs_info, next_bytenr); return cache;
}
node = rb_next(&cache->cache_node);
btrfs_put_block_group(cache);
if (node) {
cache = rb_entry(node, struct btrfs_block_group, cache_node);
btrfs_get_block_group(cache);
} else
cache = NULL;
spin_unlock(&fs_info->block_group_cache_lock);
return cache;
}
bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bool ret = true;
bg = btrfs_lookup_block_group(fs_info, bytenr);
if (!bg)
return false;
spin_lock(&bg->lock);
if (bg->ro)
ret = false;
else
atomic_inc(&bg->nocow_writers);
spin_unlock(&bg->lock);
/* No put on block group, done by btrfs_dec_nocow_writers */
if (!ret)
btrfs_put_block_group(bg);
return ret;
}
void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, bytenr);
ASSERT(bg);
if (atomic_dec_and_test(&bg->nocow_writers))
wake_up_var(&bg->nocow_writers);
/*
* Once for our lookup and once for the lookup done by a previous call
* to btrfs_inc_nocow_writers()
*/
btrfs_put_block_group(bg);
btrfs_put_block_group(bg);
}
void btrfs_wait_nocow_writers(struct btrfs_block_group *bg)
{
wait_var_event(&bg->nocow_writers, !atomic_read(&bg->nocow_writers));
}
void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
const u64 start)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, start);
ASSERT(bg);
if (atomic_dec_and_test(&bg->reservations))
wake_up_var(&bg->reservations);
btrfs_put_block_group(bg);
}
void btrfs_wait_block_group_reservations(struct btrfs_block_group *bg)
{
struct btrfs_space_info *space_info = bg->space_info;
ASSERT(bg->ro);
if (!(bg->flags & BTRFS_BLOCK_GROUP_DATA))
return;
/*
* Our block group is read only but before we set it to read only,
* some task might have had allocated an extent from it already, but it
* has not yet created a respective ordered extent (and added it to a
* root's list of ordered extents).
* Therefore wait for any task currently allocating extents, since the
* block group's reservations counter is incremented while a read lock
* on the groups' semaphore is held and decremented after releasing
* the read access on that semaphore and creating the ordered extent.
*/
down_write(&space_info->groups_sem);
up_write(&space_info->groups_sem);
wait_var_event(&bg->reservations, !atomic_read(&bg->reservations));
}
struct btrfs_caching_control *btrfs_get_caching_control(
struct btrfs_block_group *cache)
{
struct btrfs_caching_control *ctl;
spin_lock(&cache->lock);
if (!cache->caching_ctl) {
spin_unlock(&cache->lock);
return NULL;
}
ctl = cache->caching_ctl;
refcount_inc(&ctl->count);
spin_unlock(&cache->lock);
return ctl;
}
void btrfs_put_caching_control(struct btrfs_caching_control *ctl)
{
if (refcount_dec_and_test(&ctl->count))
kfree(ctl);
}
/*
* When we wait for progress in the block group caching, its because our
* allocation attempt failed at least once. So, we must sleep and let some
* progress happen before we try again.
*
* This function will sleep at least once waiting for new free space to show
* up, and then it will check the block group free space numbers for our min
* num_bytes. Another option is to have it go ahead and look in the rbtree for
* a free extent of a given size, but this is a good start.
*
* Callers of this must check if cache->cached == BTRFS_CACHE_ERROR before using
* any of the information in this block group.
*/
void btrfs_wait_block_group_cache_progress(struct btrfs_block_group *cache,
u64 num_bytes)
{
struct btrfs_caching_control *caching_ctl;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache) ||
(cache->free_space_ctl->free_space >= num_bytes));
btrfs_put_caching_control(caching_ctl);
}
int btrfs_wait_block_group_cache_done(struct btrfs_block_group *cache)
{
struct btrfs_caching_control *caching_ctl;
int ret = 0;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return (cache->cached == BTRFS_CACHE_ERROR) ? -EIO : 0;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache));
if (cache->cached == BTRFS_CACHE_ERROR)
ret = -EIO;
btrfs_put_caching_control(caching_ctl);
return ret;
}
static bool space_cache_v1_done(struct btrfs_block_group *cache)
{
bool ret;
spin_lock(&cache->lock);
ret = cache->cached != BTRFS_CACHE_FAST;
spin_unlock(&cache->lock);
return ret;
}
void btrfs_wait_space_cache_v1_finished(struct btrfs_block_group *cache,
struct btrfs_caching_control *caching_ctl)
{
wait_event(caching_ctl->wait, space_cache_v1_done(cache));
}
#ifdef CONFIG_BTRFS_DEBUG
static void fragment_free_space(struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = block_group->fs_info;
u64 start = block_group->start;
u64 len = block_group->length;
u64 chunk = block_group->flags & BTRFS_BLOCK_GROUP_METADATA ?
fs_info->nodesize : fs_info->sectorsize;
u64 step = chunk << 1;
while (len > chunk) {
btrfs_remove_free_space(block_group, start, chunk);
start += step;
if (len < step)
len = 0;
else
len -= step;
}
}
#endif
/*
* This is only called by btrfs_cache_block_group, since we could have freed
* extents we need to check the pinned_extents for any extents that can't be
* used yet since their free space will be released as soon as the transaction
* commits.
*/
u64 add_new_free_space(struct btrfs_block_group *block_group, u64 start, u64 end)
{
struct btrfs_fs_info *info = block_group->fs_info;
u64 extent_start, extent_end, size, total_added = 0;
int ret;
while (start < end) {
ret = find_first_extent_bit(&info->excluded_extents, start,
&extent_start, &extent_end,
EXTENT_DIRTY | EXTENT_UPTODATE,
NULL);
if (ret)
break;
if (extent_start <= start) {
start = extent_end + 1;
} else if (extent_start > start && extent_start < end) {
size = extent_start - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group,
start, size);
BUG_ON(ret); /* -ENOMEM or logic error */
start = extent_end + 1;
} else {
break;
}
}
if (start < end) {
size = end - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group, start,
size);
BUG_ON(ret); /* -ENOMEM or logic error */
}
return total_added;
}
static int load_extent_tree_free(struct btrfs_caching_control *caching_ctl)
{
struct btrfs_block_group *block_group = caching_ctl->block_group;
struct btrfs_fs_info *fs_info = block_group->fs_info;
struct btrfs_root *extent_root = fs_info->extent_root;
struct btrfs_path *path;
struct extent_buffer *leaf;
struct btrfs_key key;
u64 total_found = 0;
u64 last = 0;
u32 nritems;
int ret;
bool wakeup = true;
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
last = max_t(u64, block_group->start, BTRFS_SUPER_INFO_OFFSET);
#ifdef CONFIG_BTRFS_DEBUG
/*
* If we're fragmenting we don't want to make anybody think we can
* allocate from this block group until we've had a chance to fragment
* the free space.
*/
if (btrfs_should_fragment_free_space(block_group))
wakeup = false;
#endif
/*
* We don't want to deadlock with somebody trying to allocate a new
* extent for the extent root while also trying to search the extent
* root to add free space. So we skip locking and search the commit
* root, since its read-only
*/
path->skip_locking = 1;
path->search_commit_root = 1;
path->reada = READA_FORWARD;
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
next:
ret = btrfs_search_slot(NULL, extent_root, &key, path, 0, 0);
if (ret < 0)
goto out;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
while (1) {
if (btrfs_fs_closing(fs_info) > 1) {
last = (u64)-1;
break;
}
if (path->slots[0] < nritems) {
btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
} else {
ret = btrfs_find_next_key(extent_root, path, &key, 0, 0);
if (ret)
break;
if (need_resched() ||
rwsem_is_contended(&fs_info->commit_root_sem)) {
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
up_read(&fs_info->commit_root_sem);
mutex_unlock(&caching_ctl->mutex);
cond_resched();
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
goto next;
}
ret = btrfs_next_leaf(extent_root, path);
if (ret < 0)
goto out;
if (ret)
break;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
continue;
}
if (key.objectid < last) {
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
goto next;
}
if (key.objectid < block_group->start) {
path->slots[0]++;
continue;
}
if (key.objectid >= block_group->start + block_group->length)
break;
if (key.type == BTRFS_EXTENT_ITEM_KEY ||
key.type == BTRFS_METADATA_ITEM_KEY) {
total_found += add_new_free_space(block_group, last,
key.objectid);
if (key.type == BTRFS_METADATA_ITEM_KEY)
last = key.objectid +
fs_info->nodesize;
else
last = key.objectid + key.offset;
if (total_found > CACHING_CTL_WAKE_UP) {
total_found = 0;
if (wakeup)
wake_up(&caching_ctl->wait);
}
}
path->slots[0]++;
}
ret = 0;
total_found += add_new_free_space(block_group, last,
block_group->start + block_group->length);
caching_ctl->progress = (u64)-1;
out:
btrfs_free_path(path);
return ret;
}
static noinline void caching_thread(struct btrfs_work *work)
{
struct btrfs_block_group *block_group;
struct btrfs_fs_info *fs_info;
struct btrfs_caching_control *caching_ctl;
int ret;
caching_ctl = container_of(work, struct btrfs_caching_control, work);
block_group = caching_ctl->block_group;
fs_info = block_group->fs_info;
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
if (btrfs_test_opt(fs_info, SPACE_CACHE)) {
ret = load_free_space_cache(block_group);
if (ret == 1) {
ret = 0;
goto done;
}
/*
* We failed to load the space cache, set ourselves to
* CACHE_STARTED and carry on.
*/
spin_lock(&block_group->lock);
block_group->cached = BTRFS_CACHE_STARTED;
spin_unlock(&block_group->lock);
wake_up(&caching_ctl->wait);
}
/*
* If we are in the transaction that populated the free space tree we
* can't actually cache from the free space tree as our commit root and
* real root are the same, so we could change the contents of the blocks
* while caching. Instead do the slow caching in this case, and after
* the transaction has committed we will be safe.
*/
if (btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE) &&
!(test_bit(BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, &fs_info->flags)))
ret = load_free_space_tree(caching_ctl);
else
ret = load_extent_tree_free(caching_ctl);
done:
spin_lock(&block_group->lock);
block_group->caching_ctl = NULL;
block_group->cached = ret ? BTRFS_CACHE_ERROR : BTRFS_CACHE_FINISHED;
spin_unlock(&block_group->lock);
#ifdef CONFIG_BTRFS_DEBUG
if (btrfs_should_fragment_free_space(block_group)) {
u64 bytes_used;
spin_lock(&block_group->space_info->lock);
spin_lock(&block_group->lock);
bytes_used = block_group->length - block_group->used;
block_group->space_info->bytes_used += bytes_used >> 1;
spin_unlock(&block_group->lock);
spin_unlock(&block_group->space_info->lock);
fragment_free_space(block_group);
}
#endif
caching_ctl->progress = (u64)-1;
up_read(&fs_info->commit_root_sem);
btrfs_free_excluded_extents(block_group);
mutex_unlock(&caching_ctl->mutex);
wake_up(&caching_ctl->wait);
btrfs_put_caching_control(caching_ctl);
btrfs_put_block_group(block_group);
}
int btrfs_cache_block_group(struct btrfs_block_group *cache, int load_cache_only)
{
DEFINE_WAIT(wait);
struct btrfs_fs_info *fs_info = cache->fs_info;
struct btrfs_caching_control *caching_ctl = NULL;
int ret = 0;
/* Allocator for zoned filesystems does not use the cache at all */
if (btrfs_is_zoned(fs_info))
return 0;
caching_ctl = kzalloc(sizeof(*caching_ctl), GFP_NOFS);
if (!caching_ctl)
return -ENOMEM;
INIT_LIST_HEAD(&caching_ctl->list);
mutex_init(&caching_ctl->mutex);
init_waitqueue_head(&caching_ctl->wait);
caching_ctl->block_group = cache;
caching_ctl->progress = cache->start;
refcount_set(&caching_ctl->count, 2);
btrfs_init_work(&caching_ctl->work, caching_thread, NULL, NULL);
spin_lock(&cache->lock);
if (cache->cached != BTRFS_CACHE_NO) {
kfree(caching_ctl);
caching_ctl = cache->caching_ctl;
if (caching_ctl)
refcount_inc(&caching_ctl->count);
spin_unlock(&cache->lock);
goto out;
}
WARN_ON(cache->caching_ctl);
cache->caching_ctl = caching_ctl;
if (btrfs_test_opt(fs_info, SPACE_CACHE))
cache->cached = BTRFS_CACHE_FAST;
else
cache->cached = BTRFS_CACHE_STARTED;
cache->has_caching_ctl = 1;
spin_unlock(&cache->lock);
spin_lock(&fs_info->block_group_cache_lock);
refcount_inc(&caching_ctl->count);
list_add_tail(&caching_ctl->list, &fs_info->caching_block_groups);
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_get_block_group(cache);
btrfs_queue_work(fs_info->caching_workers, &caching_ctl->work);
out:
if (load_cache_only && caching_ctl)
btrfs_wait_space_cache_v1_finished(cache, caching_ctl);
if (caching_ctl)
btrfs_put_caching_control(caching_ctl);
return ret;
}
static void clear_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 extra_flags = chunk_to_extended(flags) &
BTRFS_EXTENDED_PROFILE_MASK;
write_seqlock(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
fs_info->avail_data_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_METADATA)
fs_info->avail_metadata_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
fs_info->avail_system_alloc_bits &= ~extra_flags;
write_sequnlock(&fs_info->profiles_lock);
}
/*
* Clear incompat bits for the following feature(s):
*
* - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
* in the whole filesystem
*
* - RAID1C34 - same as above for RAID1C3 and RAID1C4 block groups
*/
static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
bool found_raid56 = false;
bool found_raid1c34 = false;
if ((flags & BTRFS_BLOCK_GROUP_RAID56_MASK) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C3) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C4)) {
struct list_head *head = &fs_info->space_info;
struct btrfs_space_info *sinfo;
list_for_each_entry_rcu(sinfo, head, list) {
down_read(&sinfo->groups_sem);
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C3]))
found_raid1c34 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C4]))
found_raid1c34 = true;
up_read(&sinfo->groups_sem);
}
if (!found_raid56)
btrfs_clear_fs_incompat(fs_info, RAID56);
if (!found_raid1c34)
btrfs_clear_fs_incompat(fs_info, RAID1C34);
}
}
static int remove_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_path *path,
struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_root *root;
struct btrfs_key key;
int ret;
root = fs_info->extent_root;
key.objectid = block_group->start;
key.type = BTRFS_BLOCK_GROUP_ITEM_KEY;
key.offset = block_group->length;
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
if (ret > 0)
ret = -ENOENT;
if (ret < 0)
return ret;
ret = btrfs_del_item(trans, root, path);
return ret;
}
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
u64 group_start, struct extent_map *em)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_path *path;
struct btrfs_block_group *block_group;
struct btrfs_free_cluster *cluster;
struct inode *inode;
struct kobject *kobj = NULL;
int ret;
int index;
int factor;
struct btrfs_caching_control *caching_ctl = NULL;
bool remove_em;
bool remove_rsv = false;
block_group = btrfs_lookup_block_group(fs_info, group_start);
BUG_ON(!block_group);
BUG_ON(!block_group->ro);
trace_btrfs_remove_block_group(block_group);
/*
* Free the reserved super bytes from this block group before
* remove it.
*/
btrfs_free_excluded_extents(block_group);
btrfs_free_ref_tree_range(fs_info, block_group->start,
block_group->length);
index = btrfs_bg_flags_to_raid_index(block_group->flags);
factor = btrfs_bg_type_to_factor(block_group->flags);
/* make sure this block group isn't part of an allocation cluster */
cluster = &fs_info->data_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
/*
* make sure this block group isn't part of a metadata
* allocation cluster
*/
cluster = &fs_info->meta_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
btrfs_clear_treelog_bg(block_group);
path = btrfs_alloc_path();
if (!path) {
ret = -ENOMEM;
goto out;
}
/*
* get the inode first so any iput calls done for the io_list
* aren't the final iput (no unlinks allowed now)
*/
inode = lookup_free_space_inode(block_group, path);
mutex_lock(&trans->transaction->cache_write_mutex);
/*
* Make sure our free space cache IO is done before removing the
* free space inode
*/
spin_lock(&trans->transaction->dirty_bgs_lock);
if (!list_empty(&block_group->io_list)) {
list_del_init(&block_group->io_list);
WARN_ON(!IS_ERR(inode) && inode != block_group->io_ctl.inode);
spin_unlock(&trans->transaction->dirty_bgs_lock);
btrfs_wait_cache_io(trans, block_group, path);
btrfs_put_block_group(block_group);
spin_lock(&trans->transaction->dirty_bgs_lock);
}
if (!list_empty(&block_group->dirty_list)) {
list_del_init(&block_group->dirty_list);
remove_rsv = true;
btrfs_put_block_group(block_group);
}
spin_unlock(&trans->transaction->dirty_bgs_lock);
mutex_unlock(&trans->transaction->cache_write_mutex);
ret = btrfs_remove_free_space_inode(trans, inode, block_group);
if (ret)
goto out;
spin_lock(&fs_info->block_group_cache_lock);
rb_erase(&block_group->cache_node,
&fs_info->block_group_cache_tree);
RB_CLEAR_NODE(&block_group->cache_node);
/* Once for the block groups rbtree */
btrfs_put_block_group(block_group);
if (fs_info->first_logical_byte == block_group->start)
fs_info->first_logical_byte = (u64)-1;
spin_unlock(&fs_info->block_group_cache_lock);
down_write(&block_group->space_info->groups_sem);
/*
* we must use list_del_init so people can check to see if they
* are still on the list after taking the semaphore
*/
list_del_init(&block_group->list);
if (list_empty(&block_group->space_info->block_groups[index])) {
kobj = block_group->space_info->block_group_kobjs[index];
block_group->space_info->block_group_kobjs[index] = NULL;
clear_avail_alloc_bits(fs_info, block_group->flags);
}
up_write(&block_group->space_info->groups_sem);
clear_incompat_bg_bits(fs_info, block_group->flags);
if (kobj) {
kobject_del(kobj);
kobject_put(kobj);
}
if (block_group->has_caching_ctl)
caching_ctl = btrfs_get_caching_control(block_group);
if (block_group->cached == BTRFS_CACHE_STARTED)
btrfs_wait_block_group_cache_done(block_group);
if (block_group->has_caching_ctl) {
spin_lock(&fs_info->block_group_cache_lock);
if (!caching_ctl) {
struct btrfs_caching_control *ctl;
list_for_each_entry(ctl,
&fs_info->caching_block_groups, list)
if (ctl->block_group == block_group) {
caching_ctl = ctl;
refcount_inc(&caching_ctl->count);
break;
}
}
if (caching_ctl)
list_del_init(&caching_ctl->list);
spin_unlock(&fs_info->block_group_cache_lock);
if (caching_ctl) {
/* Once for the caching bgs list and once for us. */
btrfs_put_caching_control(caching_ctl);
btrfs_put_caching_control(caching_ctl);
]
]
|
| 387/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 73840
output: 37
thoughts: 31
Thoughts: Okay, here's my attempt at summarizing that thought process, keeping in mind the user is an expert:
**Navigating the Btrfs Codebase**
Alright, so I need to find the definition of `check_system_chunk`. My memory isn't perfect, but I seem to recall that function being involved with Btrfs block group operations. It sounds like something core to the filesystem integrity, so it's probably in a crucial area of the kernel source. Given my experience, "fs/btrfs" feels like the right place to start. And based on a hunch about its function, I'd bet it's somewhere in the block group management code. So, let's head to `fs/btrfs/block-group.c` and search!
|
| 388/2 |
2026/08/08 04:48 |
tool |
grepper |
0m
Args: map[Expression:void check_system_chunk PathPrefix:fs/btrfs/block-group.c]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
]
|
| 389/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 74203
output: 65
thoughts: 64
Thoughts: Here's my summary, as the expert:
**Diving into Btrfs Block Group Management**
Okay, let's get down to brass tacks. I need to take a look at the `check_system_chunk` function. This is in the `fs/btrfs/block-group.c` file within the Btrfs filesystem code. And specifically, the commit hash I need to reference is `79bd37120b149532af5b21953643ed74af69654f~1`, which is the correct revision. I'm guessing I'm going to be dealing with the intricacies of block group allocation or perhaps system chunk validation within Btrfs. This likely means I'm troubleshooting, or maybe even preparing to implement a new feature related to block group management, since the function name suggests there are integrity checks involved. I'll need to understand how the system chunks are laid out, what data structures are involved, and what this check does to ensure data integrity.
|
| 390/2 |
2026/08/08 04:48 |
tool |
git-show |
0m
Args: map[Commit:79bd37120b149532af5b21953643ed74af69654f~1:fs/btrfs/block-group.c]
Results: map[Output:
Full output is too long, showing 1000 out of 3623 lines.
[// SPDX-License-Identifier: GPL-2.0
#include "misc.h"
#include "ctree.h"
#include "block-group.h"
#include "space-info.h"
#include "disk-io.h"
#include "free-space-cache.h"
#include "free-space-tree.h"
#include "volumes.h"
#include "transaction.h"
#include "ref-verify.h"
#include "sysfs.h"
#include "tree-log.h"
#include "delalloc-space.h"
#include "discard.h"
#include "raid56.h"
#include "zoned.h"
/*
* Return target flags in extended format or 0 if restripe for this chunk_type
* is not in progress
*
* Should be called with balance_lock held
*/
static u64 get_restripe_target(struct btrfs_fs_info *fs_info, u64 flags)
{
struct btrfs_balance_control *bctl = fs_info->balance_ctl;
u64 target = 0;
if (!bctl)
return 0;
if (flags & BTRFS_BLOCK_GROUP_DATA &&
bctl->data.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_DATA | bctl->data.target;
} else if (flags & BTRFS_BLOCK_GROUP_SYSTEM &&
bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_SYSTEM | bctl->sys.target;
} else if (flags & BTRFS_BLOCK_GROUP_METADATA &&
bctl->meta.flags & BTRFS_BALANCE_ARGS_CONVERT) {
target = BTRFS_BLOCK_GROUP_METADATA | bctl->meta.target;
}
return target;
}
/*
* @flags: available profiles in extended format (see ctree.h)
*
* Return reduced profile in chunk format. If profile changing is in progress
* (either running or paused) picks the target profile (if it's already
* available), otherwise falls back to plain reducing.
*/
static u64 btrfs_reduce_alloc_profile(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 num_devices = fs_info->fs_devices->rw_devices;
u64 target;
u64 raid_type;
u64 allowed = 0;
/*
* See if restripe for this chunk_type is in progress, if so try to
* reduce to the target profile
*/
spin_lock(&fs_info->balance_lock);
target = get_restripe_target(fs_info, flags);
if (target) {
spin_unlock(&fs_info->balance_lock);
return extended_to_chunk(target);
}
spin_unlock(&fs_info->balance_lock);
/* First, mask out the RAID levels which aren't possible */
for (raid_type = 0; raid_type < BTRFS_NR_RAID_TYPES; raid_type++) {
if (num_devices >= btrfs_raid_array[raid_type].devs_min)
allowed |= btrfs_raid_array[raid_type].bg_flag;
}
allowed &= flags;
if (allowed & BTRFS_BLOCK_GROUP_RAID6)
allowed = BTRFS_BLOCK_GROUP_RAID6;
else if (allowed & BTRFS_BLOCK_GROUP_RAID5)
allowed = BTRFS_BLOCK_GROUP_RAID5;
else if (allowed & BTRFS_BLOCK_GROUP_RAID10)
allowed = BTRFS_BLOCK_GROUP_RAID10;
else if (allowed & BTRFS_BLOCK_GROUP_RAID1)
allowed = BTRFS_BLOCK_GROUP_RAID1;
else if (allowed & BTRFS_BLOCK_GROUP_RAID0)
allowed = BTRFS_BLOCK_GROUP_RAID0;
flags &= ~BTRFS_BLOCK_GROUP_PROFILE_MASK;
return extended_to_chunk(flags | allowed);
}
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags)
{
unsigned seq;
u64 flags;
do {
flags = orig_flags;
seq = read_seqbegin(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
flags |= fs_info->avail_data_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
flags |= fs_info->avail_system_alloc_bits;
else if (flags & BTRFS_BLOCK_GROUP_METADATA)
flags |= fs_info->avail_metadata_alloc_bits;
} while (read_seqretry(&fs_info->profiles_lock, seq));
return btrfs_reduce_alloc_profile(fs_info, flags);
}
void btrfs_get_block_group(struct btrfs_block_group *cache)
{
refcount_inc(&cache->refs);
}
void btrfs_put_block_group(struct btrfs_block_group *cache)
{
if (refcount_dec_and_test(&cache->refs)) {
WARN_ON(cache->pinned > 0);
WARN_ON(cache->reserved > 0);
/*
* A block_group shouldn't be on the discard_list anymore.
* Remove the block_group from the discard_list to prevent us
* from causing a panic due to NULL pointer dereference.
*/
if (WARN_ON(!list_empty(&cache->discard_list)))
btrfs_discard_cancel_work(&cache->fs_info->discard_ctl,
cache);
/*
* If not empty, someone is still holding mutex of
* full_stripe_lock, which can only be released by caller.
* And it will definitely cause use-after-free when caller
* tries to release full stripe lock.
*
* No better way to resolve, but only to warn.
*/
WARN_ON(!RB_EMPTY_ROOT(&cache->full_stripe_locks_root.root));
kfree(cache->free_space_ctl);
kfree(cache);
}
}
/*
* This adds the block group to the fs_info rb tree for the block group cache
*/
static int btrfs_add_block_group_cache(struct btrfs_fs_info *info,
struct btrfs_block_group *block_group)
{
struct rb_node **p;
struct rb_node *parent = NULL;
struct btrfs_block_group *cache;
ASSERT(block_group->length != 0);
spin_lock(&info->block_group_cache_lock);
p = &info->block_group_cache_tree.rb_node;
while (*p) {
parent = *p;
cache = rb_entry(parent, struct btrfs_block_group, cache_node);
if (block_group->start < cache->start) {
p = &(*p)->rb_left;
} else if (block_group->start > cache->start) {
p = &(*p)->rb_right;
} else {
spin_unlock(&info->block_group_cache_lock);
return -EEXIST;
}
}
rb_link_node(&block_group->cache_node, parent, p);
rb_insert_color(&block_group->cache_node,
&info->block_group_cache_tree);
if (info->first_logical_byte > block_group->start)
info->first_logical_byte = block_group->start;
spin_unlock(&info->block_group_cache_lock);
return 0;
}
/*
* This will return the block group at or after bytenr if contains is 0, else
* it will return the block group that contains the bytenr
*/
static struct btrfs_block_group *block_group_cache_tree_search(
struct btrfs_fs_info *info, u64 bytenr, int contains)
{
struct btrfs_block_group *cache, *ret = NULL;
struct rb_node *n;
u64 end, start;
spin_lock(&info->block_group_cache_lock);
n = info->block_group_cache_tree.rb_node;
while (n) {
cache = rb_entry(n, struct btrfs_block_group, cache_node);
end = cache->start + cache->length - 1;
start = cache->start;
if (bytenr < start) {
if (!contains && (!ret || start < ret->start))
ret = cache;
n = n->rb_left;
} else if (bytenr > start) {
if (contains && bytenr <= end) {
ret = cache;
break;
}
n = n->rb_right;
} else {
ret = cache;
break;
}
}
if (ret) {
btrfs_get_block_group(ret);
if (bytenr == 0 && info->first_logical_byte > ret->start)
info->first_logical_byte = ret->start;
}
spin_unlock(&info->block_group_cache_lock);
return ret;
}
/*
* Return the block group that starts at or after bytenr
*/
struct btrfs_block_group *btrfs_lookup_first_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 0);
}
/*
* Return the block group that contains the given bytenr
*/
struct btrfs_block_group *btrfs_lookup_block_group(
struct btrfs_fs_info *info, u64 bytenr)
{
return block_group_cache_tree_search(info, bytenr, 1);
}
struct btrfs_block_group *btrfs_next_block_group(
struct btrfs_block_group *cache)
{
struct btrfs_fs_info *fs_info = cache->fs_info;
struct rb_node *node;
spin_lock(&fs_info->block_group_cache_lock);
/* If our block group was removed, we need a full search. */
if (RB_EMPTY_NODE(&cache->cache_node)) {
const u64 next_bytenr = cache->start + cache->length;
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_put_block_group(cache);
cache = btrfs_lookup_first_block_group(fs_info, next_bytenr); return cache;
}
node = rb_next(&cache->cache_node);
btrfs_put_block_group(cache);
if (node) {
cache = rb_entry(node, struct btrfs_block_group, cache_node);
btrfs_get_block_group(cache);
} else
cache = NULL;
spin_unlock(&fs_info->block_group_cache_lock);
return cache;
}
bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bool ret = true;
bg = btrfs_lookup_block_group(fs_info, bytenr);
if (!bg)
return false;
spin_lock(&bg->lock);
if (bg->ro)
ret = false;
else
atomic_inc(&bg->nocow_writers);
spin_unlock(&bg->lock);
/* No put on block group, done by btrfs_dec_nocow_writers */
if (!ret)
btrfs_put_block_group(bg);
return ret;
}
void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, bytenr);
ASSERT(bg);
if (atomic_dec_and_test(&bg->nocow_writers))
wake_up_var(&bg->nocow_writers);
/*
* Once for our lookup and once for the lookup done by a previous call
* to btrfs_inc_nocow_writers()
*/
btrfs_put_block_group(bg);
btrfs_put_block_group(bg);
}
void btrfs_wait_nocow_writers(struct btrfs_block_group *bg)
{
wait_var_event(&bg->nocow_writers, !atomic_read(&bg->nocow_writers));
}
void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
const u64 start)
{
struct btrfs_block_group *bg;
bg = btrfs_lookup_block_group(fs_info, start);
ASSERT(bg);
if (atomic_dec_and_test(&bg->reservations))
wake_up_var(&bg->reservations);
btrfs_put_block_group(bg);
}
void btrfs_wait_block_group_reservations(struct btrfs_block_group *bg)
{
struct btrfs_space_info *space_info = bg->space_info;
ASSERT(bg->ro);
if (!(bg->flags & BTRFS_BLOCK_GROUP_DATA))
return;
/*
* Our block group is read only but before we set it to read only,
* some task might have had allocated an extent from it already, but it
* has not yet created a respective ordered extent (and added it to a
* root's list of ordered extents).
* Therefore wait for any task currently allocating extents, since the
* block group's reservations counter is incremented while a read lock
* on the groups' semaphore is held and decremented after releasing
* the read access on that semaphore and creating the ordered extent.
*/
down_write(&space_info->groups_sem);
up_write(&space_info->groups_sem);
wait_var_event(&bg->reservations, !atomic_read(&bg->reservations));
}
struct btrfs_caching_control *btrfs_get_caching_control(
struct btrfs_block_group *cache)
{
struct btrfs_caching_control *ctl;
spin_lock(&cache->lock);
if (!cache->caching_ctl) {
spin_unlock(&cache->lock);
return NULL;
}
ctl = cache->caching_ctl;
refcount_inc(&ctl->count);
spin_unlock(&cache->lock);
return ctl;
}
void btrfs_put_caching_control(struct btrfs_caching_control *ctl)
{
if (refcount_dec_and_test(&ctl->count))
kfree(ctl);
}
/*
* When we wait for progress in the block group caching, its because our
* allocation attempt failed at least once. So, we must sleep and let some
* progress happen before we try again.
*
* This function will sleep at least once waiting for new free space to show
* up, and then it will check the block group free space numbers for our min
* num_bytes. Another option is to have it go ahead and look in the rbtree for
* a free extent of a given size, but this is a good start.
*
* Callers of this must check if cache->cached == BTRFS_CACHE_ERROR before using
* any of the information in this block group.
*/
void btrfs_wait_block_group_cache_progress(struct btrfs_block_group *cache,
u64 num_bytes)
{
struct btrfs_caching_control *caching_ctl;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache) ||
(cache->free_space_ctl->free_space >= num_bytes));
btrfs_put_caching_control(caching_ctl);
}
int btrfs_wait_block_group_cache_done(struct btrfs_block_group *cache)
{
struct btrfs_caching_control *caching_ctl;
int ret = 0;
caching_ctl = btrfs_get_caching_control(cache);
if (!caching_ctl)
return (cache->cached == BTRFS_CACHE_ERROR) ? -EIO : 0;
wait_event(caching_ctl->wait, btrfs_block_group_done(cache));
if (cache->cached == BTRFS_CACHE_ERROR)
ret = -EIO;
btrfs_put_caching_control(caching_ctl);
return ret;
}
static bool space_cache_v1_done(struct btrfs_block_group *cache)
{
bool ret;
spin_lock(&cache->lock);
ret = cache->cached != BTRFS_CACHE_FAST;
spin_unlock(&cache->lock);
return ret;
}
void btrfs_wait_space_cache_v1_finished(struct btrfs_block_group *cache,
struct btrfs_caching_control *caching_ctl)
{
wait_event(caching_ctl->wait, space_cache_v1_done(cache));
}
#ifdef CONFIG_BTRFS_DEBUG
static void fragment_free_space(struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = block_group->fs_info;
u64 start = block_group->start;
u64 len = block_group->length;
u64 chunk = block_group->flags & BTRFS_BLOCK_GROUP_METADATA ?
fs_info->nodesize : fs_info->sectorsize;
u64 step = chunk << 1;
while (len > chunk) {
btrfs_remove_free_space(block_group, start, chunk);
start += step;
if (len < step)
len = 0;
else
len -= step;
}
}
#endif
/*
* This is only called by btrfs_cache_block_group, since we could have freed
* extents we need to check the pinned_extents for any extents that can't be
* used yet since their free space will be released as soon as the transaction
* commits.
*/
u64 add_new_free_space(struct btrfs_block_group *block_group, u64 start, u64 end)
{
struct btrfs_fs_info *info = block_group->fs_info;
u64 extent_start, extent_end, size, total_added = 0;
int ret;
while (start < end) {
ret = find_first_extent_bit(&info->excluded_extents, start,
&extent_start, &extent_end,
EXTENT_DIRTY | EXTENT_UPTODATE,
NULL);
if (ret)
break;
if (extent_start <= start) {
start = extent_end + 1;
} else if (extent_start > start && extent_start < end) {
size = extent_start - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group,
start, size);
BUG_ON(ret); /* -ENOMEM or logic error */
start = extent_end + 1;
} else {
break;
}
}
if (start < end) {
size = end - start;
total_added += size;
ret = btrfs_add_free_space_async_trimmed(block_group, start,
size);
BUG_ON(ret); /* -ENOMEM or logic error */
}
return total_added;
}
static int load_extent_tree_free(struct btrfs_caching_control *caching_ctl)
{
struct btrfs_block_group *block_group = caching_ctl->block_group;
struct btrfs_fs_info *fs_info = block_group->fs_info;
struct btrfs_root *extent_root = fs_info->extent_root;
struct btrfs_path *path;
struct extent_buffer *leaf;
struct btrfs_key key;
u64 total_found = 0;
u64 last = 0;
u32 nritems;
int ret;
bool wakeup = true;
path = btrfs_alloc_path();
if (!path)
return -ENOMEM;
last = max_t(u64, block_group->start, BTRFS_SUPER_INFO_OFFSET);
#ifdef CONFIG_BTRFS_DEBUG
/*
* If we're fragmenting we don't want to make anybody think we can
* allocate from this block group until we've had a chance to fragment
* the free space.
*/
if (btrfs_should_fragment_free_space(block_group))
wakeup = false;
#endif
/*
* We don't want to deadlock with somebody trying to allocate a new
* extent for the extent root while also trying to search the extent
* root to add free space. So we skip locking and search the commit
* root, since its read-only
*/
path->skip_locking = 1;
path->search_commit_root = 1;
path->reada = READA_FORWARD;
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
next:
ret = btrfs_search_slot(NULL, extent_root, &key, path, 0, 0);
if (ret < 0)
goto out;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
while (1) {
if (btrfs_fs_closing(fs_info) > 1) {
last = (u64)-1;
break;
}
if (path->slots[0] < nritems) {
btrfs_item_key_to_cpu(leaf, &key, path->slots[0]);
} else {
ret = btrfs_find_next_key(extent_root, path, &key, 0, 0);
if (ret)
break;
if (need_resched() ||
rwsem_is_contended(&fs_info->commit_root_sem)) {
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
up_read(&fs_info->commit_root_sem);
mutex_unlock(&caching_ctl->mutex);
cond_resched();
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
goto next;
}
ret = btrfs_next_leaf(extent_root, path);
if (ret < 0)
goto out;
if (ret)
break;
leaf = path->nodes[0];
nritems = btrfs_header_nritems(leaf);
continue;
}
if (key.objectid < last) {
key.objectid = last;
key.offset = 0;
key.type = BTRFS_EXTENT_ITEM_KEY;
if (wakeup)
caching_ctl->progress = last;
btrfs_release_path(path);
goto next;
}
if (key.objectid < block_group->start) {
path->slots[0]++;
continue;
}
if (key.objectid >= block_group->start + block_group->length)
break;
if (key.type == BTRFS_EXTENT_ITEM_KEY ||
key.type == BTRFS_METADATA_ITEM_KEY) {
total_found += add_new_free_space(block_group, last,
key.objectid);
if (key.type == BTRFS_METADATA_ITEM_KEY)
last = key.objectid +
fs_info->nodesize;
else
last = key.objectid + key.offset;
if (total_found > CACHING_CTL_WAKE_UP) {
total_found = 0;
if (wakeup)
wake_up(&caching_ctl->wait);
}
}
path->slots[0]++;
}
ret = 0;
total_found += add_new_free_space(block_group, last,
block_group->start + block_group->length);
caching_ctl->progress = (u64)-1;
out:
btrfs_free_path(path);
return ret;
}
static noinline void caching_thread(struct btrfs_work *work)
{
struct btrfs_block_group *block_group;
struct btrfs_fs_info *fs_info;
struct btrfs_caching_control *caching_ctl;
int ret;
caching_ctl = container_of(work, struct btrfs_caching_control, work);
block_group = caching_ctl->block_group;
fs_info = block_group->fs_info;
mutex_lock(&caching_ctl->mutex);
down_read(&fs_info->commit_root_sem);
if (btrfs_test_opt(fs_info, SPACE_CACHE)) {
ret = load_free_space_cache(block_group);
if (ret == 1) {
ret = 0;
goto done;
}
/*
* We failed to load the space cache, set ourselves to
* CACHE_STARTED and carry on.
*/
spin_lock(&block_group->lock);
block_group->cached = BTRFS_CACHE_STARTED;
spin_unlock(&block_group->lock);
wake_up(&caching_ctl->wait);
}
/*
* If we are in the transaction that populated the free space tree we
* can't actually cache from the free space tree as our commit root and
* real root are the same, so we could change the contents of the blocks
* while caching. Instead do the slow caching in this case, and after
* the transaction has committed we will be safe.
*/
if (btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE) &&
!(test_bit(BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, &fs_info->flags)))
ret = load_free_space_tree(caching_ctl);
else
ret = load_extent_tree_free(caching_ctl);
done:
spin_lock(&block_group->lock);
block_group->caching_ctl = NULL;
block_group->cached = ret ? BTRFS_CACHE_ERROR : BTRFS_CACHE_FINISHED;
spin_unlock(&block_group->lock);
#ifdef CONFIG_BTRFS_DEBUG
if (btrfs_should_fragment_free_space(block_group)) {
u64 bytes_used;
spin_lock(&block_group->space_info->lock);
spin_lock(&block_group->lock);
bytes_used = block_group->length - block_group->used;
block_group->space_info->bytes_used += bytes_used >> 1;
spin_unlock(&block_group->lock);
spin_unlock(&block_group->space_info->lock);
fragment_free_space(block_group);
}
#endif
caching_ctl->progress = (u64)-1;
up_read(&fs_info->commit_root_sem);
btrfs_free_excluded_extents(block_group);
mutex_unlock(&caching_ctl->mutex);
wake_up(&caching_ctl->wait);
btrfs_put_caching_control(caching_ctl);
btrfs_put_block_group(block_group);
}
int btrfs_cache_block_group(struct btrfs_block_group *cache, int load_cache_only)
{
DEFINE_WAIT(wait);
struct btrfs_fs_info *fs_info = cache->fs_info;
struct btrfs_caching_control *caching_ctl = NULL;
int ret = 0;
/* Allocator for zoned filesystems does not use the cache at all */
if (btrfs_is_zoned(fs_info))
return 0;
caching_ctl = kzalloc(sizeof(*caching_ctl), GFP_NOFS);
if (!caching_ctl)
return -ENOMEM;
INIT_LIST_HEAD(&caching_ctl->list);
mutex_init(&caching_ctl->mutex);
init_waitqueue_head(&caching_ctl->wait);
caching_ctl->block_group = cache;
caching_ctl->progress = cache->start;
refcount_set(&caching_ctl->count, 2);
btrfs_init_work(&caching_ctl->work, caching_thread, NULL, NULL);
spin_lock(&cache->lock);
if (cache->cached != BTRFS_CACHE_NO) {
kfree(caching_ctl);
caching_ctl = cache->caching_ctl;
if (caching_ctl)
refcount_inc(&caching_ctl->count);
spin_unlock(&cache->lock);
goto out;
}
WARN_ON(cache->caching_ctl);
cache->caching_ctl = caching_ctl;
if (btrfs_test_opt(fs_info, SPACE_CACHE))
cache->cached = BTRFS_CACHE_FAST;
else
cache->cached = BTRFS_CACHE_STARTED;
cache->has_caching_ctl = 1;
spin_unlock(&cache->lock);
spin_lock(&fs_info->block_group_cache_lock);
refcount_inc(&caching_ctl->count);
list_add_tail(&caching_ctl->list, &fs_info->caching_block_groups);
spin_unlock(&fs_info->block_group_cache_lock);
btrfs_get_block_group(cache);
btrfs_queue_work(fs_info->caching_workers, &caching_ctl->work);
out:
if (load_cache_only && caching_ctl)
btrfs_wait_space_cache_v1_finished(cache, caching_ctl);
if (caching_ctl)
btrfs_put_caching_control(caching_ctl);
return ret;
}
static void clear_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 extra_flags = chunk_to_extended(flags) &
BTRFS_EXTENDED_PROFILE_MASK;
write_seqlock(&fs_info->profiles_lock);
if (flags & BTRFS_BLOCK_GROUP_DATA)
fs_info->avail_data_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_METADATA)
fs_info->avail_metadata_alloc_bits &= ~extra_flags;
if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
fs_info->avail_system_alloc_bits &= ~extra_flags;
write_sequnlock(&fs_info->profiles_lock);
}
/*
* Clear incompat bits for the following feature(s):
*
* - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
* in the whole filesystem
*
* - RAID1C34 - same as above for RAID1C3 and RAID1C4 block groups
*/
static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
bool found_raid56 = false;
bool found_raid1c34 = false;
if ((flags & BTRFS_BLOCK_GROUP_RAID56_MASK) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C3) ||
(flags & BTRFS_BLOCK_GROUP_RAID1C4)) {
struct list_head *head = &fs_info->space_info;
struct btrfs_space_info *sinfo;
list_for_each_entry_rcu(sinfo, head, list) {
down_read(&sinfo->groups_sem);
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
found_raid56 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C3]))
found_raid1c34 = true;
if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID1C4]))
found_raid1c34 = true;
up_read(&sinfo->groups_sem);
}
if (!found_raid56)
btrfs_clear_fs_incompat(fs_info, RAID56);
if (!found_raid1c34)
btrfs_clear_fs_incompat(fs_info, RAID1C34);
}
}
static int remove_block_group_item(struct btrfs_trans_handle *trans,
struct btrfs_path *path,
struct btrfs_block_group *block_group)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_root *root;
struct btrfs_key key;
int ret;
root = fs_info->extent_root;
key.objectid = block_group->start;
key.type = BTRFS_BLOCK_GROUP_ITEM_KEY;
key.offset = block_group->length;
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
if (ret > 0)
ret = -ENOENT;
if (ret < 0)
return ret;
ret = btrfs_del_item(trans, root, path);
return ret;
}
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
u64 group_start, struct extent_map *em)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_path *path;
struct btrfs_block_group *block_group;
struct btrfs_free_cluster *cluster;
struct inode *inode;
struct kobject *kobj = NULL;
int ret;
int index;
int factor;
struct btrfs_caching_control *caching_ctl = NULL;
bool remove_em;
bool remove_rsv = false;
block_group = btrfs_lookup_block_group(fs_info, group_start);
BUG_ON(!block_group);
BUG_ON(!block_group->ro);
trace_btrfs_remove_block_group(block_group);
/*
* Free the reserved super bytes from this block group before
* remove it.
*/
btrfs_free_excluded_extents(block_group);
btrfs_free_ref_tree_range(fs_info, block_group->start,
block_group->length);
index = btrfs_bg_flags_to_raid_index(block_group->flags);
factor = btrfs_bg_type_to_factor(block_group->flags);
/* make sure this block group isn't part of an allocation cluster */
cluster = &fs_info->data_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
/*
* make sure this block group isn't part of a metadata
* allocation cluster
*/
cluster = &fs_info->meta_alloc_cluster;
spin_lock(&cluster->refill_lock);
btrfs_return_cluster_to_free_space(block_group, cluster);
spin_unlock(&cluster->refill_lock);
btrfs_clear_treelog_bg(block_group);
path = btrfs_alloc_path();
if (!path) {
ret = -ENOMEM;
goto out;
}
/*
* get the inode first so any iput calls done for the io_list
* aren't the final iput (no unlinks allowed now)
*/
inode = lookup_free_space_inode(block_group, path);
mutex_lock(&trans->transaction->cache_write_mutex);
/*
* Make sure our free space cache IO is done before removing the
* free space inode
*/
spin_lock(&trans->transaction->dirty_bgs_lock);
if (!list_empty(&block_group->io_list)) {
list_del_init(&block_group->io_list);
WARN_ON(!IS_ERR(inode) && inode != block_group->io_ctl.inode);
spin_unlock(&trans->transaction->dirty_bgs_lock);
btrfs_wait_cache_io(trans, block_group, path);
btrfs_put_block_group(block_group);
spin_lock(&trans->transaction->dirty_bgs_lock);
}
if (!list_empty(&block_group->dirty_list)) {
list_del_init(&block_group->dirty_list);
remove_rsv = true;
btrfs_put_block_group(block_group);
}
spin_unlock(&trans->transaction->dirty_bgs_lock);
mutex_unlock(&trans->transaction->cache_write_mutex);
ret = btrfs_remove_free_space_inode(trans, inode, block_group);
if (ret)
goto out;
spin_lock(&fs_info->block_group_cache_lock);
rb_erase(&block_group->cache_node,
&fs_info->block_group_cache_tree);
RB_CLEAR_NODE(&block_group->cache_node);
/* Once for the block groups rbtree */
btrfs_put_block_group(block_group);
if (fs_info->first_logical_byte == block_group->start)
fs_info->first_logical_byte = (u64)-1;
spin_unlock(&fs_info->block_group_cache_lock);
down_write(&block_group->space_info->groups_sem);
/*
* we must use list_del_init so people can check to see if they
* are still on the list after taking the semaphore
*/
list_del_init(&block_group->list);
if (list_empty(&block_group->space_info->block_groups[index])) {
kobj = block_group->space_info->block_group_kobjs[index];
block_group->space_info->block_group_kobjs[index] = NULL;
clear_avail_alloc_bits(fs_info, block_group->flags);
}
up_write(&block_group->space_info->groups_sem);
clear_incompat_bg_bits(fs_info, block_group->flags);
if (kobj) {
kobject_del(kobj);
kobject_put(kobj);
}
if (block_group->has_caching_ctl)
caching_ctl = btrfs_get_caching_control(block_group);
if (block_group->cached == BTRFS_CACHE_STARTED)
btrfs_wait_block_group_cache_done(block_group);
if (block_group->has_caching_ctl) {
spin_lock(&fs_info->block_group_cache_lock);
if (!caching_ctl) {
struct btrfs_caching_control *ctl;
list_for_each_entry(ctl,
&fs_info->caching_block_groups, list)
if (ctl->block_group == block_group) {
caching_ctl = ctl;
refcount_inc(&caching_ctl->count);
break;
}
}
if (caching_ctl)
list_del_init(&caching_ctl->list);
spin_unlock(&fs_info->block_group_cache_lock);
if (caching_ctl) {
/* Once for the caching bgs list and once for us. */
btrfs_put_caching_control(caching_ctl);
btrfs_put_caching_control(caching_ctl);
]
]
|
| 391/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 86086
output: 32
thoughts: 81
Thoughts: Okay, here's my thought process on this, distilled into a first-person summary as you requested, assuming I'm already deep into this particular domain:
**Finding the Right Code**
Right, so I need to find the specific implementation of `check_system_chunk` in that old btrfs code, but digging through the entire commit history seems cumbersome. Wait a minute…I can leverage `git show`! I know that `79bd37120b149532af5b21953643ed74af69654f` is the commit ID I'm interested in, and I *think* the relevant function is in `fs/btrfs/block-group.c`. Ah ha, I can use a simple trick. I can use `git show 79bd37120b149532af5b21953643ed74af69654f~1:fs/btrfs/block-group.c` to look back one commit to get the code for this function. And then I can just pipe that into my favorite search tool, `grepper`, to quickly locate `check_system_chunk` within the file. Or actually, since I already have it in a text format I could use `git grep` directly. That's *much* faster than manually browsing the commit diffs or trying to remember which file path. This is the way to do it.
|
| 392/2 |
2026/08/08 04:48 |
tool |
grepper |
0m
Args: map[Expression:void check_system_chunk PathPrefix:fs/btrfs/]
Results: map[Output:fs/btrfs/block-group.c=4491=static void reserve_chunk_space(struct btrfs_trans_handle *trans,
--
fs/btrfs/block-group.c-4573- */
fs/btrfs/block-group.c:4574:void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
fs/btrfs/block-group.c-4575-{
--
fs/btrfs/block-group.h=380=int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
fs/btrfs/block-group.h:381:void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
fs/btrfs/block-group.h-382-void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
]
|
| 393/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 86781
output: 53
thoughts: 52
Thoughts: Okay, here's my interpretation of that thought process, assuming I'm deep in the code:
**Diving into the System Chunk**
Alright, my internal radar just pinged on `check_system_chunk` within `79bd37120b149532af5b21953643ed74af69654f~1`. That specific commit hash and the function name both immediately trigger certain lines of thinking. "System chunk" implies we're dealing with a core component, something fundamental to the operation of... well, whatever this system is. `check_` suggests a validation or integrity check. I'm now mentally preparing to parse the code within that function to fully grasp its role in this system. I'm likely hunting for a bug, a performance bottleneck, or perhaps just trying to understand the control flow. I'm anticipating some potentially intricate logic and I'll be drilling down to the details.
|
| 394/2 |
2026/08/08 04:48 |
tool |
git-show |
0m
Args: map[Commit:79bd37120b149532af5b21953643ed74af69654f~1]
Results: map[Output:commit 1cb3db1cf383a3c7dbda1aa0ce748b0958759947
Author: Filipe Manana <fdmanana@suse.com>
Date: Tue Jun 29 14:43:05 2021 +0100
btrfs: fix deadlock with concurrent chunk allocations involving system chunks
When a task attempting to allocate a new chunk verifies that there is not
currently enough free space in the system space_info and there is another
task that allocated a new system chunk but it did not finish yet the
creation of the respective block group, it waits for that other task to
finish creating the block group. This is to avoid exhaustion of the system
chunk array in the superblock, which is limited, when we have a thundering
herd of tasks allocating new chunks. This problem was described and fixed
by commit eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array
due to concurrent allocations").
However there are two very similar scenarios where this can lead to a
deadlock:
1) Task B allocated a new system chunk and task A is waiting on task B
to finish creation of the respective system block group. However before
task B ends its transaction handle and finishes the creation of the
system block group, it attempts to allocate another chunk (like a data
chunk for an fallocate operation for a very large range). Task B will
be unable to progress and allocate the new chunk, because task A set
space_info->chunk_alloc to 1 and therefore it loops at
btrfs_chunk_alloc() waiting for task A to finish its chunk allocation
and set space_info->chunk_alloc to 0, but task A is waiting on task B
to finish creation of the new system block group, therefore resulting
in a deadlock;
2) Task B allocated a new system chunk and task A is waiting on task B to
finish creation of the respective system block group. By the time that
task B enter the final phase of block group allocation, which happens
at btrfs_create_pending_block_groups(), when it modifies the extent
tree, the device tree or the chunk tree to insert the items for some
new block group, it needs to allocate a new chunk, so it ends up at
btrfs_chunk_alloc() and keeps looping there because task A has set
space_info->chunk_alloc to 1, but task A is waiting for task B to
finish creation of the new system block group and release the reserved
system space, therefore resulting in a deadlock.
In short, the problem is if a task B needs to allocate a new chunk after
it previously allocated a new system chunk and if another task A is
currently waiting for task B to complete the allocation of the new system
chunk.
Unfortunately this deadlock scenario introduced by the previous fix for
the system chunk array exhaustion problem does not have a simple and short
fix, and requires a big change to rework the chunk allocation code so that
chunk btree updates are all made in the first phase of chunk allocation.
And since this deadlock regression is being frequently hit on zoned
filesystems and the system chunk array exhaustion problem is triggered
in more extreme cases (originally observed on PowerPC with a node size
of 64K when running the fallocate tests from stress-ng), revert the
changes from that commit. The next patch in the series, with a subject
of "btrfs: rework chunk allocation to avoid exhaustion of the system
chunk array" does the necessary changes to fix the system chunk array
exhaustion problem.
Reported-by: Naohiro Aota <naohiro.aota@wdc.com>
Link: https://lore.kernel.org/linux-btrfs/20210621015922.ewgbffxuawia7liz@naota-xeon/
Fixes: eafa4fd0ad0607 ("btrfs: fix exhaustion of the system chunk array due to concurrent allocations")
CC: stable@vger.kernel.org # 5.12+
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Tested-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Tested-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index fec7a34b27f3..a26209f98279 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3377,7 +3377,6 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
*/
void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
- struct btrfs_transaction *cur_trans = trans->transaction;
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
@@ -3392,7 +3391,6 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
lockdep_assert_held(&fs_info->chunk_mutex);
info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
-again:
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
@@ -3411,58 +3409,6 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (left < thresh) {
u64 flags = btrfs_system_alloc_profile(fs_info);
- u64 reserved = atomic64_read(&cur_trans->chunk_bytes_reserved);
-
- /*
- * If there's not available space for the chunk tree (system
- * space) and there are other tasks that reserved space for
- * creating a new system block group, wait for them to complete
- * the creation of their system block group and release excess
- * reserved space. We do this because:
- *
- * *) We can end up allocating more system chunks than necessary
- * when there are multiple tasks that are concurrently
- * allocating block groups, which can lead to exhaustion of
- * the system array in the superblock;
- *
- * *) If we allocate extra and unnecessary system block groups,
- * despite being empty for a long time, and possibly forever,
- * they end not being added to the list of unused block groups
- * because that typically happens only when deallocating the
- * last extent from a block group - which never happens since
- * we never allocate from them in the first place. The few
- * exceptions are when mounting a filesystem or running scrub,
- * which add unused block groups to the list of unused block
- * groups, to be deleted by the cleaner kthread.
- * And even when they are added to the list of unused block
- * groups, it can take a long time until they get deleted,
- * since the cleaner kthread might be sleeping or busy with
- * other work (deleting subvolumes, running delayed iputs,
- * defrag scheduling, etc);
- *
- * This is rare in practice, but can happen when too many tasks
- * are allocating blocks groups in parallel (via fallocate())
- * and before the one that reserved space for a new system block
- * group finishes the block group creation and releases the space
- * reserved in excess (at btrfs_create_pending_block_groups()),
- * other tasks end up here and see free system space temporarily
- * not enough for updating the chunk tree.
- *
- * We unlock the chunk mutex before waiting for such tasks and
- * lock it again after the wait, otherwise we would deadlock.
- * It is safe to do so because allocating a system chunk is the
- * first thing done while allocating a new block group.
- */
- if (reserved > trans->chunk_bytes_reserved) {
- const u64 min_needed = reserved - thresh;
-
- mutex_unlock(&fs_info->chunk_mutex);
- wait_event(cur_trans->chunk_reserve_wait,
- atomic64_read(&cur_trans->chunk_bytes_reserved) <=
- min_needed);
- mutex_lock(&fs_info->chunk_mutex);
- goto again;
- }
/*
* Ignore failure to create system chunk. We might end up not
@@ -3477,10 +3423,8 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
thresh, BTRFS_RESERVE_NO_FLUSH);
- if (!ret) {
- atomic64_add(thresh, &cur_trans->chunk_bytes_reserved);
+ if (!ret)
trans->chunk_bytes_reserved += thresh;
- }
}
}
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 50318231c1a8..443c348bc6f3 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -260,7 +260,6 @@ static inline int extwriter_counter_read(struct btrfs_transaction *trans)
void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_transaction *cur_trans = trans->transaction;
if (!trans->chunk_bytes_reserved)
return;
@@ -269,8 +268,6 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
trans->chunk_bytes_reserved, NULL);
- atomic64_sub(trans->chunk_bytes_reserved, &cur_trans->chunk_bytes_reserved);
- cond_wake_up(&cur_trans->chunk_reserve_wait);
trans->chunk_bytes_reserved = 0;
}
@@ -386,8 +383,6 @@ static noinline int join_transaction(struct btrfs_fs_info *fs_info,
spin_lock_init(&cur_trans->dropped_roots_lock);
INIT_LIST_HEAD(&cur_trans->releasing_ebs);
spin_lock_init(&cur_trans->releasing_ebs_lock);
- atomic64_set(&cur_trans->chunk_bytes_reserved, 0);
- init_waitqueue_head(&cur_trans->chunk_reserve_wait);
list_add_tail(&cur_trans->list, &fs_info->trans_list);
extent_io_tree_init(fs_info, &cur_trans->dirty_pages,
IO_TREE_TRANS_DIRTY_PAGES, fs_info->btree_inode);
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index 07d76029f598..a18d67796b54 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -96,13 +96,6 @@ struct btrfs_transaction {
spinlock_t releasing_ebs_lock;
struct list_head releasing_ebs;
-
- /*
- * The number of bytes currently reserved, by all transaction handles
- * attached to this transaction, for metadata extents of the chunk tree.
- */
- atomic64_t chunk_bytes_reserved;
- wait_queue_head_t chunk_reserve_wait;
};
#define __TRANS_FREEZABLE (1U << 0)
]
|
| 395/2 |
2026/08/08 04:48 |
llm |
fixes-finder |
1m
Model:
Tokens:
input: 90310
output: 39
thoughts: 8399
Thoughts: **Analyzing the Btrfs Chunk Removal Transaction Abort Bug**
Okay, here's the breakdown of my thought process in tackling this Btrfs bug, presented as if I'm walking through the problem myself. I'm starting from the observation that Btrfs sometimes aborts transactions during chunk removal when it shouldn't.
My initial thought is to figure out when and why the failure is silently ignored and then where it goes wrong. I begin by examining the `check_system_chunk` function, which, I gather, is responsible for ensuring there's enough space for chunk operations. I see it includes a peculiar comment: "Ignore failure to create system chunk". So, even if allocating a new system chunk fails, the code just... keeps going? That seems highly suspicious.
The code then tries to reserve space using `btrfs_block_rsv_add`. Now, wait, if it *ignored* the allocation failure, would it still reserve space? Looking closely, the reservation *only* happens if the allocation succeeds ( `if (!ret)`). If `ret` is an error, this crucial space reservation is skipped. This immediately raises a red flag. If allocating space fails and it isn't noted, then the removal process can continue without adequate resources.
Okay, let's trace the full sequence now. `check_system_chunk` is called, it might fail (and is ignored). Then, `btrfs_remove_dev_extents` is called, and then finally `remove_chunk_item`. If the chunk allocation failed and space was *not* reserved, then `remove_chunk_item` will hit `-ENOSPC` when it tries to COW a node. Since no space was reserved, the whole transaction must now abort.
The crucial problem seems to be `check_system_chunk` ignoring the allocation failure for the system chunk. The code *says* it's okay because we might not need all that space; some nodes might be already COWed or created. But it's a huge trade-off: in the event that we actually *do* need the space, we get a transaction abort, which is way worse than an out-of-space error.
I now focus on why ignoring the failure causes the transaction to abort. If `check_system_chunk` is called *before* `btrfs_remove_dev_extents`, the device extents have not been freed, which would lead to a transaction abort when a chunk btree node allocation fails. The bug report highlights that the freed device extents are "just freed" in the current transaction. This suggests to me the order of operations is important. If, as the bug report implies, `check_system_chunk` is called *after* freeing the extents via `btrfs_remove_dev_extents`, then `find_free_dev_extent` in the *commit root* will not see the recently freed space, which is why the code fails with `-ENOSPC`.
The core issue is that `btrfs_remove_dev_extents` (which modifies the device tree) is happening before `remove_chunk_item`. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` must complete the transaction; it can't gracefully return `-ENOSPC` now, because returning that would leave the file system inconsistent.
Ah! The fix mentions that the space reservation must happen *before* device tree modifications to avoid these inconsistencies. The problem is that chunk allocation can't be done before modifications because `btrfs_remove_dev_extents` can trigger nested allocations and then release the reserved space.
The solution in the fix does the following: call `check_system_chunk` early *before* device modifications, make sure we have enough space and *then* release the reservation. Now, if it fails, we can *gracefully* return `-ENOSPC` before anything is modified. Then after `btrfs_remove_dev_extents`, we call it again, but this time to actually reserve the space, and not just verify that it is available.
The key is that the order of operations matters. If we fail *before* modifying anything, we return an error. If we fail *after* modifications, we abort.
Now, I zero in on the commit that introduced this problematic behavior. The commit message makes it clear: the initial code ignored failures to allocate system chunks because we might not need the space. If we don't handle the failure, then if the code *does* need the space, we have a transaction abort instead of a user-friendly error. I examine the code in the commit. Aha! It's the one that made `check_system_chunk` ignore the allocation failure and not
|
| 396/2 |
2026/08/08 04:49 |
tool |
git-log |
0m
Args: map[CodeRegexp:reserve_chunk_space PathPrefix:fs/btrfs/block-group.c]
Results: map[Output:2bb2e00ed978 btrfs: fix deadlock between chunk allocation and chunk btree modifications
]
|
| 397/2 |
2026/08/08 04:49 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 99811
output: 25
thoughts: 16
Thoughts: Okay, here's my attempt at summarizing the thought process, as requested, aimed at an expert audience and written in the first person:
**Analyzing the Hash `2bb2e00ed978`**
Alright, let's dive in. First order of business is checking out this hash: `2bb2e00ed978`. Just looking at it, it's not immediately telling me anything, which is as expected for a seemingly random string. Now, my initial thought process kicks in, as this looks like some sort of hash or ID. My mind immediately begins running through potential hashing algorithms that would generate a hex string of that length. MD5 is too short, SHA-1 is plausible but potentially deprecated depending on the context. SHA-256 would be the ideal, but would be twice the length.
The length indicates it might be an identifier of some sort and would be useful to check against a database of known hashes. I'll want to quickly determine its origin - is it from a file? A network packet? Some other application? The context is key. If I have that, I can leverage that to help determine what is meant by it and then move to a next step.
Now, I'm thinking about the tools I might need to gather additional information. Hash lookup services, of course. VirusTotal, perhaps? That would be an early step to determine if this is flagged anywhere. Then I'd want to consider: Is there any related data or context I have? Is there a log file? Any surrounding data? Even a simple `grep` to see if that string appears elsewhere in a relevant data set could be insightful. I need to establish a base of knowledge and build an approach from there. I'll need to know whether the hash originated from data that is of importance, or if this could be an internal tracking/unique identification number that is of little consequence.
|
| 398/2 |
2026/08/08 04:49 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2
Author: Filipe Manana <fdmanana@suse.com>
Date: Wed Oct 13 10:12:49 2021 +0100
btrfs: fix deadlock between chunk allocation and chunk btree modifications
When a task is doing some modification to the chunk btree and it is not in
the context of a chunk allocation or a chunk removal, it can deadlock with
another task that is currently allocating a new data or metadata chunk.
These contexts are the following:
* When relocating a system chunk, when we need to COW the extent buffers
that belong to the chunk btree;
* When adding a new device (ioctl), where we need to add a new device item
to the chunk btree;
* When removing a device (ioctl), where we need to remove a device item
from the chunk btree;
* When resizing a device (ioctl), where we need to update a device item in
the chunk btree and may need to relocate a system chunk that lies beyond
the new device size when shrinking a device.
The problem happens due to a sequence of steps like the following:
1) Task A starts a data or metadata chunk allocation and it locks the
chunk mutex;
2) Task B is relocating a system chunk, and when it needs to COW an extent
buffer of the chunk btree, it has locked both that extent buffer as
well as its parent extent buffer;
3) Since there is not enough available system space, either because none
of the existing system block groups have enough free space or because
the only one with enough free space is in RO mode due to the relocation,
task B triggers a new system chunk allocation. It blocks when trying to
acquire the chunk mutex, currently held by task A;
4) Task A enters btrfs_chunk_alloc_add_chunk_item(), in order to insert
the new chunk item into the chunk btree and update the existing device
items there. But in order to do that, it has to lock the extent buffer
that task B locked at step 2, or its parent extent buffer, but task B
is waiting on the chunk mutex, which is currently locked by task A,
therefore resulting in a deadlock.
One example report when the deadlock happens with system chunk relocation:
INFO: task kworker/u9:5:546 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u9:5 state:D stack:25936 pid: 546 ppid: 2 flags:0x00004000
Workqueue: events_unbound btrfs_async_reclaim_metadata_space
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
rwsem_down_read_slowpath+0x4ee/0x9d0 kernel/locking/rwsem.c:993
__down_read_common kernel/locking/rwsem.c:1214 [inline]
__down_read kernel/locking/rwsem.c:1223 [inline]
down_read_nested+0xe6/0x440 kernel/locking/rwsem.c:1590
__btrfs_tree_read_lock+0x31/0x350 fs/btrfs/locking.c:47
btrfs_tree_read_lock fs/btrfs/locking.c:54 [inline]
btrfs_read_lock_root_node+0x8a/0x320 fs/btrfs/locking.c:191
btrfs_search_slot_get_root fs/btrfs/ctree.c:1623 [inline]
btrfs_search_slot+0x13b4/0x2140 fs/btrfs/ctree.c:1728
btrfs_update_device+0x11f/0x500 fs/btrfs/volumes.c:2794
btrfs_chunk_alloc_add_chunk_item+0x34d/0xea0 fs/btrfs/volumes.c:5504
do_chunk_alloc fs/btrfs/block-group.c:3408 [inline]
btrfs_chunk_alloc+0x84d/0xf50 fs/btrfs/block-group.c:3653
flush_space+0x54e/0xd80 fs/btrfs/space-info.c:670
btrfs_async_reclaim_metadata_space+0x396/0xa90 fs/btrfs/space-info.c:953
process_one_work+0x9df/0x16d0 kernel/workqueue.c:2297
worker_thread+0x90/0xed0 kernel/workqueue.c:2444
kthread+0x3e5/0x4d0 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
INFO: task syz-executor:9107 blocked for more than 143 seconds.
Not tainted 5.15.0-rc3+ #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor state:D stack:23200 pid: 9107 ppid: 7792 flags:0x00004004
Call Trace:
context_switch kernel/sched/core.c:4940 [inline]
__schedule+0xcd9/0x2530 kernel/sched/core.c:6287
schedule+0xd3/0x270 kernel/sched/core.c:6366
schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6425
__mutex_lock_common kernel/locking/mutex.c:669 [inline]
__mutex_lock+0xc96/0x1680 kernel/locking/mutex.c:729
btrfs_chunk_alloc+0x31a/0xf50 fs/btrfs/block-group.c:3631
find_free_extent_update_loop fs/btrfs/extent-tree.c:3986 [inline]
find_free_extent+0x25cb/0x3a30 fs/btrfs/extent-tree.c:4335
btrfs_reserve_extent+0x1f1/0x500 fs/btrfs/extent-tree.c:4415
btrfs_alloc_tree_block+0x203/0x1120 fs/btrfs/extent-tree.c:4813
__btrfs_cow_block+0x412/0x1620 fs/btrfs/ctree.c:415
btrfs_cow_block+0x2f6/0x8c0 fs/btrfs/ctree.c:570
btrfs_search_slot+0x1094/0x2140 fs/btrfs/ctree.c:1768
relocate_tree_block fs/btrfs/relocation.c:2694 [inline]
relocate_tree_blocks+0xf73/0x1770 fs/btrfs/relocation.c:2757
relocate_block_group+0x47e/0xc70 fs/btrfs/relocation.c:3673
btrfs_relocate_block_group+0x48a/0xc60 fs/btrfs/relocation.c:4070
btrfs_relocate_chunk+0x96/0x280 fs/btrfs/volumes.c:3181
__btrfs_balance fs/btrfs/volumes.c:3911 [inline]
btrfs_balance+0x1f03/0x3cd0 fs/btrfs/volumes.c:4301
btrfs_ioctl_balance+0x61e/0x800 fs/btrfs/ioctl.c:4137
btrfs_ioctl+0x39ea/0x7b70 fs/btrfs/ioctl.c:4949
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:874 [inline]
__se_sys_ioctl fs/ioctl.c:860 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:860
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
So fix this by making sure that whenever we try to modify the chunk btree
and we are neither in a chunk allocation context nor in a chunk remove
context, we reserve system space before modifying the chunk btree.
Reported-by: Hao Sun <sunhao.th@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/CACkBjsax51i4mu6C0C3vJqQN3NR_iVuucoeG3U1HXjrgzn5FFQ@mail.gmail.com/
Fixes: 79bd37120b1495 ("btrfs: rework chunk allocation to avoid exhaustion of the system chunk array")
CC: stable@vger.kernel.org # 5.14+
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index de9aeb3733cf..f971d043469c 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -3425,25 +3425,6 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
goto out;
}
- /*
- * If this is a system chunk allocation then stop right here and do not
- * add the chunk item to the chunk btree. This is to prevent a deadlock
- * because this system chunk allocation can be triggered while COWing
- * some extent buffer of the chunk btree and while holding a lock on a
- * parent extent buffer, in which case attempting to insert the chunk
- * item (or update the device item) would result in a deadlock on that
- * parent extent buffer. In this case defer the chunk btree updates to
- * the second phase of chunk allocation and keep our reservation until
- * the second phase completes.
- *
- * This is a rare case and can only be triggered by the very few cases
- * we have where we need to touch the chunk btree outside chunk allocation
- * and chunk removal. These cases are basically adding a device, removing
- * a device or resizing a device.
- */
- if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
- return 0;
-
ret = btrfs_chunk_alloc_add_chunk_item(trans, bg);
/*
* Normally we are not expected to fail with -ENOSPC here, since we have
@@ -3576,14 +3557,14 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags)
* This has happened before and commit eafa4fd0ad0607 ("btrfs: fix exhaustion of
* the system chunk array due to concurrent allocations") provides more details.
*
- * For allocation of system chunks, we defer the updates and insertions into the
- * chunk btree to phase 2. This is to prevent deadlocks on extent buffers because
- * if the chunk allocation is triggered while COWing an extent buffer of the
- * chunk btree, we are holding a lock on the parent of that extent buffer and
- * doing the chunk btree updates and insertions can require locking that parent.
- * This is for the very few and rare cases where we update the chunk btree that
- * are not chunk allocation or chunk removal: adding a device, removing a device
- * or resizing a device.
+ * Allocation of system chunks does not happen through this function. A task that
+ * needs to update the chunk btree (the only btree that uses system chunks), must
+ * preallocate chunk space by calling either check_system_chunk() or
+ * btrfs_reserve_chunk_metadata() - the former is used when allocating a data or
+ * metadata chunk or when removing a chunk, while the later is used before doing
+ * a modification to the chunk btree - use cases for the later are adding,
+ * removing and resizing a device as well as relocation of a system chunk.
+ * See the comment below for more details.
*
* The reservation of system space, done through check_system_chunk(), as well
* as all the updates and insertions into the chunk btree must be done while
@@ -3620,11 +3601,27 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
if (trans->allocating_chunk)
return -ENOSPC;
/*
- * If we are removing a chunk, don't re-enter or we would deadlock.
- * System space reservation and system chunk allocation is done by the
- * chunk remove operation (btrfs_remove_chunk()).
+ * Allocation of system chunks can not happen through this path, as we
+ * could end up in a deadlock if we are allocating a data or metadata
+ * chunk and there is another task modifying the chunk btree.
+ *
+ * This is because while we are holding the chunk mutex, we will attempt
+ * to add the new chunk item to the chunk btree or update an existing
+ * device item in the chunk btree, while the other task that is modifying
+ * the chunk btree is attempting to COW an extent buffer while holding a
+ * lock on it and on its parent - if the COW operation triggers a system
+ * chunk allocation, then we can deadlock because we are holding the
+ * chunk mutex and we may need to access that extent buffer or its parent
+ * in order to add the chunk item or update a device item.
+ *
+ * Tasks that want to modify the chunk tree should reserve system space
+ * before updating the chunk btree, by calling either
+ * btrfs_reserve_chunk_metadata() or check_system_chunk().
+ * It's possible that after a task reserves the space, it still ends up
+ * here - this happens in the cases described above at do_chunk_alloc().
+ * The task will have to either retry or fail.
*/
- if (trans->removing_chunk)
+ if (flags & BTRFS_BLOCK_GROUP_SYSTEM)
return -ENOSPC;
space_info = btrfs_find_space_info(fs_info, flags);
@@ -3723,17 +3720,14 @@ static u64 get_profile_num_devs(struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-/*
- * Reserve space in the system space for allocating or removing a chunk
- */
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+static void reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
u64 left;
- u64 thresh;
int ret = 0;
- u64 num_devs;
/*
* Needed because we can end up allocating a system chunk and for an
@@ -3746,19 +3740,13 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
- num_devs = get_profile_num_devs(fs_info, type);
-
- /* num_devs device items to update and 1 chunk item to add or remove */
- thresh = btrfs_calc_metadata_size(fs_info, num_devs) +
- btrfs_calc_insert_metadata_size(fs_info, 1);
-
- if (left < thresh && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ if (left < bytes && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
- left, thresh, type);
+ left, bytes, type);
btrfs_dump_space_info(fs_info, info, 0, 0);
}
- if (left < thresh) {
+ if (left < bytes) {
u64 flags = btrfs_system_alloc_profile(fs_info);
struct btrfs_block_group *bg;
@@ -3767,21 +3755,20 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
* needing it, as we might not need to COW all nodes/leafs from
* the paths we visit in the chunk tree (they were already COWed
* or created in the current transaction for example).
- *
- * Also, if our caller is allocating a system chunk, do not
- * attempt to insert the chunk item in the chunk btree, as we
- * could deadlock on an extent buffer since our caller may be
- * COWing an extent buffer from the chunk btree.
*/
bg = btrfs_create_chunk(trans, flags);
if (IS_ERR(bg)) {
ret = PTR_ERR(bg);
- } else if (!(type & BTRFS_BLOCK_GROUP_SYSTEM)) {
+ } else {
/*
* If we fail to add the chunk item here, we end up
* trying again at phase 2 of chunk allocation, at
* btrfs_create_pending_block_groups(). So ignore
- * any error here.
+ * any error here. An ENOSPC here could happen, due to
+ * the cases described at do_chunk_alloc() - the system
+ * block group we just created was just turned into RO
+ * mode by a scrub for example, or a running discard
+ * temporarily removed its free space entries, etc.
*/
btrfs_chunk_alloc_add_chunk_item(trans, bg);
}
@@ -3790,12 +3777,61 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
if (!ret) {
ret = btrfs_block_rsv_add(fs_info->chunk_root,
&fs_info->chunk_block_rsv,
- thresh, BTRFS_RESERVE_NO_FLUSH);
+ bytes, BTRFS_RESERVE_NO_FLUSH);
if (!ret)
- trans->chunk_bytes_reserved += thresh;
+ trans->chunk_bytes_reserved += bytes;
}
}
+/*
+ * Reserve space in the system space for allocating or removing a chunk.
+ * The caller must be holding fs_info->chunk_mutex.
+ */
+void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ const u64 num_devs = get_profile_num_devs(fs_info, type);
+ u64 bytes;
+
+ /* num_devs device items to update and 1 chunk item to add or remove. */
+ bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
+ btrfs_calc_insert_metadata_size(fs_info, 1);
+
+ reserve_chunk_space(trans, bytes, type);
+}
+
+/*
+ * Reserve space in the system space, if needed, for doing a modification to the
+ * chunk btree.
+ *
+ * @trans: A transaction handle.
+ * @is_item_insertion: Indicate if the modification is for inserting a new item
+ * in the chunk btree or if it's for the deletion or update
+ * of an existing item.
+ *
+ * This is used in a context where we need to update the chunk btree outside
+ * block group allocation and removal, to avoid a deadlock with a concurrent
+ * task that is allocating a metadata or data block group and therefore needs to
+ * update the chunk btree while holding the chunk mutex. After the update to the
+ * chunk btree is done, btrfs_trans_release_chunk_metadata() should be called.
+ *
+ */
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ u64 bytes;
+
+ if (is_item_insertion)
+ bytes = btrfs_calc_insert_metadata_size(fs_info, 1);
+ else
+ bytes = btrfs_calc_metadata_size(fs_info, 1);
+
+ mutex_lock(&fs_info->chunk_mutex);
+ reserve_chunk_space(trans, bytes, BTRFS_BLOCK_GROUP_SYSTEM);
+ mutex_unlock(&fs_info->chunk_mutex);
+}
+
void btrfs_put_block_group_cache(struct btrfs_fs_info *info)
{
struct btrfs_block_group *block_group;
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 07f977d3816c..5878b7ce3b78 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -293,6 +293,8 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
+ bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index fed823596248..33a0ee7ac590 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -2692,8 +2692,12 @@ static int relocate_tree_block(struct btrfs_trans_handle *trans,
list_add_tail(&node->list, &rc->backref_cache.changed);
} else {
path->lowest_level = node->level;
+ if (root == root->fs_info->chunk_root)
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, key, path, 0, 1);
btrfs_release_path(path);
+ if (root == root->fs_info->chunk_root)
+ btrfs_trans_release_chunk_metadata(trans);
if (ret > 0)
ret = 0;
}
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index debba6f04858..9eab8a741166 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1847,8 +1847,10 @@ static int btrfs_add_dev_item(struct btrfs_trans_handle *trans,
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, true);
ret = btrfs_insert_empty_item(trans, trans->fs_info->chunk_root, path,
&key, sizeof(*dev_item));
+ btrfs_trans_release_chunk_metadata(trans);
if (ret)
goto out;
@@ -1921,7 +1923,9 @@ static int btrfs_rm_dev_item(struct btrfs_device *device)
key.type = BTRFS_DEV_ITEM_KEY;
key.offset = device->devid;
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret) {
if (ret > 0)
ret = -ENOENT;
@@ -2513,7 +2517,9 @@ static int btrfs_finish_sprout(struct btrfs_trans_handle *trans)
key.type = BTRFS_DEV_ITEM_KEY;
while (1) {
+ btrfs_reserve_chunk_metadata(trans, false);
ret = btrfs_search_slot(trans, root, &key, path, 0, 1);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0)
goto error;
@@ -2862,6 +2868,7 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
struct btrfs_super_block *super_copy = fs_info->super_copy;
u64 old_total;
u64 diff;
+ int ret;
if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state))
return -EACCES;
@@ -2890,7 +2897,11 @@ int btrfs_grow_device(struct btrfs_trans_handle *trans,
&trans->transaction->dev_update_list);
mutex_unlock(&fs_info->chunk_mutex);
- return btrfs_update_device(trans, device);
+ btrfs_reserve_chunk_metadata(trans, false);
+ ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
+
+ return ret;
}
static int btrfs_free_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
@@ -4925,8 +4936,10 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
round_down(old_total - diff, fs_info->sectorsize));
mutex_unlock(&fs_info->chunk_mutex);
+ btrfs_reserve_chunk_metadata(trans, false);
/* Now btrfs_update_device() will change the on-disk size. */
ret = btrfs_update_device(trans, device);
+ btrfs_trans_release_chunk_metadata(trans);
if (ret < 0) {
btrfs_abort_transaction(trans, ret);
btrfs_end_transaction(trans);
]
|
| 399/2 |
2026/08/08 04:49 |
llm |
fixes-finder |
0m
Model:
Tokens:
input: 107781
output: 53
thoughts: 684
Thoughts: **Analyzing the `reserve_chunk_space` and System Chunk Allocation Bug**
Okay, let me break this down. My initial thought process revolves around a specific change related to Btrfs chunk allocation. The function `reserve_chunk_space` was introduced in commit `2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2`. Looking at the code from that commit, it's immediately apparent that the function *ignores* the failure of `btrfs_create_chunk` (which, if I recall, was renamed from `btrfs_alloc_chunk` for system chunks). Specifically, the code snippet shows that `reserve_chunk_space` calls `btrfs_create_chunk`, and if an error occurs, it's set in the local variable `ret`. However, `reserve_chunk_space` doesn't actually *return* this error code; it's a `void` function in `2bb2e00ed9787e52580bb651264b8d6a2b7a9dd2`.
Aha! This is a clear indicator that error handling is missing. And `check_system_chunk` also returns `void` in this commit.
Now, from the fix, I see that both `reserve_chunk_space` and `check_system_chunk` are changed to return an `int`. This provides the crucial clue. This change implies that the lack of error propagation was a bug that was fixed in the later commit. I'm tracking this back, and the commit hash `79bd37120b149532af5b21953643ed74af69654f` seems to have caused the issue. Indeed, this commit moved `check_system_chunk` after `btrfs_remove_dev_extents`. Let me double-check that commit hash to be certain. Yes, `79bd37120b149532af5b21953643ed74af69654f` is the one.
Thus, based on this analysis, the bug-introducing commit is `79bd37120b149532af5b21953643ed74af69654f`.
|
| 400/2 |
2026/08/08 04:49 |
tool |
set-results |
0m
Args: map[FixesHash:79bd37120b149532af5b21953643ed74af69654f]
Results: map[FixesHash:79bd37120b149532af5b21953643ed74af69654f]
|
| 401/1 |
2026/08/08 04:49 |
action |
format-fixes |
0m
Results: map[Fixes:map[AuthorEmail:fdmanana@suse.com AuthorName:Filipe Manana Hash:79bd37120b149532af5b21953643ed74af69654f Title:btrfs: rework chunk allocation to avoid exhaustion of the system chunk array]]
|
| 402/1 |
2026/08/08 04:49 |
action |
get-maintainers |
0m
Results: map[Recipients:[map[Email:clm@fb.com Name:Chris Mason To:true] map[Email:dsterba@suse.com Name:David Sterba To:true] map[Email:linux-btrfs@vger.kernel.org Name: To:true] map[Email:linux-kernel@vger.kernel.org Name: To:false] map[Email:fdmanana@suse.com Name:Filipe Manana To:true]]]
|
| 403/1 |
2026/08/08 04:49 |
action |
get-recent-commits |
0m
Results: map[RecentCommits:btrfs: zoned: fix missing chunk metadata reservation
btrfs: print a message when a missing device re-appears
btrfs: use mapping shared locking for reading super block
btrfs: switch local indicator variables to bools
btrfs: avoid unnecessary dev stats updates
btrfs: always update/create the dev stats item when adding a new device
btrfs: remove the dev stats item when removing a device
btrfs: remove the dev stats item for replace target device
btrfs: limit size of bios submitted from writeback
btrfs: add missing unlikely to if branches leading to a DEBUG_WARN()
btrfs: use the enums instead of int type in struct btrfs_block_group fields
btrfs: balance: fix potential bg lookup failure in btrfs_may_alloc_data_chunk()
btrfs: balance: fix potential bg lookup failure in chunk_usage_range_filter()
btrfs: balance: fix potential bg lookup failure in chunk_usage_filter()
btrfs: use a kmem_cache for free space control structures
btrfs: reduce size of struct btrfs_block_group
btrfs: use a kmem_cache for block groups
btrfs: fix check_chunk_block_group_mappings() to iterate all chunk maps
btrfs: tag as unlikely if statements that check for fs in error state
btrfs: do not reject a valid running dev-replace
]
|
| 404/1 |
2026/08/08 04:49 |
agent |
description-generator |
0m
Model:
Instruction:
You are an experienced Linux kernel developer tasked with writing a commit description for
a kernel bug fixing commit. The description should start with a one-line summary,
and then include description of the bug being fixed, and how it's fixed by the provided patch.
The one-line summary should describe the change being made, rather than mention the tool that
detected the bug.
The description must not contain lines starting with '#' because they will dropped by git as comments.
The description must not contain lines starting with '--' or '---' (including inline code diffs)
because they may confuse git/patch utilities.
Your final reply should contain only the text of the commit description.
The one-line summary must be not longer than 72 characters.
IMPORTANT: Do not wrap lines manually (e.g., at 80 characters); we will reformat the text
automatically, so keep paragraphs as single lines without newlines.
Generally try to phrase the description without mentioning syzkaller
(avoid phrases like "the bug was triggered by syzkaller" or "the bug was triggered by fuzzer", etc).
How the bug was triggered is generally an irrelevant detail.
Any bug triggered by a fuzzer can also be triggered by a malicious user, or a buggy program.
If the crash is reported by a sanitizer (e.g., KASAN, KMSAN, lockdep), include the relevant
parts of the sanitizer output to illustrate the problem. Exclude less relevant sections,
as the stack trace can be very long. Describe the execution path that leads to the manifestation
of the kernel bug.
If the patch removes the WARN_ON macro, refer to the fact that WARN_ON
must not be used for conditions that can legitimately happen, and that pr_err
should be used instead if necessary.
Don't assume that panic_on_warn is set, and that WARNINGs are fatal.
While panic_on_warn may be set when the bug was reproduced, it's generally not set on production systems.
Prefer calling several tools at the same time to save round-trips.
Prompt:
The crash that corresponds to the bug is:
BTRFS: device fsid d552757d-9c39-40e3-95f0-16d819589928 devid 1 transid 8 /dev/loop2 (7:2) scanned by syz.2.31 (5828)
------------[ cut here ]------------
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526, CPU#0: syz.2.31/5828
Modules linked in:
CPU: 0 UID: 0 PID: 5828 Comm: syz.2.31 Not tainted syzkaller #1 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:btrfs_remove_chunk+0xc9b/0x1070 fs/btrfs/volumes.c:3526
Code: 24 fb 74 12 83 3c 24 f4 75 1a e8 60 fb d4 fd eb 1c e8 59 06 5b 07 e8 54 fb d4 fd eb 10 e8 4d fb d4 fd eb 09 e8 46 fb d4 fd 90 <0f> 0b 90 48 8b 04 24 89 c1 f7 d9 e9 43 fc ff ff 44 89 f1 80 e1 07
RSP: 0018:ffffc9000427f960 EFLAGS: 00010293
RAX: ffffffff83ec5d5a RBX: ffffffffffffffe4 RCX: ffff88810db60000
RDX: 0000000000000000 RSI: ffffffff8e764c70 RDI: 00000000ffffffe4
RBP: ffffc9000427fa98 R08: ffff88810db60000 R09: 0000000000000003
R10: 00000000fffffffb R11: 0000000000000000 R12: 1ffff9200084ff3c
R13: ffff888115098000 R14: ffff888115098001 R15: dffffc0000000000
FS: 00007f16e863e6c0(0000) GS:ffff8881a6abe000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559e6665f088 CR3: 00000001fcc2c000 CR4: 0000000000352ef0
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f16e779e0d9
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f16e863e028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f16e7a25fa0 RCX: 00007f16e779e0d9
RDX: 0000200000001200 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007f16e7835024 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f16e7a26038 R14: 00007f16e7a25fa0 R15: 00007fff69061918
</TASK>
The explanation of the root cause of the bug is:
An analysis of the crash reveals that it is caused by a transaction abort (`-ENOSPC`) in `btrfs_remove_chunk` during a chunk relocation/balance operation. The abort happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
### Root Cause
1. **Order of Operations**: During chunk removal, `btrfs_remove_chunk` first removes the device extents from the device tree by calling `btrfs_remove_dev_extents`.
2. **System Space Reservation**: It then calls `check_system_chunk` to reserve system space for removing the chunk item from the chunk tree. If the system space is full, `check_system_chunk` attempts to allocate a new system chunk.
3. **Commit Root Search**: To allocate a new chunk, `find_free_dev_extent` is called. However, `find_free_dev_extent` searches the *commit root* of the device tree to prevent reusing space freed in the current transaction (which could lead to corruption if the transaction aborts).
4. **Invisible Freed Space**: Because the device extents were just freed in the *current* transaction by `btrfs_remove_dev_extents`, `find_free_dev_extent` does not see this freed space. If the device is otherwise full, it returns `-ENOSPC`.
5. **Ignored Failure**: `check_system_chunk` ignores the failure to allocate a system chunk and does not reserve any space in `trans->chunk_bytes_reserved`.
6. **Transaction Abort**: `remove_chunk_item` is then called, which fails with `-ENOSPC` because it needs to COW a node in the chunk tree but no system space was reserved. Because `btrfs_remove_dev_extents` has already modified the device tree, `btrfs_remove_chunk` cannot fail gracefully (returning `-ENOSPC` would leave the filesystem in an inconsistent state with the chunk item present but device extents missing). Thus, it is forced to abort the transaction.
### The Fix
We must ensure that we have enough system space *before* we modify the device tree. However, we cannot simply move the `check_system_chunk` call before `btrfs_remove_dev_extents` and keep the reservation, because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations (`btrfs_trans_release_chunk_metadata`).
Instead, we can call `check_system_chunk` early to *ensure* the system `space_info` has enough free space (allocating a system chunk if necessary), and then immediately release the `chunk_block_rsv` reservation. If it fails to allocate a system chunk, we can fail gracefully by returning `-ENOSPC` before any trees are modified. Then, after `btrfs_remove_dev_extents`, we call `check_system_chunk` again to make the actual reservation for `remove_chunk_item`.
Here is the proposed fix for `fs/btrfs/volumes.c` in `btrfs_remove_chunk`:
```c
map = em->map_lookup;
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ u64 chunk_bytes_reserved = trans->chunk_bytes_reserved;
+ check_system_chunk(trans, map->type);
+ if (trans->chunk_bytes_reserved == chunk_bytes_reserved) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ ret = -ENOSPC;
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
/*
* Take the device list mutex to prevent races with the final phase of
* a device replace operation...
*/
mutex_lock(&fs_devices->device_list_mutex);
for (i = 0; i < map->num_stripes; i++) {
// ... existing btrfs_remove_dev_extents logic ...
}
mutex_unlock(&fs_devices->device_list_mutex);
trans->removing_chunk = true;
mutex_lock(&fs_info->chunk_mutex);
check_system_chunk(trans, map->type);
ret = remove_chunk_item(trans, map, chunk_offset);
```
### Why this works:
1. **Graceful Failure**: By checking `trans->chunk_bytes_reserved`, we can detect if `check_system_chunk` failed to allocate a system chunk. If it failed, we return `-ENOSPC` *before* any trees are modified, allowing the balance operation to fail cleanly instead of crashing the kernel.
2. **No Metadata Leaks**: We immediately release the early reservation using `btrfs_trans_release_chunk_metadata(trans)`.
3. **Safe from Nested Allocations**: Because we release the reservation, we don't care if `btrfs_remove_dev_extents` triggers a nested chunk allocation that clears `trans->chunk_bytes_reserved`.
4. **Guaranteed Space**: The early `check_system_chunk` ensures the system `space_info` has enough free space (by allocating a 32MB system chunk if needed). When we call `check_system_chunk` the second time, it will see the free space in the `space_info` and successfully reserve the few KB needed for `remove_chunk_item`.
The diff of the bug fix is:
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 8def7abb7..b99bd7bdf 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -4488,9 +4488,9 @@ static u64 get_profile_num_devs(const struct btrfs_fs_info *fs_info, u64 type)
return num_dev;
}
-static void reserve_chunk_space(struct btrfs_trans_handle *trans,
- u64 bytes,
- u64 type)
+static int reserve_chunk_space(struct btrfs_trans_handle *trans,
+ u64 bytes,
+ u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_space_info *info;
@@ -4565,13 +4565,15 @@ static void reserve_chunk_space(struct btrfs_trans_handle *trans,
if (!ret)
trans->chunk_bytes_reserved += bytes;
}
+
+ return ret;
}
/*
* Reserve space in the system space for allocating or removing a chunk.
* The caller must be holding fs_info->chunk_mutex.
*/
-void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
+int check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
{
struct btrfs_fs_info *fs_info = trans->fs_info;
const u64 num_devs = get_profile_num_devs(fs_info, type);
@@ -4581,7 +4583,7 @@ void check_system_chunk(struct btrfs_trans_handle *trans, u64 type)
bytes = btrfs_calc_metadata_size(fs_info, num_devs) +
btrfs_calc_insert_metadata_size(fs_info, 1);
- reserve_chunk_space(trans, bytes, type);
+ return reserve_chunk_space(trans, bytes, type);
}
/*
diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h
index 790c2d467..43800523b 100644
--- a/fs/btrfs/block-group.h
+++ b/fs/btrfs/block-group.h
@@ -378,7 +378,7 @@ int btrfs_chunk_alloc(struct btrfs_trans_handle *trans,
struct btrfs_space_info *space_info, u64 flags,
enum btrfs_chunk_alloc_enum force);
int btrfs_force_chunk_alloc(struct btrfs_trans_handle *trans, u64 type);
-void check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
+int check_system_chunk(struct btrfs_trans_handle *trans, const u64 type);
void btrfs_reserve_chunk_metadata(struct btrfs_trans_handle *trans,
bool is_item_insertion);
u64 btrfs_get_alloc_profile(struct btrfs_fs_info *fs_info, u64 orig_flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73..e77306e09 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3463,6 +3463,24 @@ int btrfs_remove_chunk(struct btrfs_trans_handle *trans, u64 chunk_offset)
return PTR_ERR(map);
}
+ /*
+ * Check if we have enough system space to remove the chunk.
+ * If we don't, and we can't allocate a new system chunk, we fail gracefully
+ * before modifying the device tree.
+ */
+ mutex_lock(&fs_info->chunk_mutex);
+ ret = check_system_chunk(trans, map->type);
+ if (ret) {
+ mutex_unlock(&fs_info->chunk_mutex);
+ goto out;
+ }
+ /*
+ * Release the block reserve. We just wanted to ensure the system
+ * space_info has enough free space. We will reserve it again later.
+ */
+ btrfs_trans_release_chunk_metadata(trans);
+ mutex_unlock(&fs_info->chunk_mutex);
+
ret = btrfs_remove_dev_extents(trans, map);
if (ret)
goto out;
Additional description of the patch:
I have implemented a fix for the transaction abort issue in `btrfs_remove_chunk`.
### What I did in the patch:
1. **Changed `check_system_chunk` and `reserve_chunk_space` to return `int`:** Instead of returning `void` and ignoring errors, these functions now return an `int` error code if they fail to allocate a system chunk. This is the standard, idiomatic way to handle errors in BTRFS, avoiding the need for hacky workarounds like checking `trans->chunk_bytes_reserved` before and after the call.
2. **Early System Space Check:** In `btrfs_remove_chunk`, I added an early call to `check_system_chunk` *before* `btrfs_remove_dev_extents` modifies the device tree. We check the return value directly.
3. **Graceful Failure:** If the early `check_system_chunk` fails (e.g., returns `-ENOSPC`), we can now fail gracefully and return the error before any irreversible tree modifications have occurred, preventing the transaction abort.
4. **Immediate Reservation Release:** If the early check succeeds, we immediately release the reserved metadata space using `btrfs_trans_release_chunk_metadata(trans)`. This is crucial because `btrfs_remove_dev_extents` can trigger nested chunk allocations which unconditionally release all chunk metadata reservations. We then proceed to call `check_system_chunk` a second time later in the function to make the actual reservation, which is now guaranteed to succeed since we ensured the system `space_info` has enough free space.
### Why this is the best approach:
- **Idiomatic Error Handling:** By modifying `check_system_chunk` to return an error code, we align with BTRFS's standard error handling practices. Other callers of `check_system_chunk` (like `btrfs_inc_block_group_ro` and `do_chunk_alloc`) can safely continue to ignore the return value, as they don't require the same strict early-failure guarantees.
- **Prevents Transaction Aborts:** The root cause of the crash was that `btrfs_remove_chunk` was forced to abort the transaction because it couldn't fail gracefully after modifying the device tree. By checking for space *before* modifying the tree, we allow the balance/relocation operation to fail cleanly with `-ENOSPC`.
- **Safe from Nested Allocations:** The "reserve and immediately release" pattern perfectly works around the fact that `btrfs_remove_dev_extents` can clear our reservations. We use the first call purely to guarantee that the system `space_info` has enough capacity (allocating a new system chunk if necessary), and the second call to make the actual reservation for the chunk tree modification.
Here are summaries of recent commits that touched the same files.
Format the summary line consistently with these, look how prefixes
are specified, letter capitalization, style, etc.
btrfs: zoned: fix missing chunk metadata reservation
btrfs: print a message when a missing device re-appears
btrfs: use mapping shared locking for reading super block
btrfs: switch local indicator variables to bools
btrfs: avoid unnecessary dev stats updates
btrfs: always update/create the dev stats item when adding a new device
btrfs: remove the dev stats item when removing a device
btrfs: remove the dev stats item for replace target device
btrfs: limit size of bios submitted from writeback
btrfs: add missing unlikely to if branches leading to a DEBUG_WARN()
btrfs: use the enums instead of int type in struct btrfs_block_group fields
btrfs: balance: fix potential bg lookup failure in btrfs_may_alloc_data_chunk()
btrfs: balance: fix potential bg lookup failure in chunk_usage_range_filter()
btrfs: balance: fix potential bg lookup failure in chunk_usage_filter()
btrfs: use a kmem_cache for free space control structures
btrfs: reduce size of struct btrfs_block_group
btrfs: use a kmem_cache for block groups
btrfs: fix check_chunk_block_group_mappings() to iterate all chunk maps
btrfs: tag as unlikely if statements that check for fs in error state
btrfs: do not reject a valid running dev-replace
Reply: btrfs: check system chunk space before removing device extents
During a chunk relocation/balance operation, a transaction abort with
-ENOSPC can occur in btrfs_remove_chunk(). This happens because the
filesystem runs out of system space while trying to remove a chunk item
from the chunk tree, and it cannot allocate a new system chunk.
The crash trace is as follows:
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
</TASK>
The sequence of events leading to the abort is as follows. First,
btrfs_remove_chunk() removes the device extents from the device tree by
calling btrfs_remove_dev_extents(). It then calls check_system_chunk() to
reserve system space for removing the chunk item from the chunk tree. If
the system space is full, check_system_chunk() attempts to allocate a new
system chunk. To allocate a new chunk, find_free_dev_extent() is called.
However, find_free_dev_extent() searches the commit root of the device tree
to prevent reusing space freed in the current transaction. Because the
device extents were just freed in the current transaction by
btrfs_remove_dev_extents(), find_free_dev_extent() does not see this freed
space. If the device is otherwise full, it returns -ENOSPC.
check_system_chunk() ignores the failure to allocate a system chunk and
does not reserve any space. Finally, remove_chunk_item() is called, which
fails with -ENOSPC because it needs to COW a node in the chunk tree but no
system space was reserved. Since btrfs_remove_dev_extents() has already
modified the device tree, btrfs_remove_chunk() cannot fail gracefully and
is forced to abort the transaction.
To fix this, we must ensure that we have enough system space before
modifying the device tree. We change check_system_chunk() and
reserve_chunk_space() to return an int error code. In btrfs_remove_chunk(),
we call check_system_chunk() early. If it fails, we can fail gracefully and
return -ENOSPC before any irreversible tree modifications occur. If it
succeeds, we immediately release the reserved metadata space using
btrfs_trans_release_chunk_metadata(). This is necessary because
btrfs_remove_dev_extents() can trigger nested chunk allocations which
unconditionally release all chunk metadata reservations. We then proceed to
call check_system_chunk() a second time later in the function to make the
actual reservation, which is now guaranteed to succeed since we ensured the
system space_info has enough free space.
|
| 405/2 |
2026/08/08 04:49 |
llm |
description-generator |
0m
Model:
Tokens:
input: 9382
output: 900
thoughts: 4225
Thoughts: **btrfs: check system chunk space before removing device extents**
During a chunk relocation/balance operation, a transaction abort with -ENOSPC can occur in btrfs_remove_chunk(). This happens because the filesystem runs out of system space while trying to remove a chunk item from the chunk tree, and it cannot allocate a new system chunk.
The crash trace is as follows:
WARNING: fs/btrfs/volumes.c:3526 at btrfs_remove_chunk+0xc9b/0x1070
Call Trace:
<TASK>
btrfs_relocate_chunk_finish fs/btrfs/volumes.c:3614 [inline]
btrfs_relocate_chunk+0x3e5/0x810 fs/btrfs/volumes.c:3668
__btrfs_balance+0x1b5f/0x29d0 fs/btrfs/volumes.c:4586
btrfs_balance+0xaa6/0x1180 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance+0x3da/0x640 fs/btrfs/ioctl.c:3481
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x15f/0x560 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
</TASK>
The sequence of events leading to the abort is as follows. First, btrfs_remove_chunk() removes the device extents from the device tree by calling btrfs_remove_dev_extents(). It then calls check_system_chunk() to reserve system space for removing the chunk item from the chunk tree. If the system space is full, check_system_chunk() attempts to allocate a new system chunk. To allocate a new chunk, find_free_dev_extent() is called. However, find_free_dev_extent() searches the commit root of the device tree to prevent reusing space freed in the current transaction. Because the device extents were just freed in the current transaction by btrfs_remove_dev_extents(), find_free_dev_extent() does not see this freed space. If the device is otherwise full, it returns -ENOSPC. check_system_chunk() ignores the failure to allocate a system chunk and does not reserve any space. Finally, remove_chunk_item() is called, which fails with -ENOSPC because it needs to COW a node in the chunk tree but no system space was reserved. Since btrfs_remove_dev_extents() has already modified the device tree, btrfs_remove_chunk() cannot fail gracefully and is forced to abort the transaction.
To fix this, we must ensure that we have enough system space before modifying the device tree. We change check_system_chunk() and reserve_chunk_space() to return an int error code. In btrfs_remove_chunk(), we call check_system_chunk() early. If it fails, we can fail gracefully and return -ENOSPC before any irreversible tree modifications occur. If it succeeds, we immediately release the reserved metadata space using btrfs_trans_release_chunk_metadata(). This is necessary because btrfs_remove_dev_extents() can trigger nested chunk allocations which unconditionally release all chunk metadata reservations. We then proceed to call check_system_chunk() a second time later in the function to make the actual reservation, which is now guaranteed to succeed since we ensured the system space_info has enough free space.
|