How to rebalance your btrfs filesystem on your Linux data center servers

If you find your btrfs file systems not behaving properly, they might need to be rebalanced. Jack Wallen shows you how.

linuxhero1.jpg

Image: Jack Wallen

The btrfs file system is quickly becoming more widespread. It's used on a number of Linux distributions and offers plenty of features that make sense in a data center environment--features like snapshots, load balancing, online defragmentation, pooling, and error detection. 

To get the most out of the btrfs file system, you're going to need to know how to use some of the more advanced features. One such feature is called balancing (or rebalancing). 

SEE: MSP best practices: Server deployment checklist (TechRepublic Premium)

Why you need balancing

The btrfs file system uses a two-stage allocator. The first stage allocates large chunks for specific types of data (data, metadata, system). The second stage allocates smaller blocks, such as a file system. 

What can happen, however, is when a file system runs out of room for data or metadata and cannot allocate a new chunk. 

How to run a test

When your system runs out of room for the creation of new chunks, things will go sideways very quickly. When that happens, you turn to the command line. In order to test to see if you have a system that's run out of room, you can issue the command:

btrfs filesystem df MOUNT

Where MOUNT is the mount point the btrfs file system is attached to. If you're btrfs file system is mounted at /data, you'd issue the  command:

btrfs filesystem df /data

You should see a line of output that looks like:

Data, single: total=50.00GiB, used=45GiB

So we have 50 GB - 45 GB = 5 GB, which equates to 90% usage. In this example, a rebalance could free up space on the file system.

How to rebalance btrfs

When you balance btrfs it sends everything back through the allocator and compacts the chunks. If you have two metadata chunks that are both 45%, after running a balance, you'll have only a single metadata chunk that is 90% full. The balance will then delete the newly-emptied chunks to free up room for allocating new chunks.

This is an important task that can save you quite the headache. How do you run the balance? Let's stick with our /data example. Say you've found the data partition's metadata chunks to be full and need to run a balance. To do that, you'd issue the command:

sudo btrfs balance /data

Here's the caveat: The balance can take quite some time, depending on how much data is on the partition and how full the chunks are, and could possibly disrupt operations. Because of this, you'll want to run the balance at a time when a disruption wouldn't cause any problems.

Once the balance completes, you should see something like:

Done, had to relocate 12 out of 12 chunks

When you see that, your btrfs filesystem is back in balance and shouldn't give you any problems--at least with regards to chunk allocation. Rerun the command:

btrfs filesystem df /data

You should now see a much more acceptable difference between total and used.

You can also run the balance with the usage filter. This filter allows you to increase or decrease the usage filter, which balances only block groups with usage under the given percentage. For example, say you want to balance only the blocks with a usage under 85%. To do that, you'd run the balance command like:

sudo btrfs balance start -dusage=85 /data

Of course, it's actually much more complicated than that. In some circles it is said that you shouldn't do manual rebalancing, but if you come up against a situation where btrfs has run out of space for the allocation of chunks, you might not have any choice.

At least now you can save yourself the headache of downtime with your btrfs-enabled Linux system in your data center.

As with everything, you should experiment with btrfs balancing on non-production systems. Once you understand how this tool works, and have used it successfully, you can then move to your production systems.

Subscribe to TechRepublic's How To Make Tech Work on YouTube for all the latest tech advice for business pros from Jack Wallen.

Also see

By Jack Wallen

Jack Wallen is an award-winning writer for TechRepublic, The New Stack, and Linux New Media. He's covered a variety of topics for over twenty years and is an avid promoter of open source. For more news about Jack Wallen, visit his website jackwallen....