How to analyze systemd boot performance

Learn how to start troubleshooting systemd-enabled machines with the systemd-analyze tool.

Your data center Linux servers probably use systemd as its initialization system. To that end, you might be interested in knowing there's a tool available that can analyze the boot performance of your server. The tool is called systemd-analyze and I'm going to show you how to use it.

SEE: 10 things companies are keeping in their own data centers (TechRepublic download)

What you'll need

All you'll need to make this happen is a systemd-enabled Linux machine and a user account with sudo privileges. 

How to install systemd-analyze 

Good news. There is no installation to be had, as systemd-analyze is part of the systemd suite of tools. You can confirm the tool is installed by issuing the command:

which systemd-analyze

You should see the path to the executable command listed (Figure A).

Figure A

screen-shot-2019-11-19-at-6-12-52-pm.png

Our systemd-analyze executable path.

How to analyze systemd

Now we run the analysis command. First let's find out the actual boot time of the machine by running the command with no arguments, like so:

sudo systemd-analyze

As you can see (in Figure B) the total boot time of my System76 Thelio is 55.252 seconds.

Figure B

screen-shot-2019-11-19-at-6-10-04-pm.png

Total boot time for the machine.

This boot time is broken down into:

  • Firmware: 33.568 seconds.

  • Boot loader: 8.323 seconds 

  • Kernel: 5.555 seconds

  • Userspace: 7.805 seconds

If that time is too long for you, how do you find out what's to blame? I'm glad you asked. Issue the command:

sudo systemd-analyze blame

The above command will list out every single service that starts up on the machine, and the time it takes (Figure C). For example, on my machine the biggest offenders are:

  • NetworkManager-wait-online: 6.401 seconds

  • Plymouth-quit-wait: 2.793 seconds

  • E2scrub_all: 2.002 seconds

  • Snap-core: 283 ms

  • Snap-bluemail: 253 ms

  • Udisks2: 251 ms

  • Systemd-resolved: 250 ms

  • Logrotate: 238 ms

  • Dev-nvme0n1p3: 232 ms

Figure C

screen-shot-2019-11-19-at-6-16-38-pm.png

All services and their boot time.

That list continues to "blame" quite a large number of services--most of which are under 100 ms. Clearly I have nothing that's causing any long delays during startup. However, should a service be causing such a delay, it'll be listed and you can begin to troubleshoot why.

You can also have the command print out the results in a chain of events style with the command:

sudo systemd-analyze critical-chain

The results will display the time when a unit became active, as a chain of events (Figure C).

Figure C

screen-shot-2019-11-19-at-10-18-03-pm.png

The critical-chain display.

And that's how you can use the built-in systemd tool to help you begin troubleshooting why your Linux systems might be slow to boot. To learn more about the systemd-analyze tool, issue the command man systemd-analyze and read all about it.

Also see

datacenterhero.jpg

Image: Jack Wallen