Shutdown, Reboot and Services That Delay Shutdown

A VM still shows activity after a shutdown request. Is a user session inhibiting the request, a service stopping slowly, a filesystem busy, or the guest already shut down while the host display remains open? Each requires different evidence. A forced power cut can destroy the information needed to distinguish them.

This chapter uses the disposable bl03-loader-lab clone from the earlier GRUB exercise, or a fresh equivalent full powered-off clone of bl02. Keep the original shut down while its clone runs because their guest identities are copied. The lab performs normal reboot and poweroff operations on the clone. Keep the VM manager console visible and save any necessary work first.

What a clean transition attempts

A normal shutdown asks the system manager to stop services and sessions, terminate remaining processes, unmount filesystems where possible and complete the requested transition. Dependencies and ordering affect stop jobs too; stop ordering is generally the reverse of start ordering when both jobs are ordered. Some final work happens after ordinary userspace services have stopped. The systemd boot and shutdown overview describes this lifecycle.

Request	Intended outcome
Reboot	Shut down the running system and start it again
Poweroff	Shut down and request power removal
Halt	Stop the system; whether power is removed depends on platform and command behavior
Suspend	Preserve a running session for later resume; not a shutdown
Forced VM power cut	Remove virtual power without waiting for guest cooperation

The VM manager’s shut off state and a blank console are not identical observations. A guest may have powered off while the console window remains open. Conversely, a blank display does not establish that the guest has stopped writing.

Preflight the transition

On the clone’s console:

hostnamectl
cat /proc/sys/kernel/random/boot_id
systemctl is-system-running
systemctl --failed --no-pager
systemctl list-jobs
systemd-inhibit --list --no-pager

Record the VM identity from the manager and the boot ID from the guest. Inhibitors can request a block or delay for operations such as shutdown; they are distinct from a service’s stop job. The systemd-inhibit manual describes their modes. An empty list does not guarantee that every service will stop quickly.

A privileged shutdown request can behave differently from an unprivileged desktop request and may override certain inhibitors. Do not bypass an inhibitor to make this lab faster. Identify its owner and reason; wait for the legitimate operation or coordinate with its owner before a real shutdown.

Reboot and compare boot instances

With the console open, request a normal reboot:

sudo systemctl reboot

Wait for the guest to return and log in. Check:

cat /proc/sys/kernel/random/boot_id
systemctl is-system-running
systemctl --failed --no-pager
journalctl --list-boots --no-pager

The boot ID should differ from the pre-reboot value. If the journal retained the previous boot, inspect its final messages:

sudo journalctl -b -1 -e --no-pager

-b -1 means the previous boot in the available journal, not an absolute guarantee of the immediately preceding physical boot. If the journal was volatile, there may be no previous boot to inspect. Record the unavailable evidence instead of assuming a clean shutdown. Journal timestamps and final messages can show progress, but missing final messages do not alone prove an abrupt power loss.

Now request a normal poweroff:

sudo systemctl poweroff

In the host’s VM manager, wait for the clone to show shut off. If needed, check from the host:

sudo virsh --connect qemu:///system domstate bl03-loader-lab

Use the actual clone name if you created a fresh equivalent. A normal shutdown followed by an observed shut-off domain is the acceptance result. Do not select a forced power action merely because the console has stopped updating for a few seconds.

Diagnose a slow or blocked shutdown

Before starting a shutdown on an accessible system, systemd-inhibit --list, systemctl list-jobs, service status and logs show different classes of evidence. During a slow transition, the console may show a stopping job and elapsed time. Preserve the unit name and message. If another console session remains usable, inspect jobs and the affected service without restarting it; access may disappear as shutdown progresses.

Observation	Next question
A shutdown inhibitor is listed	Who holds it, what operation is underway, and is it block or delay?
A stop job waits for a service	Is the service finishing work, blocked on a dependency, or failing to exit?
A mount cannot unmount	Which process or nested mount is keeping it busy, and what data is at risk?
Domain is already shut off	Is the manager window simply still open?
A guest stops responding entirely	What was the last console evidence, and what recovery point exists?

A service stop timeout is not automatically proof that its process is safe to kill. An uninterruptible task can be waiting for I/O; storage and remote dependencies require their own investigation. Do not set all timeouts to zero, issue repeated forced resets, or kill random processes. In a real incident, weigh data consistency and availability, record the decision and use the service’s recovery procedure. In this disposable clone, the established baseline supplies a bounded reset if a controlled experiment cannot complete.

Practice and recall

Record the clone name, boot IDs before and after reboot, any inhibitors, failed units, prior-boot journal availability and final host-side domain state. Explain why a missing prior journal and an empty inhibitor list cannot each prove a clean shutdown.

• Reboot, poweroff, halt, suspend and forced power removal are different transitions.
• Inhibitors and stopping jobs are different sources of delay.
• Preserve the console message and service evidence before interrupting a stuck transition.
• Verify the guest’s actual state from the host after poweroff.