systemd: start and stop dependencies
A web application which requires a Samba-Share (cifs) via openvpn (IPv6) runs on a Debian server (jessie, systemd 215). With the naive configuration, manual starting and stopping works just fine, but rebooting doesn’t work properly.
- Debian packages:
- cifs mount in
- openvpn configuration in
When you start everything individually, it works with no problem:
systemctl start email@example.com
systemctl start apache2.service
Stopping also works fine manually:
systemctl stop apache2.service
systemctl stop firstname.lastname@example.org
But with a reboot, you can’t start up…
- When starting, the mount is attempted before the VPN is available and, of course, it fails.
- Apache does run, but the web application has a problem, as soon as it tries to access files in the cifs mount.
… or shut down:
- The VPN was interrupted before the umount was completed.
- The system freezes for 2 minutes until the umount times out. Very annoying.
The solution shouldn’t be too difficult with systemd units.
Since no dependencies can be explicitly formulated in
/etc/fstab, a unit file is created for the mount. We start with the automatically generated unit file:
systemctl cat var-www-data.mount > /etc/systemd/system/var-www-data.mount
And we delete the entries
Documentation, so that we’re left with just the minimum:
#/etc/systemd/system/var-www-data.mount [Unit] Before=remote-fs.target [Mount] What=//v6.smb.example.com/WwwDataShare Where=/var/www/data Type=cifs Options=ro,guest,iocharset=utf8
Then the entry from
/etc/fstab is deleted, and the unit is activated with:
systemctl enable var-www-data.mount; systemctl daemon-reload
It should now be possible to properly mount and unmount
Now we can formulate the dependency on openvpn:
#/etc/systemd/system/var-www-data.mount [Unit] Before=remote-fs.target Afteremail@example.com Requiresfirstname.lastname@example.org
Still doesn’t work, unfortunately. After various analyses with
journalctl -f, it became apparent that
- when it starts, the openvpn unit thinks that it is “finished”, even though the IPv6 routes haven’t been correctly set, yet.
- when it stops, the umount of /var/www/data thinks it is finished, even though it still has pending network communication.
First, it needs a script to check the preconditions for starting (
ping) and stopping (
#!/bin/bash LOGGER_RED="systemd-cat -t $0 -p err" LOGGER_BOLD="systemd-cat -t $0 -p notice" LOGGER_NORM="systemd-cat -t $0 -p info" case "$1" in ping) while true do if ping6 -q -c 3 v6.smb.example.com > /dev/null 2>&1 then echo ping6 ok | $LOGGER_BOLD exit 0 else echo "." | $LOGGER_NORM sleep 4 fi done ;; umount) while true do if mount | grep "//v6.smb.example.com/" >/dev/null 2>&1 then echo ";" | $LOGGER_NORM sleep 4 else echo umount ok | $LOGGER_BOLD exit 0 fi done ;; *) echo "invalid call: $1" | tee | $LOGGER_RED exit 1 ;; esac
For this script, we are building our own unit:
# /etc/systemd/system/vpnbarrier.service [Unit] Requiresemail@example.com Afterfirstname.lastname@example.org [Service] Type=oneshot RemainAfterExit=yes ExecStart=/etc/openvpn/checks_updown ping ExecStop=/etc/openvpn/checks_updown umount [Install] RequiredBy=var-www-data.mount
RemainAfterExit=yes causes (along with
ExecStop to only be executed when this unit is stopped. Without this setting,
ExecStop would be executed immediately after
ExecStart, while start-up is still in progress.
Expand dependencies in the
# /etc/systemd/system/var-www-data.mount [Unit] Before=apache2.service Requires=vpnbarrier.service After=vpnbarrier.service [Mount] ... [Install] RequiredBy=apache2.service
systemctl enable vpnbarrier.service; systemctl daemon-reload
Only now does everything work as it should:
systemctl stop email@example.com: First apache2 was stopped, then
/var/www/datawas suspended, and only after that, the VPN was stopped.
systemctl start apache2.service: First the VPN was started, then the cifs-mount and finally apache2.
- A reboot takes just about 20 seconds.
- systemd documentation is really good, for example with digitalocean. However, the emphasis is usually on starting up, and it’s hard to find anything on shutting down.
- A somewhat different discussion has shown that there are also others that think something is missing.
- The most important insight: Always use both
After. That’s the only way to get shut-down under control, too.
- Root cause: Is the problem a generic systemd problem or a bug in the openvpn package? And with cifs umount?
- Is there a somewhat smarter solution to the problem, e.g. , forgoing a separate
vpnbarrier.serviceunit and properly integrating the “ping”- and “umount” check with entries in
- Occasionally, it wasn’t enough to do a
systemd daemon-reload, and a reboot had to be done before everything worked as expected.