systemd units#

The package installs three services, one timer and one systemd generator, and enables them by symlink rather than from postinst, so the package works inside a live-build chroot where systemctl cannot run. Everything about the agent's command line is in one unit file, which is why systemctl cat wheelhouse-agent is the fastest way to find out what a router is actually running.

wheelhouse-agent.service#

The daemon. Type=simple, User=root.

After=network-online.target vyos-router.service wheelhouse-firstboot.service
Wants=network-online.target wheelhouse-firstboot.service
ExecStartPre=/usr/lib/wheelhouse/wait-for-vyos.sh
TimeoutStartSec=600
Restart=on-failure
RestartSec=5
KillSignal=SIGTERM
TimeoutStopSec=20

Its ExecStart passes ten flags and nothing else:

--api-url https://127.0.0.1
--api-key-file /config/wheelhouse/api-key
--admin-token-file /config/wheelhouse/admin-token
--license-key-file /config/wheelhouse/license-key
--addr 0.0.0.0:8443
--tls-self-signed
--ui-dir /usr/share/wheelhouse/ui
--data-dir /config/wheelhouse
--log-level info

Every secret is a file under /config/wheelhouse, never a flag value, because ExecStart is world-readable through /proc. The break-glass token and the licence file are optional: the unit names both paths unconditionally and the agent degrades to "no break-glass" and "unlicensed" when they are absent.

SIGTERM is deliberate — the agent drains in-flight requests for up to 15 seconds rather than cutting an operator off mid-commit, and TimeoutStopSec=20 gives it room.

Why it waits#

vyos-router.service is Type=simple with RemainAfterExit, so systemd calls it active the moment it forks — while /config is still being mounted and the configuration is still being applied several seconds later. Anything merely After=vyos-router.service that touches /config writes into a directory about to be mounted over. So both the agent and first boot run wait-for-vyos.sh first, which polls the unit's SubState until it reads exited — the signal that the configuration is loaded.

It gives up after five minutes, and it gives up immediately if the unit reports failed, because waiting out the full timeout on a box where the router service has already died only delays the journal line that says why.

Hardening#

NoNewPrivileges=true      ProtectKernelTunables=true    RestrictRealtime=true
ProtectSystem=strict      ProtectKernelModules=true     LockPersonality=true
ProtectHome=true          ProtectControlGroups=true     SystemCallArchitectures=native
PrivateTmp=true           RestrictSUIDSGID=true
ReadWritePaths=/config
MemoryMax=200M
CPUQuota=300%

ProtectSystem=strict makes the whole filesystem read-only, so ReadWritePaths has to name every path the agent writes. It names the whole of /config rather than the two subdirectories that would be tighter, and the unit says why in its own comment: /config is a partition VyOS mounts during boot, and systemd builds this unit's mount namespace before that happens, so /config/wheelhouse does not exist yet at namespace-setup time. Naming it fails the unit; naming it with a - prefix makes systemd skip it and leaves the agent with nothing writable at all.

wheelhouse-firstboot.service#

After=vyos-router.service
Requires=vyos-router.service
Before=wheelhouse-agent.service
ConditionPathExists=!/config/wheelhouse/api-key
Type=oneshot
RemainAfterExit=yes
TimeoutStartSec=600

It runs until it has produced the API key and then never again — including after an image upgrade, because /config survives one and the condition stays false. The script refuses to write the key file unless the commit really landed, so a failed run leaves the condition true and the next boot tries again rather than starting an agent holding a key the router never accepted. See What first boot does.

wheelhouse-console.service and its timer#

Before=getty-pre.target getty.target getty@tty1.service serial-getty@ttyS0.service
After=vyos-router.service
Type=oneshot
ExecStart=/usr/lib/wheelhouse/console-banner.sh
WantedBy=multi-user.target getty.target

agetty reads /etc/issue once, when it opens the terminal. If the banner lands after that, the first screen a new box ever shows is the stock one, and it stays until somebody presses enter — so the unit is ordered before the getty units by name. Ordering only Before=getty.target is not enough: getty@.service and serial-getty@.service both carry Before=getty.target themselves, so the target is reached after them.

WantedBy names two targets on purpose. multi-user.target is what makes it run at all; getty.target is what makes the ordering bite, because ordering only constrains units already in the same transaction. Wants=, not Requires=: a banner that cannot be written must never keep the login prompt off the console.

The package pre-creates only this unit's multi-user.target.wants symlink. The getty.target want in its [Install] section materialises the ordinary way, when the unit is enabled with systemctl enable wheelhouse-console.service.

It is deliberately not ordered on network-online.target or on the agent. The agent waits for the configuration to load and can take minutes, and every getty would wait with it, leaving the console black. The banner draws with what it knows, and the timer fills in the addresses as they arrive.

[Timer]
OnBootSec=0
OnUnitActiveSec=20s
AccuracySec=5s

OnBootSec=0 rather than a delay: the service has normally already run by the time timers.target is reached, so on the boots where it has not — a getty started early, an address that arrived late — the first refresh is immediate instead of a quarter of a minute of stale banner.

wheelhouse-live-installer#

A systemd generator at /lib/systemd/system-generators/wheelhouse-live-installer. It exits immediately unless /usr/lib/live/mount/medium/live/filesystem.squashfs exists — that is, unless this is a live boot. On a live boot it drops one file:

# getty@tty1.service.d/wheelhouse-installer.conf
[Unit]
Description=Wheelhouse installer on tty1 (live image)
[Service]
ExecStart=
ExecStart=-/usr/bin/wheelhouse-install --auto
StandardOutput=tty
StandardError=tty
Environment=TERM=linux

A generator rather than a competing unit, because two units that both want tty1 in one boot transaction are resolved in an order that is not ours to choose — and that fight was lost on real hardware. This way getty@tty1 keeps its restart-forever semantics: when the installer hands the console back as a login prompt and that session ends, the installer returns. Installed systems keep the normal login prompt.

Both streams are the terminal because dialog draws on stdout and reports its answers on stderr.

What postinst does#

if [ "$1" = configure ] && [ -d /run/systemd/system ]; then
    systemctl daemon-reload || true
    systemctl start wheelhouse-console.timer || true
    systemctl try-restart wheelhouse-agent.service || true
fi

Every call is allowed to fail, because inside the image build's chroot there is no systemd to talk to. On a running router the effect is that installing the package picks up new units and restarts the agent, so an upgrade takes effect without a reboot. prerm stops the agent on removal, under the same guard.

Documentation= on all three services points at file:/usr/share/doc/wheelhouse-agent/README, the one file on the box that says what this is and where the corresponding source comes from.

Changing the agent's flags#

Never edit /lib/systemd/system/wheelhouse-agent.service — the next package upgrade replaces it. Use a drop-in, and remember that adding to ExecStart means clearing it first:

sudo mkdir -p /etc/systemd/system/wheelhouse-agent.service.d
sudo tee /etc/systemd/system/wheelhouse-agent.service.d/oidc.conf >/dev/null <<'EOF'
[Service]
ExecStart=
ExecStart=/usr/bin/wheelhouse-agent \
  --api-url https://127.0.0.1 \
  --api-key-file /config/wheelhouse/api-key \
  --admin-token-file /config/wheelhouse/admin-token \
  --license-key-file /config/wheelhouse/license-key \
  --addr 0.0.0.0:8443 \
  --tls-self-signed \
  --ui-dir /usr/share/wheelhouse/ui \
  --data-dir /config/wheelhouse \
  --log-level info \
  --oidc-issuer https://auth.example.com/application/o/wheelhouse/ \
  --oidc-client-id wheelhouse \
  --oidc-client-secret-file /config/wheelhouse/oidc-secret
EOF
sudo systemctl daemon-reload
sudo systemctl restart wheelhouse-agent
systemctl cat wheelhouse-agent          # confirm what is actually in force

The developer install's unit#

install/wheelhouse-agent.service is a separate, smaller unit for the lab path. It is not what ships. The shipped unit carries everything that one carries and more; the two used to disagree, with the shipping one having less.

See also#

• What first boot does
• The console banner
• Every flag — what each flag in ExecStart means.
• Files and directories — the paths the unit names.
• The .deb — how the units get installed and enabled.
• Log lines worth alerting on
• Upgrade the agent

Checked against#

packaging/wheelhouse-agent.service, packaging/wheelhouse-firstboot.service, packaging/wheelhouse-console.service, packaging/wheelhouse-console.timer, packaging/wheelhouse-live-installer, packaging/wait-for-vyos.sh, packaging/build-deb.py, install/wheelhouse-agent.service, packaging/README.md.