How we protect a database that holds the SSH and WP-DB credentials for ~150 SpinupWP sites, plus the OAuth2 credentials reaching ~90 more on Pressable (no SSH — see Integrations → Pressable for that provider’s different trust model). The short version: defense in depth — local-LAN-only network posture, OAuth + allowlist for humans, HMAC for plugin calls, dual-scoped CF tokens, encrypted-at-rest credentials.
Threat model in one paragraph
Clockwork Control is local-LAN-only. The DB is the highest-value target on the host: SSH private keys for every SpinupWP server, MySQL credentials for every WordPress site, plus the per-site Companion HMAC secrets and the Pressable API credentials. We assume an attacker who reaches the LAN, an attacker who steals an API token from the env, and an attacker who compromises a single managed WordPress site. Each gets a different bounded blast radius.
Network posture
- The app is bound to localhost. Don’t run
php artisan serve --host 0.0.0.0and don’t expose Herd.testdomains beyond the LAN without a deliberate decision. - The home network NAT is the actual perimeter. There is no public URL and no inbound from outside.
- Outbound calls are HTTPS to known third parties only (DigitalOcean, Hetzner Cloud, Azure, Vultr, Linode, SpinupWP, Pressable, WP Engine, Kinsta, Cloudways, Cloudflare, Bill.com, Mattermost, Slack, Twilio, Mailgun, GTmetrix, PageSpeed Insights, Companion sites, blacklist/malware-scan services, and Clockwork Control’s own anonymous telemetry endpoint — see below). LM Studio is loopback-only.
- Anonymous usage telemetry is on by default (
CLOCKWORK_TELEMETRY_ENABLED, weekly cron + viewable/one-click-opt-out on/settings/maintenance).App\Services\Telemetry\TelemetryPayloadBuilderdeliberately narrows the payload to exact site/server counts, per-module breakdowns, and an install ID — never domains, IPs, emails, or database contents. Still worth knowing about for a local-LAN-only threat model: it’s the one outbound call that’s on by default rather than opt-in.
Physical & host security
The network posture above assumes the machine itself is safe. It might not be — and that machine is a bigger prize than it looks, for a reason encryption-at-rest doesn’t fix:
- The DB holds ciphertext;
.envon the same disk holds the key that reads it.servers.ssh_private_key,servers.ssh_password,sites.db_password, andsites.companion_secretare allCrypt-cast (AES-256-GCM, derived fromAPP_KEY) — safe against someone who steals only a database dump (see “App database backups” below). ButAPP_KEYlives in.envon the exact same host that runs the app and (typically) the DB. Whoever walks off with the machine walks off with both halves at once — the ciphertext and the key — which collapses “encrypted at rest” down to roughly the protection of an unlocked filing cabinet. This is a different risk from the network posture above, and encryption-at-rest doesn’t address it at all. - Run this on a desktop, not a laptop. A laptop is designed to leave the building — a bag on a train, a car break-in, a conference. A desktop that never leaves a locked office removes the theft-in-transit scenario this app is specifically exposed to: a single stolen device would otherwise hand an attacker SSH keys and passwords for the entire managed fleet, not just this app’s own data.
- If a portable machine is genuinely unavoidable, full-disk encryption (FileVault/BitLocker/LUKS) plus a locked screen and a real login password are the floor, not the solution — they reduce the window of opportunity, they don’t change the “one stolen device = one fully compromised fleet” property above.
- Never put this app on a public IP. Reinforcing the Network posture section above: no port-forwarding, no public DNS record, no
php artisan serve --host 0.0.0.0. Combined with the point above, a publicly reachable instance turns a remote compromise into the same blast radius as a physical theft — an attacker doesn’t even need to be in the same city.
Human auth
- Dual Auth: Local Password & Allowlisted OAuth. Local email/password authentication is universally available, alongside modular OAuth single sign-on (Google, GitHub, and Microsoft Entra ID).
- Local Passwords: Stored as Bcrypt/Argon2 hashes via Eloquent’s
hashedcast onusers.password. Handled securely atPOST /loginwith rate limiting (throttle:5,1) and session regeneration. - OAuth Providers: Each implements
Modules\Core\Contracts\AuthProviderin its own module (AuthGoogle/AuthGitHub/AuthMicrosoft) and shares one allowlist/audit path throughApp\Services\Auth\OAuthLoginHandler.LoginControlleronly shows a provider’s button once it’s actually configured, avoiding broken sign-in paths when SSO is not set up.
- Local Passwords: Stored as Bcrypt/Argon2 hashes via Eloquent’s
- The
userstable is the allowlist, regardless of whether the operator authenticates via local password or OAuth. A row exists ⇒ allowed;revoked_at IS NULL⇒ active. No auto-provisioning — the user must already exist on the allowlist, or login is denied with an informative banner. - Bootstrap and recovery via
clockwork:add-user <email> [--name=] [--password=]andclockwork:set-password <email> [--password=]. Idempotent — restores revoked rows and sets or resets operator credentials via masked CLI prompts. Always-works escape hatch when the UI is locked out. - Optional Workspace pin —
GOOGLE_HD=your-agency.comrestricts the Google account picker to that domain and is re-checked on the OAuth callback (hdclaim). Off by default so personal accounts work for testing. - Revoke and password change kill live sessions.
User::invalidateSessions()cyclesremember_tokenand deletessessionsrows for that user.EnsureUserIsActivemiddleware (active) re-checksrevoked_aton every web request and logs the operator out immediately. This middleware is strictly registered on core route groups as well as all modular routes (modules/*/routes/web.phpincluding AiRemedy, Slack, Mattermost, Feedback, BillCom, ClientReports, and ContactForms), closing any window for revoked users to hit modular endpoints with unexpired cookies. Login + add/revoke/restore/password-change all land inaction_logs. - Roles.
users.roleisadmin(default for existing rows, the installer, andclockwork:add-user) oroperator. Operators can run the fleet; only admins can manage the allowlist, download the Clockwork Control DB backup, apply in-app system updates, or execute Code Snippets. New teammates added from/settings/usersdefault to operator. /dev-loginis local-loopback only.DevLoginController404s unlessAPP_ENV=localand the request is un-proxied loopback (127.0.0.1/::1/localhost, noX-Forwarded-*). The optional?redirect=query is same-origin paths starting with/only — absolute URLs,//host, encoded/%2f%2f…, and backslash variants fall back to Companion settings. Never treat this as a production login.
Users settings page (/settings/users)
App\Http\Controllers\UsersSettingsController is the UI for the allowlist described above — a row in users is the allowlist, and this page is where an admin edits that table without touching clockwork:add-user from a shell.
indexlists every user (active and revoked), active first (orderBy('revoked_at')— NULLs sort first), then by email. Indicates whether the user has a local password configured, SSO only, or both.store(POST /settings/users) adds a new allowlist row with an optional initial password, or — if the email already exists and is revoked — restores it (clearsrevoked_at, updates name and optional password) rather than erroring on a duplicate. LogsTYPE_USER_ADDEDorTYPE_USER_RESTOREDaccordingly.updatePassword(PATCH /settings/users/{user}/password) allows administrators to set or change an operator’s local password directly from the UI. LogsTYPE_USER_PASSWORD_CHANGED.revoke(PATCH /settings/users/{user}/revoke) setsrevoked_at = now(). Blocked at the controller for self-revoke —Auth::id() === $user->idbounces back with an error rather than letting you lock yourself out of the UI, since the artisan recovery path doesn’t help if you can’t reach the host to run it. LogsTYPE_USER_REVOKED.restore(PATCH /settings/users/{user}/restore) clearsrevoked_atfor an already-revoked user. LogsTYPE_USER_RESTORED.destroy(DELETE /settings/users/{user}) permanently deletes an allowlist row and invalidates any remaining sessions. Blocked for self-deletion and the last administrator. LogsTYPE_USER_DELETED.
All user actions go through App\Services\ActionLog\ActionLogger with the acting admin’s email as actor, landing in action_logs alongside every other audited action — see Architecture → Data model.
App-side encrypted columns
Crypt-cast in the model, ciphertext at rest using APP_KEY-derived AES-256-GCM:
servers.ssh_private_key,servers.ssh_passwordsites.db_password,sites.companion_secret
Never log them. Never render them in views or API responses. The companion_secret cast also marks it hidden from serialization so it can’t accidentally end up in a JSON response.
Re-generating APP_KEY breaks all encrypted columns in place. Don’t do it casually — there’s no rotation flow.
Outbound API tokens
| Token | Scope | Why scoped narrowly |
|---|---|---|
CLOCKWORK_DIGITALOCEAN_TOKEN |
Read | Used only for monitoring metrics. Never provisions or destroys droplets. |
CLOCKWORK_SPINUPWP_TOKEN |
Inventory | Used for read + the migration runner’s create-site call only. |
CLOCKWORK_CLOUDFLARE_API_TOKEN |
Zone:Read + per-phase Read | Diagnostics only. Cannot mutate DNS or firewall rules. |
CLOCKWORK_CLOUDFLARE_WRITE_TOKEN |
Zone.DNS:Edit + Zone.Firewall Services:Edit | Migration cutover (DNS) and rate-limit/custom-WAF-rule writes. Independently rotatable from the read token — a leak of the read token grants neither. See Integrations → Cloudflare for why it needs two scopes, not one. |
CLOCKWORK_DO_SPACES_KEY/SECRET |
Bucket scope | S3-style HMAC, not the DO PAT. List-only access pattern; we never write or delete. |
CLOCKWORK_BILL_COM_* |
Account login | Read-only sync. We don’t mutate Bill.com. |
CLOCKWORK_PRESSABLE_CLIENT_ID/SECRET |
OAuth2 client_credentials, account-level | No per-site scoping available — Pressable’s API doesn’t offer it. This token can reach every Pressable site on the account, including running arbitrary shell/wp-cli commands via the async command API. Treat it at the same sensitivity as an SSH private key, not as a scoped read token. |
CLOCKWORK_AZURE_CLIENT_ID/SECRET |
Reader on the subscription/resource group | Monitoring only — same posture as the DO/Hetzner tokens, just OAuth2 instead of a static bearer token. |
CLOCKWORK_VULTR_API_KEY / CLOCKWORK_LINODE_TOKEN |
Read-only | Monitoring + IP-matching only, same posture as DO/Hetzner. Vultr’s API has no metrics surface to begin with; Linode’s is CPU-only. |
CLOCKWORK_WPENGINE_SSH_PRIVATE_KEY / CLOCKWORK_KINSTA_SSH_PASSWORD |
Real per-install/per-environment SSH | Same sensitivity class as a server’s own ssh_private_key/ssh_password columns — not scoped per-site by the provider, treat with SSH-key handling discipline. The WP Engine/Kinsta REST API credentials themselves (api_user_id/api_password, bearer token) are separate and lower-stakes — inventory/read only, command execution goes over the SSH credential instead. |
CLOCKWORK_CLOUDWAYS_API_KEY |
OAuth2 (API key → bearer token), account-level | Cloudways provisions real servers on a cloud of its own choosing (DO/AWS/GCP/Vultr/Linode) that this app never holds credentials for — see Integrations → Cloudways. Command execution and Companion install go through Cloudways’ own async command API, same account-wide-reach caveat as the Pressable token below. |
Companion plugin auth
The mu-plugin sits inside each WordPress site and exposes signed REST endpoints under /wp-json/clockwork/v1/. Auth is HMAC-SHA256 with a per-site shared secret.
- Secret is 32 random bytes (
bin2hex(random_bytes(32))= 64 hex chars, 256 bits), generated Laravel-side at install, pushed intowp_optionsvia wp-cli, mirrored encrypted intosites.companion_secret. - Two storage modes:
wp_options(default; rotation viaclockwork:rotate-companion-secret) or aCLOCKWORK_COMPANION_SECRETconstant inwp-config.php(operator-pinned; rotation request returns 409secret_pinned). - Per-site secret — compromising one site doesn’t pwn the fleet.
- Encrypted at rest in Clockwork Control’s DB; in WP
wp_optionsit’s plaintext, same trust posture aswp-config.phpkeys.
Signature payload (both sides must match exactly):
METHOD\n/wp-json/clockwork/v1<route>\nTIMESTAMP\nBODY
Headers on every request: X-Clockwork Control-Signature (hex), X-Clockwork Control-Timestamp (unix seconds). Verifier rejects on missing headers, |now − ts| > 300s, missing secret, or signature mismatch (hash_equals constant-time compare). Replay window is 5 minutes. Requests with query params (other than rest_route) must also carry X-Clockwork Control-Signature-V2, signed over the path plus the canonical query, so a captured request can’t be resent with altered params. Mutating requests are marked used with a single INSERT IGNORE into wp_options, so two concurrent copies of one signed request can’t both pass.
POST bodies use Laravel’s withBody($jsonBody, 'application/json') so the byte-exact JSON we signed is what gets sent — ->post($url, $array) would re-encode and could drift.
The signature is computed over the logical route string (/wp-json/clockwork/v1<route>), not the literal URL, which matters because ClockworkCompanionClient has two transport forms for the same route: the normal pretty-permalink URL, and a ?rest_route= query-string fallback used when a site’s web server configuration is missing the WordPress REST rewrite (where /wp-json/... might resolve to homepage HTML with a 200 status instead of dispatching to WordPress — isNonJsonResponse() detects that case and retries via ?rest_route=). Because both forms hit the same logical route, the signature is identical either way and doesn’t need special-casing.
Companion-side defense in depth
- HMAC failure rate limit — 30 fails per IP per 60s → 429. Per-IP transient counter. Defends
wp_optionsand CPU from noise; not the cryptographic protocol. - HMAC failure audit log —
wp_clockwork_auth_failurestable, lazy-pruned to 1000 rows. Surfaces in Tools → Clockwork → Security → Authentication audit. - SSO nonce ceiling —
/sso/magic-linkrefuses to mint past 100 active. Preventswp_optionsbloat under abuse. - Plugin-side admin pages — The parent menu is visible to all users with
manage_options. Sensitive tools (such as the LLAR Unlock Hub and team governance actions) use strict fail-closed authorization checks (see below). REST endpoints remain HMAC-gated regardless.
Team-Wide 2FA Governance & Role-Aware Capabilities
Clockwork Companion and Renegade feature native Two-Factor Authentication (2FA) governance designed for client teams:
- Role-Aware Self-Enrollment: Self-enrollment pages use the
readcapability. This ensures that non-admin team members (such as editors, authors, and contributors) can safely configure and manage their own 2FA credentials without requiringmanage_optionsor triggering infinite 403 redirect loops. - Team-Wide Administration (
TwoFactorPage): Users withmanage_optionscan view an aggregated roster of all registered WordPress accounts, inspecting active 2FA adoption, enrolled methods, and grace period status across the entire organization. - Admin Actions: Authorized administrators can:
- Require 2FA: Flags a specific user account for mandatory 2FA enforcement, prompting them on their next login.
- Stop Requiring: Relaxes the mandatory enforcement requirement for a user while preserving any already configured credentials.
- Turn Off 2FA: Immediately wipes a user’s enrolled 2FA credentials and emergency recovery codes during client lockout recovery scenarios.
- Emergency Bypass Hatch: If an administrator or agency operator is locked out due to a broken authenticator app or email transport failure, defining
define('CLOCKWORK_2FA_DISABLE', true);inwp-config.phpcompletely bypasses 2FA checks until recovery is complete.
Fail-Closed 403 Architecture (LLAR Unlock Hub & Client Isolation)
The remote emergency lockout clearing console (Clockwork → Unlock) operates on a strict fail-closed isolation model:
- Client Site Isolation: Monitored client sites never register the Unlock submenu or navigation tab.
- Direct URL Guard: If a user attempts to bypass navigation by visiting
admin.php?page=clockwork-unlockdirectly,UnlockPage::renderBody()immediately executeswp_die(__('You do not have sufficient permissions to access this page.'), 403)before any HTML chrome or sensitive forms render. - AJAX Endpoint Gate: The background AJAX handler (
clockwork_companion_unlock_target) validates WordPress security nonces, checkscurrent_user_can('manage_options'), confirms agency staff email identity, and confirms hub domain matching. Any failure immediately returnswp_send_json_error(['message' => 'Unauthorized'], 403). - Target Site Verification: When the Unlock console triggers an unlock on a client site, it signs an HMAC-SHA256 request to
DELETE /wp-json/clockwork/v1/lockouts. The target site verifies the signature against its own encrypted secret with a 5-minute replay window before flushing lockouts from native Gatekeeper (and legacy LLAR tables if present).
Standalone Pairing & 256-Bit Cryptographic Connection Key
For sites without cloud hosting API access or SSH (e.g. WP Engine, Kinsta, or client-managed VPS):
- 256-Bit Cryptographic Secret: When either plugin variant is activated, it initializes a fresh 256-bit cryptographically secure secret using PHP’s
random_bytes(32)(encoded as a 64-character hex string). - Base64 Connection Key Envelope: The site packages the target URL, the 256-bit secret, and the plugin variant (
renegadeorcompanion) into a base64-encoded JSON payload:- Clockwork Renegade (WordPress.org): Located under Clockwork → Connection (
admin.php?page=clockwork-connection). - Clockwork Companion (Private mu-plugin): Located under Tools → Clockwork Control (or white-labeled custom menu).
- Clockwork Renegade (WordPress.org): Located under Clockwork → Connection (
- When pasted into Clockwork Control, Clockwork Control decodes the key, performs an immediate HMAC
/healthhandshake to verify mutual possession of the secret, and persists the secret encrypted at rest (sites.companion_secret). - Never flash the secret back.
SitesController::enrollSafeInput()stripscompanion_secretandconnection_keyfrom old input on every validation error. The create-site form does not repopulate those fields, and the domain goes through@js()so a crafted domain cannot break out of Alpine state. Handshake failures other than 401/404 are reported server-side; the form only shows a generic “could not connect” message.
Direct S3 Glacier Backup Upload Security
When streaming backups directly from WordPress to AWS S3 Glacier Instant Retrieval (POST /wp-json/clockwork/v1/backup/create):
- IAM Credentials Never Touch WordPress: The AWS IAM access key and secret live solely on Clockwork Control.
- Time-Limited Presigned PUT URLs: Clockwork Control signs an S3 PUT URL valid for only 2 hours.
- Strict Scope & Storage Class: The presigned URL is locked to a specific object key (
archives/{domain}/{timestamp}_{id}.zip) and enforcesx-amz-storage-class: GLACIER_IR. The WordPress site cannot read, delete, or list other objects in the bucket. - Companion refuses non-public upload URLs before it dumps the site. HTTPS only, no private/reserved IPs, no redirects, TLS hostname verified. Header names/values with CR/LF are rejected so a stolen HMAC cannot inject extra headers.
Archive-key ownership (restore / download)
A stolen or guessed S3 object key must not let an authenticated operator pull another tenant’s archive. BackupArchiveEnumerator::belongsToSite() is the single gate: the key must sit under {archive_prefix}/{domain}/ or {domain}/, and .. plus sibling-domain prefixes (example.com.evil/…) are refused. That check runs on presigned GET minting, SHA-256 sidecar lookup, the settings download proxy, UI restore stage, and clockwork:backup-restore --phase=stage.
Companion’s restore download is the same public-HTTPS rule as upload: HMAC already gates the route; the URL check stops a stolen secret from turning Companion into a metadata/SSRF client. Redirects are refused (presigned S3 GETs are direct). Zip extraction already rejects ../ and absolute paths; file apply skips those names again.
See Architecture → Companion plugin for the full picture.
Pressable transport: a different secret-exposure risk
Pressable has no SSH equivalent — the Companion secret has to reach the site some other way, and Pressable logs every command it runs verbatim in its own control panel for ~30 days. Pushing the real long-lived HMAC secret through that channel (the way SSH tarball push does over an encrypted, non-logged connection) would put it in a third party’s logs.
PressableCompanionInstaller avoids this with a bootstrap-then-rotate sequence: a throwaway secret goes through the logged install command, /health verifies the plugin loaded, then rotateSecret() replaces it with the real secret over authenticated HTTPS — a channel Pressable doesn’t log. If rotation fails, the install is reported as a loud warning rather than a silent success, precisely so a site left running on the throwaway (logged) secret doesn’t go unnoticed. See Integrations → Pressable.
SSH
- Auth precedence: per-server stored key → local default key (
CLOCKWORK_SSH_KEY_PATH) → per-server password. - Default user is
clockwork-deploy(overridable per server). Non-root, with NOPASSWD on/usr/bin/fail2ban-clientonly — granted by/etc/sudoers.d/clockworkduring fail2ban provisioning. - Other privileged commands run via
sudo -Swith the SSH password fed on stdin. Never write that password into a file (the v0 provisioner did and broke sudo on the box — see Runbook → Bad IP ban recovery and theclockwork:clean-failed-provisionrecovery flow).
This section (and Integrations → SSH + fail2ban) is SpinupWP/Cloudways-scoped — those go through servers.ssh_private_key/ssh_password and App\Services\Ssh\SshClient. WP Engine and Kinsta use a separate, parallel SSH credential store: modules/Core/src/Support/SshConnector.php connects directly via phpseclib3\Net\SSH2 (not SshClient/the servers table) using per-environment env vars — CLOCKWORK_WPENGINE_SSH_PRIVATE_KEY, CLOCKWORK_KINSTA_SSH_PASSWORD (see the outbound-token table above) — solely to install the Companion mu-plugin on those hosts. Same sensitivity class as the encrypted servers columns, same physical/network-security stakes above apply to those env vars too, just a different storage location — don’t assume “it’s not in the servers table” means “not SSH.”
Is SSH avoidable for any of this? Not really, and not just for one feature. SSH is the substrate for a dozen unrelated things on an SSH-capable provider — apt-get updates, reboot scheduling, fail2ban bans, wp-cli plugin/checksum checks, and nginx log tailing for the Traffic feature (see Features → Traffic + capacity) all go through the same credential. Even where an alternative exists for one narrow piece — Pressable proves traffic data can be sourced over plain HTTPS instead of a log tail, see that doc — the same server still needs the SSH credential stored for everything else on this list. Isolating “just the traffic feature” behind its own SSH toggle wouldn’t shrink the fleet’s actual credential-storage risk at all, since updates/fail2ban/plugin-detection would still need the exact same key on the exact same server. The credential is fleet-wide the moment you want any one of these; there’s no smaller boundary to draw.
App database backups (/settings/maintenance)
MaintenanceController::index() shows the operator a one-glance summary of Clockwork Control’s own database (name, host, driver, and a rough total-byte size via a single information_schema.tables query) before they commit to downloading anything. downloadBackup() then streams a gzipped mysqldump (--single-transaction --quick --no-tablespaces, piped through gzip) straight to the browser via passthru() — no temp file ever touches the server’s disk, and the password goes through the MYSQL_PWD env var rather than the command line so it never shows up in ps output.
This is the highest-value single file an attacker could get: it contains every encrypted column’s ciphertext (SSH private keys, per-site DB passwords, Companion secrets) plus all snapshot/threat-log data. The ciphertext alone is safe — decrypting it requires pairing the dump with APP_KEY from .env, which never leaves the server through this path — but treat any downloaded copy of this file with the same handling discipline as .env itself. Downloads are limited to admin-role operators and recorded as TYPE_BACKUP_DOWNLOADED in action_logs.
Don’t do these
- Do not
php artisan serve --host 0.0.0.0. - Do not log decrypted credentials. The
ActionLoggeris generic JSON — sanitize before passing details. - Do not generate a single Cloudflare token with both Read and Edit scopes. Keep them separate.
- Do not run
php artisan migrate:freshagainst a real DB. Confirm before using. - Do not commit
.env. The.gitignorecovers it; don’t override. - Do not push a long-lived Companion secret through Pressable’s command API — it’s logged verbatim for ~30 days. Use the bootstrap-then-rotate pattern (see above).
- Do not flash
companion_secretor a Connection Key into session old-input, Blade, or Alpine state. - Do not mint a Glacier download URL or stage a restore for an S3 key that is not that site’s own prefix.
