CI / CD

GitHub Actions workflows live in .github/workflows/. This document explains what each one does, when it runs, where it runs, and what to do when it’s red.

main is prod: Railway (API) and Vercel (web) deploy on every push to main. There is no pre-deploy gate to hold — the PR checks are the gate, and the full gate is a periodic alarm behind them.

Workflows

“CI_RUNS_ON” means runs-on: ${{ vars.CI_RUNS_ON || 'ubuntu-latest' }}: the job lands on the self-hosted Hetzner box while the repo variable is set to taikan-ci, and falls back to GitHub-hosted minutes when it is blank. Steps that differ between the two (pnpm cache, Playwright browsers) branch on runner.environment.

PR workflow (ci-smoke-tests.yml)

Runs on every PR. Job graph:

The box (CI_RUNS_ON) has one runner, so every job sent to it runs after the previous one finishes and a full PR’s wall time is their sum. Only changes is GitHub-hosted (10 s, and every other job waits on it). Hosted minutes are not the way out: at 6–8 PR pushes a day (peaks of 17), the old 45-minute run would have spent the 3,000 included minutes in under a week, which is why the box exists. The lever is what the suites cost, not where they run.

• changes (~10 s, hosted) — dorny/paths-filter decides which suites the PR affects: apps/api/** → api, apps/web/** → web, libs/** or root config → both.
• api (~6 min) — Postgres (pgvector/pg16) + Redis service containers on dynamic host ports (jobs share one box, so fixed 55432/6379 would collide); the “Export service URLs” step writes DATABASE_URL/REDIS_URL to $GITHUB_ENV. Migrates, then unit, integration and e2e as three steps so the summary names the suite that failed. Skipped for web-only PRs.
• typecheck (~2 min) — pnpm exec nx run-many -t typecheck --projects=@taikan/api,@taikan/web plus the brand-asset drift check.
• web (~12 min) — unit → integration. No service containers (the web suites mock the API). Skipped for api-only PRs.
• e2e (~15 min) — Playwright browser suite. needs: [changes, typecheck, api, web] and runs when typecheck passed and api/web each either passed or were skipped — a type error or failing unit test never spends a Playwright run. apps/web/.next/cache is kept under the runner user’s ~/.cache/taikan/next-cache on the box (linked in, since checkout cleans the workspace) so a production build (E2E_WEB_MODE=prod) is incremental; on a hosted runner the Actions cache plays the same role.

Expect ~35 min for a full run on the box, against ~45 before the API unit change below; the web suites are the next target. The next real step down is a second runner — a second CX23 (see “Self-hosted runner”): typecheck and web would then overlap api, the 2-hourly gate would stop holding PRs back, and Playwright would stop competing with the API server for cores.

CI_NODE_RUNS_ON (repo variable, unset by default) is the emergency valve: set it to ubuntu-latest and typecheck + web run hosted, in parallel with the box, at ~14 hosted minutes per full run. Use it when the box is down or wedged and a review is waiting; unset it after — at this push rate it would spend the month’s 3,000 minutes in about a week if left on.

No job passes --coverage. Coverage is a ratchet in the full gate (below), so PR runs are fast and never flake on thresholds.

Failures annotate the PR: vitest adds its github-actions reporter whenever GITHUB_ACTIONS is set, and apps/web/e2e/playwright.config.ts adds Playwright’s github reporter under CI, so a failing assertion shows on the Files tab at its line. The Playwright HTML report and traces upload as the playwright-report-pr artifact on failure.

Duplicate runs for the same ref are cancelled via concurrency.cancel-in-progress. Budgets (timeout-minutes): changes 5, typecheck 10, api 28, web 25, e2e 30. They are sized for a 2-vCPU runner with margin — a timeout should mean “hung”, never “slow runner”: a job cancelled mid-run on the box can also wedge the runner’s session (see “jobs queued, runner Idle” below), which costs far more than the minutes a tight budget saves.

Where the suite time goes

Measured locally on 4 cores with vitest’s phase breakdown, before the pool changes described in testing/strategy.md:

“Import” is summed across workers. With the default forks pool and per-file isolation, every spec file re-imports its whole dependency graph from disk (drizzle + schema + NestJS on the API side, react-dom + radix + jsdom on the web side), and on the API unit suite that is 70× the time the tests themselves take. The API unit suite now shares one registry per worker (specs that call vi.mock stay isolated): 4.5 min → 47 s on the box. The same idea was tried on the web suites (vmThreads) and API e2e (isolate: false), passed locally and failed on the box — see the strategy doc for what leaked — so those keep the default pool.

When it’s red

• typecheck: pnpm exec nx run-many -t typecheck --projects=@taikan/api,@taikan/web locally. If the brand step fails, pnpm brand:assets and commit the result.
• api: the failing step names the suite; rerun it locally with the same env (make test-unit-api, make test-integration-api, make test-e2e-api). Most flakes are timing-related — see testing/strategy.md. An API unit spec that passes alone and fails in the suite (or the reverse) after adding a vi.mock, a module-level singleton or a process.env write is a pool-isolation question: see testing/strategy.md.
• web: make test-unit-web, make test-integration-web.
• e2e: make test-e2e-web (E2E_WEB_MODE=prod make test-e2e-web to serve a production build instead of next dev; see testing/strategy.md). Traces, screenshots and video are uploaded as the playwright-report-pr artifact on failure.
• Migration step: the journal-monotonicity bug (migrations.md) sometimes appears as “migration applied successfully” but a follow-up assertion fails because the schema didn’t change. Check libs/db/drizzle/meta/_journal.json for any out-of-order when values.
• Auth bypass surprises: TEST_AUTH_BYPASS=true lets x-test-user-id header replace Clerk. If a test fails because the user isn’t found, the seed step probably didn’t run — check the testApi.seedX() calls in the spec.
• Export service URLs / connection refused: the service container’s health check passed but the mapped port is wrong — check the job’s “Initialize containers” log for the 5432/tcp -> 0.0.0.0:NNNNN line. DATABASE_URL must never be set at job-level env (the job.services context isn’t available there).

Full gate (scheduled + on demand, cd-full-test-gate.yml)

Runs every 2 hours and on manual dispatch (Actions → Main Full Gate → Run workflow). One job, full-test-gate, on the self-hosted box:

1. Migrate, then one step per suite: API unit (pnpm test:coverage:api), API integration, API e2e, web unit + integration (pnpm test:coverage:web), Playwright. The suites with a coverage floor run with --coverage in that single pass and fail the step on an undercut coverage.thresholds (pinned in apps/api/vitest.unit.config.ts, apps/web/vitest.unit.config.ts, apps/web/vitest.integration.config.ts) as well as on a failing test; the log line says which.
2. Every step runs even after an earlier one fails (if: ${{ !cancelled() }}), so one red gate lists everything that is broken on main.

Budget: 65 minutes (expect ~20-30 on the box now that each suite runs once). Coverage reports are uploaded as api-coverage-full / web-coverage-full artifacts; the Playwright report as playwright-report-full on failure. concurrency is cancel-in-progress, so a 2-hourly tick that lands while a gate is still running replaces it instead of queueing behind it. Know the cost: PR checks cannot preempt a running job, so a PR pushed while the gate runs waits up to the gate’s length on a single-runner box — a second runner (see below) is what removes that wait.

When it’s red

A red gate means main — which is already deployed — is broken or lowered coverage. It is not tied to one merge, so check what landed since the last green gate. Read the failing step:

• A suite step red on a test → a real regression in prod. Same diagnostics as the PR jobs; fix forward or revert the offending merge. For an immediate re-check of a candidate fix, ./scripts/ci-local.sh --with-e2e runs the same suites in ~3 min.
• A “(coverage ratchet)” step red on does not meet global threshold → the merge dropped a metric below its floor. Either add tests to bring it back or, deliberately, lower the floor in the vitest config in the same PR that explains why. The floors are ratchets: when coverage rises, raise them (round down to whole percents).

Self-hosted runner (Hetzner)

Everything under CI_RUNS_ON in the table above executes on one Hetzner box. Minutes on it are free; only the ubuntu-latest workflows spend the GitHub quota.

Files: scripts/ci/hetzner-bootstrap.sh (idempotent box setup), scripts/ci/actions-runner@.service (systemd unit template, one instance per runner).

Box layout:

User runner (system user, home /home/runner, in docker group) owns all of it. Node 22 via NodeSource, corepack pnpm pinned to root package.json’s packageManager, Docker CE for the service containers and the semgrep container job.

Setup

1. Hetzner Cloud → New server: image Ubuntu 24.04, type CX33 (shared Intel, 4 vCPU / 8 GB, ~€8.49/mo + VAT + ~€0.60 IPv4 as of the June 2026 repricing; CX32 is the deprecated predecessor) in Falkenstein, Nuremberg or Helsinki — the CX line is only sold there; US locations only offer CPX/CCX at 3–5× the price. api + web + typecheck run concurrently on two runners; if Playwright swaps, step up to CX43. Add your SSH key. Firewall: inbound allow only TCP 22 (from your IP if you have a static one), outbound allow all. No public services run on this box.

   Greyed-out types/locations mean sold out, not disallowed. CX plans go out of stock regularly. If only a CX23 (2 vCPU / 4 GB, ~€4–5) is available: take it, run the bootstrap with RUNNER_COUNT=1, and when CX33 is back in stock use Server → Rescale → “CPU and RAM only” (one click, reversible, no rebuild), then re-run the bootstrap with RUNNER_COUNT=2. Location doesn’t matter for CI. The bootstrap adds a 4 GB swapfile on every size.

   A second CX23 beats waiting for a CX33. The runner needs two vCPU per job, not four per box, so two 2-vCPU servers give the same two runners as one 4-vCPU server — and a CX23 is what is usually in stock, in any location (pick whichever has it; the boxes never talk to each other). Create it, run the same bootstrap with RUNNER_COUNT=1; the runner name is hetzner-<hostname>-1, so it registers alongside the first under the same taikan-ci label and GitHub spreads jobs across both. Each box keeps its own pnpm store, browser cache and ~/.cache/taikan. Leave /etc/taikan/prod-smoke.env off the second box so the prod smoke timer runs once. Do not put two runners on one CX23 (see below) — that is what made two boxes the answer.

2. ssh root@<IP>.

3. The repo is private, so curl raw.githubusercontent.com/... for the script 404s. Copy it from your checkout instead:

   scp scripts/ci/hetzner-bootstrap.sh scripts/ci/actions-runner@.service root@<IP>:/root/

   (both files must sit in the same directory; the script installs the unit from its own directory.)

4. Registration token: repo Settings → Actions → Runners → New self-hosted runner → Linux, copy the value after --token in the “Configure” snippet. It is a one-hour registration token, not a PAT.

5. Clone access for /opt/taikan: either an https URL with a fine-grained PAT (Contents: read) embedded, or a read-only deploy key (Settings → Deploy keys) — the script prints the exact ssh-keygen steps if the clone fails.

6. Run it:

   GH_RUNNER_TOKEN=<token> \
   GIT_CLONE_URL='https://x-access-token:<pat>@github.com/desmotech/taikan' \
   bash /root/hetzner-bootstrap.sh

   Knobs (env): RUNNER_COUNT (2; 1 on a CX23), SWAP_GB (4), RUNNER_VERSION (latest release), RUNNER_EPHEMERAL (0), PNPM_VERSION, PLAYWRIGHT_VERSION (from pnpm-lock.yaml). Re-running is safe: configured runners, existing caches and the checkout are skipped/updated, not rebuilt.

   One runner per two vCPU, measured. Two concurrent test jobs on the 2-vCPU CX23 put the box at load 4.0+ (vitest sizes its worker pool from nproc, so each job alone already saturates both cores). Memory was fine — 1.3 GB of 3.7 GB, 89 MB swap — but the contention stretches timing-sensitive specs past their timeouts, which buys throughput with flakes. Deeper queues are the lesser evil: keep RUNNER_COUNT=1 until the box has 4 vCPU. To take an extra instance out without uninstalling it: systemctl disable --now actions-runner@2 (see the drain note in “Operating it”).

7. Verify: Settings → Actions → Runners shows hetzner-<host>-1 (and -2 with RUNNER_COUNT=2) as Idle with label taikan-ci. On the box: systemctl status 'actions-runner@*'.

8. Flip CI over: Settings → Secrets and variables → Actions → Variables → New repository variable CI_RUNS_ON = taikan-ci.

9. Open a trivial PR (touch a comment under apps/web). Every job in “CI Smoke Tests” should show Runner name: hetzner-… at the top of its “Set up job” log, and pnpm install should take seconds, not minutes.

10. Rollback: delete (or blank) the CI_RUNS_ON variable. Every job falls back to ubuntu-latest on the next run; nothing else changes. Do this before taking the box down for maintenance, otherwise PR checks queue forever.

Persistent vs ephemeral runners

The default is persistent (RUNNER_EPHEMERAL=0): each runner registers once and serves jobs forever; Restart=always in the unit only covers crashes and the runner’s self-update (which exits run.sh). Job workspaces under /opt/actions-runner-N/_work are reused between jobs — actions/checkout cleans the repo dir, pnpm/Playwright are fine with it, and _work growth is bounded by the number of workflows. No credentials live on the box.

Ephemeral (RUNNER_EPHEMERAL=1): the runner de-registers after exactly one job and run.sh exits, so the unit restarts and reconfigure.sh (ExecStartPre) must register it again with a fresh registration token. That needs a fine-grained PAT with repository permission Administration: Read and write (the scope for POST /repos/{owner}/{repo}/actions/runners/registration-token) stored at /etc/taikan/runner-pat:

install -o runner -g runner -m 600 /dev/stdin /etc/taikan/runner-pat <<<'github_pat_…'

Pick ephemeral only if you want a clean workspace per job badly enough to keep an admin-scoped PAT on the box. Switching modes: RUNNER_EPHEMERAL=1 GH_RUNNER_TOKEN=… bash hetzner-bootstrap.sh after rm /opt/actions-runner-*/.runner (the script re-registers whatever is unregistered and rewrites reconfigure.sh).

Operating it

journalctl -u 'actions-runner@*' -f     # live job log lines
systemctl restart actions-runner@1      # after a runner self-update wedges
docker ps                               # leftover service containers = a job that was killed mid-run
docker system prune -af --filter until=168h
du -sh /var/cache/pnpm-store /var/cache/ms-playwright /opt/actions-runner-*/_work

Symptom: jobs queued, runner “Idle”, nothing dispatches. This is the single most disruptive failure this box has; it cost ~80 min of dead queue on 2026-09-04 alone. Settings → Actions → Runners shows the box online, busy=false with the matching taikan-ci label, journalctl shows Listening for Jobs and no errors, nothing in the repo is in_progress, and the queued jobs sit unassigned (runner_id=0, runner_name=""). Both ends look healthy and no work moves.

Cause: the listener’s server-side session still holds a job assignment from a session that died while a job was assigned to it. Two ways to get there, both routine here — a job cancelled mid-run (GitHub cancels a PR’s running jobs when the head branch is deleted on merge), and systemctl restart landing in the same second the listener accepts a job (runsvc.sh escalates SIGINT to SIGKILL immediately, so the assignment is never released).

A restart does not fix it. Observed: restarted 17:46, Listening for Jobs at 17:46:09, then 44 minutes idle with four runs queued. Re-registering took a job in 3 seconds. Escalate in this order:

# 1. Confirm it is actually wedged, not just serialized behind a long job.
#    busy=true, or any run in_progress -> the box is working; a restart would
#    kill a live job. Do nothing.
gh api /repos/desmotech/taikan/actions/runners --jq '.runners[]|"\(.name) busy=\(.busy)"'
gh api "/repos/desmotech/taikan/actions/runs?status=in_progress" --jq '.workflow_runs|length'
 
# 2. Cheap attempt: restart, then wait 60s and re-check for dispatch.
systemctl restart actions-runner@1
 
# 3. The actual cure - re-register, which clears the stale session. Mint both
#    tokens from a machine with `gh` (they are short-lived; nothing is stored
#    on the box).
RM=$(gh api -X POST /repos/desmotech/taikan/actions/runners/remove-token --jq .token)
REG=$(gh api -X POST /repos/desmotech/taikan/actions/runners/registration-token --jq .token)
ssh taikan-ci "systemctl stop actions-runner@1; cd /opt/actions-runner-1; \
  su runner -c './config.sh remove --token $RM'; \
  su runner -c './config.sh --unattended --replace \
    --url https://github.com/desmotech/taikan --token $REG \
    --name hetzner-ubuntu-4gb-hel1-3-1 --labels taikan-ci --work _work'; \
  systemctl start actions-runner@1"

To make step 3 unnecessary, put a fine-grained PAT (repo desmotech/taikan, Administration: read and write) at /etc/taikan/runner-pat, chmod 600. reconfigure.sh then mints its own registration token, so dropping .runner and restarting re-registers unattended - and RUNNER_EPHEMERAL=1 becomes usable, which sidesteps this class of wedge entirely by giving every job a fresh session. Without that file reconfigure.sh exits 1 by design.

Runs whose head branch was deleted keep their remaining jobs in queued forever - they never dispatch and they are not what is blocking you, but they make the queue unreadable. Cancel them: gh api -X POST /repos/desmotech/taikan/actions/runs/<id>/cancel.

Stopping a busy runner drains it. systemctl stop sends SIGTERM, which runsvc.sh turns into the runner’s “finish the current job, then exit” — so the command blocks for as long as the running job needs (TimeoutStopSec is 30 min, above the 28-min api budget). That is correct, not a hang: use --no-block if you don’t want to wait, and confirm afterwards that nothing survived:

systemctl stop --no-block actions-runner@1
pgrep -af 'Runner.Listener|Runner.Worker'   # must be empty for that instance

If a listener or worker is still there with the unit inactive, it is orphaned — it keeps claiming and running jobs with nothing supervising it, and systemctl is-active cannot see it. Kill it by pid. (This is what ExecStart=run.sh + KillMode=process used to cause; the unit now uses bin/runsvc.sh + KillMode=mixed so the cgroup is always cleaned.)

• Upgrading the runner: actions/runner self-updates in place; if GitHub starts refusing an old version, re-run bootstrap with RUNNER_VERSION= unset — it only downloads for directories missing config.sh, so rm -rf /opt/actions-runner-N first (and re-register with a new token).
• Node / pnpm bumps: pnpm follows root package.json packageManager via corepack at bootstrap; re-run bootstrap with the new PNPM_VERSION after bumping it. setup-node in the workflows still selects Node 22 per job.
• Playwright bumps: the job step playwright install chromium downloads the new build into /var/cache/ms-playwright on first use; OS deps were installed once by bootstrap (playwright install-deps chromium) — re-run bootstrap if a new Playwright major needs new libs.
• Disk: the weekly prune handles Docker. pnpm store and browser cache are shared and small; _work holds one checkout per workflow per runner.
• NEVER add a container: job. Container jobs run as uid 0 and bind-mount _work, so they leave root-owned files behind; the next job runs as runner and dies in actions/checkout with could not delete reference refs/heads/main: … packed-refs.lock: Permission denied / EACCES: permission denied, rmdir …. Every job on the box is then red until the workspace is repaired. (This is why smegrep.yml installs semgrep with pip instead of using semgrep/semgrep as a job container.) Service containers under services: are fine — they never touch the workspace. Recovery:

  systemctl stop actions-runner@1
  chown -R runner:runner /opt/actions-runner-1/_work
  rm -rf /opt/actions-runner-1/_work/taikan /opt/actions-runner-1/_work/_temp/_github_home
  systemctl start actions-runner@1

• Security: the runner executes whatever a workflow says, as runner, with Docker access (effectively root). Acceptable for a private single-maintainer repo; never for a repo taking outside PRs. Keep the firewall at SSH-only.

Required status checks (GitHub Pro)

With GitHub Pro, rulesets can require checks on a private repo. Settings → Rules → Rulesets → New branch ruleset:

• Name main, enforcement Active, target branch: default branch.
• Restrict deletions, Block force pushes: on.
• Require a pull request before merging: on, 0 approvals (solo dev). Optionally “Allowed merge methods: squash”.
• Require status checks to pass: add typecheck, api, web, e2e (from “CI Smoke Tests (Web + API)”) and semgrep. Do not tick “Require branches to be up to date before merging” — with serial merges it only forces a rebase + full re-run per PR.
• Leave “Do not require status checks on creation” off.

api and web are path-conditional (if: needs.changes.outputs.api == 'true'). That is fine to require: a job whose if evaluates false still reports to the PR with conclusion skipped, and GitHub counts a skipped check as satisfying a required status check. What would block a merge is a check that never reports at all — which is why every job in ci-smoke-tests.yml is a real job (not a matrix entry) and why the workflow’s on.pull_request.paths filter stays equal to the union of the changes filters: a PR that touches none of those paths never triggers the workflow, and required checks that never run leave the PR unmergeable. If you ever add a path to changes but not to on.paths (or vice versa), PRs will hang on “Expected — Waiting for status to be reported”.

Checks are matched by job name, so renaming a job means editing the ruleset.

Prod smoke monitor

A read-only Playwright suite runs against production every 10 minutes from the same box (prod-smoke.timer → prod-smoke.service → scripts/prod-smoke/run.sh). It signs in as a synthetic user via a Clerk sign-in token, walks the main surfaces, and checks the API health endpoint. Files: apps/web/e2e/playwright.prod.config.ts, apps/web/e2e/prod.setup.ts, apps/web/e2e/prod/*.spec.ts; storage state apps/web/e2e/.auth/prod-state.json; pnpm script test:smoke:prod.

Setup:

1. Synthetic tenant in prod Clerk — never a customer org. Create an organization (e.g. “Taikan Smoke”) and a user in it; onboard the org through the app so it has a plan/schedule to look at. Note the user id (user_…). The user must belong only to this org: the smoke signs in as that user and everything it sees is that org’s data.
2. On the box, fill the env file from the example and lock it down:

   cp /opt/taikan/scripts/prod-smoke/prod-smoke.env.example /etc/taikan/prod-smoke.env
   chmod 600 /etc/taikan/prod-smoke.env
   $EDITOR /etc/taikan/prod-smoke.env   # SMOKE_CLERK_SECRET_KEY (prod sk_live_…), SMOKE_CLERK_USER_ID, SENTRY_AUTH_TOKEN

   SMOKE_BASE_URL / SMOKE_API_URL default to https://app.taikan.fit / https://api.taikan.fit.
3. Enable and watch:

   systemctl enable --now prod-smoke.timer
   systemctl list-timers prod-smoke.timer
   journalctl -u prod-smoke -f

   (hetzner-bootstrap.sh enables the timer itself when the env file already exists.) Run once by hand with systemctl start prod-smoke.service.
4. Sentry cron monitor — with SENTRY_AUTH_TOKEN set, run.sh wraps the suite in sentry-cli monitors run prod-smoke --schedule '*/10 * * * *' (org taikan, https://de.sentry.io). The monitor prod-smoke is auto-created on the first check-in (Sentry → Crons). Open it, set the failure/missed thresholds (1 missed, 2 consecutive failures is a sane start) and attach an alert rule that routes to wherever prod alerts go. Sentry’s token needs project:write on the project the monitor lands in.
5. Locally, against prod, for debugging: SMOKE_CLERK_SECRET_KEY=… SMOKE_CLERK_USER_ID=… pnpm test:smoke:prod (or SMOKE_BASE_URL pointed at a preview).

When it’s red: journalctl -u prod-smoke -n 200; the Playwright report is under /opt/taikan/apps/web/playwright-report. A failing smoke with green Railway/Vercel usually means Clerk (sign-in token minting) or a UI change that moved a test id — the specs use the same driver/data-testid rules as the e2e suite.

Minutes budget

Expected total: ~150–300 GitHub-hosted minutes/month, against the 3,000 included with Pro. For scale: the PR workflow ran ~45 min per push before the pool changes, at 6–8 pushes a day, so hosted it would have been ~10,000 minutes a month — that is the calculation that put the suites on the box, and it still holds.

Cap the blast radius anyway: Settings → Billing and plans → Spending limits → Actions → $10. If the box goes down with CI_RUNS_ON still set, its jobs queue rather than spend; if you blank the variable to fall back, this limit is the most a runaway month can cost.

PR preview

deploy-pr-preview.yml — builds a preview env for visual review. Not enforced, but useful for stakeholder sign-off before merge.

Required GitHub secrets

For ci-smoke-tests.yml and cd-full-test-gate.yml:

• CLERK_SECRET_KEY, CLERK_PUBLISHABLE_KEY — a Clerk test instance dedicated to CI.
• E2E_CLERK_USER_EMAIL, E2E_CLERK_USER_PASSWORD, E2E_CLERK_USER_ID — credentials for the Playwright sign-in flow (e2e job and the full gate).

Repository variable: CI_RUNS_ON (see above). Add secrets/variables via Repo Settings → Secrets and variables → Actions.

Prod smoke secrets live on the box in /etc/taikan/prod-smoke.env, not in GitHub.

Local “act-like-CI”

To reproduce the PR env locally:

make test-db-up
make test-db-migrate
make test-local-smoke   # unit + integration for API & web (no Playwright)
make test-local-all     # the above + Playwright web e2e
make test-coverage      # the post-merge coverage ratchet

When Actions is unavailable (quota exhausted, billing lapse, outage)

A run that “fails” in 2-4 seconds and whose jobs API 404s never executed — the workflow was rejected before any job was created. That is a billing/quota signal, not a code signal, and re-running it will not help. With the self-hosted runner in place this should only happen to the ubuntu-latest workflows; if the box is the problem, see “Operating it” above.

./scripts/ci-local.sh runs the same gate on your machine. It invokes the same pnpm scripts the workflows invoke, so a pass means what a green check means:

./scripts/ci-local.sh              # api/web suites + semgrep (+ api typecheck, build)
./scripts/ci-local.sh --with-e2e   # + Playwright, the slow `e2e` job
./scripts/ci-local.sh --quick      # skip install/migrate when already current

Every stage runs even after one fails, then a summary lists each stage with its duration — a red gate should tell you everything that is broken, not just the first thing. It refuses to start if DATABASE_URL points at taikan_dev, because the e2e reset truncates its target.

It loads .env.test and fills in the same defaults the workflow env: blocks set (encryption keys, TEST_AUTH_BYPASS, Morning/R2 dummies), so the suites see what they see in CI. A whole local gate is ~3 min on an M-series laptop against ~25 min on the box — when you need an answer now, run it here.

Stages that cannot run locally are reported as SKIP, never as pass, and the footer says the run is not equivalent to a green PR. Today that is build and web e2e when NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY is missing or still the .env.test.example placeholder: next build prerenders Clerk-wrapped routes and dies with “The publishableKey passed to Clerk is invalid”. Put your Clerk dev publishable key in .env.test to enable both.

What it does NOT cover:

• Linux. The runners are Ubuntu; you are on macOS/arm64. Native modules and case-sensitive imports can pass here and fail there.
• A clean checkout. It runs against your working tree, including untracked files. git stash -u first if you want the real thing.
• Managed Semgrep rules. Without SEMGREP_APP_TOKEN it falls back to semgrep scan --config=auto, the open ruleset only — the managed supply-chain policy that blocks on this repo is not evaluated.
• PR preview deploys and anything else keyed to pull_request events.

If you need the real Linux environment, act (brew install act) runs the workflow YAML in Docker. Slower, and secrets have to be wired by hand.

Adding a workflow

Pre-flight:

• runs-on: ${{ vars.CI_RUNS_ON || 'ubuntu-latest' }} for anything that is a test or check; keep deploys on ubuntu-latest (they need nothing from the box and shouldn’t wait behind test jobs). The box has one runner, so a new check there adds its whole duration to every PR — measure it before adding it.
• Service containers on the box must publish dynamic ports (ports: ['5432']) and export URLs in a step — see the api job.
• No container: jobs — they poison the self-hosted workspace with root-owned files (see “Operating it”). Install the tool in a step instead.
• Use concurrency.cancel-in-progress: true for everything that doesn’t deploy.
• Set timeout-minutes so a hung step doesn’t tie up a runner.
• Pin actions to a full commit SHA with the version in a trailing comment.
• If the workflow needs secrets, document them in this file. If it should gate merges, add its job name to the main ruleset.