In-flight charges

An in-flight charge is one where we called the payment provider and never learned the outcome. Money may or may not have moved, and the local record cannot tell you which.

This is the failure the 2026-08-10 outage would have produced had it landed on the 02:00 charge sweep: outbound HTTPS to Cardcom stayed up the whole time while Postgres was unreachable, so a charge could succeed milliseconds before the write recording it failed.

How the system behaves

A charge intent row is written before the provider is called:

While a row is in flight the subscription is frozen: the next sweep recomputes an identical idempotency key (neither the billing period nor the settled failure count moved), the partial unique index refuses the claim, and the sweep skips the subscription rather than charging it again.

The freeze is deliberate and is the safe failure. The alternative — assume it failed and retry — bills a customer twice. It is safe only for as long as it takes someone to settle the row, which is why it alerts.

The alert

alertInFlightCharges runs every 5 minutes and reports any row in flight for more than 15 minutes, to logs and to Sentry (kind: reconciler.in-flight):

Platform billing charge outcome UNKNOWN — transaction <id> (org <orgId>) called the provider and never recorded a result. Money may have moved. The subscription will NOT be charged again until this row is settled.

Nothing else will find these. The reconciler skips them (no provider transaction id to query with) and the stale-pending sweep explicitly refuses to touch them, precisely so they cannot be silently written off after 7 days.

Settling one

1. Find the row.

   SELECT id, organization_id, amount, idempotency_key, charge_started_at
   FROM platform_billing_transactions
   WHERE status = 'pending' AND charge_started_at IS NOT NULL
   ORDER BY charge_started_at;

   For member subscriptions the same query applies to payment_transactions.

2. Ask the provider what actually happened. This is the only source of truth — do not infer it from our own records. In the Cardcom dashboard, search transactions for the terminal around charge_started_at, matching on amount and the card token. Look for a transaction in a ±15 minute window.

3. Settle the row to match reality.

   If the provider DID charge — record it as completed so the customer is not billed again, and advance the period:

   UPDATE platform_billing_transactions
   SET status = 'completed',
       provider_transaction_id = '<id from provider>',
       charge_started_at = NULL,
       updated_at = now()
   WHERE id = '<txn id>';

   Then advance the subscription’s current_period_start / current_period_end and set last_charged_at, exactly as a successful sweep would have. Until you do, the subscription still looks due — but it stays frozen, because the key for that period is now permanently consumed.

   If the provider did NOT charge — record the failure so the key changes and tomorrow’s sweep retries normally:

   UPDATE platform_billing_transactions
   SET status = 'failed',
       error_message = 'settled manually: provider has no matching transaction',
       charge_started_at = NULL,
       updated_at = now()
   WHERE id = '<txn id>';
    
   UPDATE platform_billing_subscriptions
   SET failed_payment_count = failed_payment_count + 1,
       last_failed_at = now(),
       updated_at = now()
   WHERE id = '<subscription id>';

   Incrementing the failure count is what mints a new idempotency key. Skip it and the subscription stays frozen forever.

4. Confirm the alert clears on the next 5-minute tick.

What NOT to do

• Do not delete the row. That frees the key and the next sweep charges again — the exact double-charge this design exists to prevent.
• Do not clear charge_started_at without settling status. The row would stop alerting while still being unresolved, and the stale-pending sweep would then be free to cancel it unverified after 7 days.
• Do not guess. If the provider dashboard is ambiguous, leave it in flight and escalate. A frozen subscription costs a delayed renewal; a wrong guess costs a double charge or a free month.

Related

• docs/runbooks/api-web-scheduler-split.md — where the crons run
• docs/runbooks/incident-response.md — general incident flow
• apps/api/src/common/charge-idempotency.ts — why the key is built the way it is