Scheduled vs PR: which cadence for which contract
ProdVerdict runs seven contract types. They are not all the same shape. Some belong on every pull request. Some belong on a schedule. Running them on the wrong cadence wastes CI minutes and misses the bugs that matter.
The short version
| Contract | Cadence | Why |
|---|---|---|
| Access | Scheduled (hourly or daily) | Compares live Stripe vs live DB. Drift only exists after the webhook fires in production. |
| Entitlements migration | On demand (during migration) | Verifies DB flags match Stripe Entitlements grants before cutover. |
| Config | PR | Catches missing env vars in the diff. Belongs before merge. |
| Migration | PR | Catches unsafe DDL in the migration file you're about to ship. |
| Boundary | PR | Catches mass-assignment in the route handler you just added. |
| Webhook | PR | Catches missing signature verification in the handler you just wrote. |
| Restore | Scheduled (daily or weekly) | Verifies backups actually restore. |
Why Access is scheduled, not a PR gate
The Access Contract reads live state from two places: your billing system (Stripe or Paddle) and your app database. It compares them and reports disagreement.
That comparison is only meaningful after the webhook has fired in production. Consider the failure modes:
- A PR refactors
invoice.payment_failedand accidentally drops thehas_paid_access = falseupdate. Tests pass because Stripe is mocked. The PR merges. The drift doesn't exist yet — the customer's subscription is still active. The bug only manifests weeks later when the customer's card fails and the webhook fires with the broken handler. - A PR adds a new price ID without updating the
plans:map. Tests pass. The PR merges. The drift doesn't exist yet — no customer is on the new price. The bug only manifests when a customer subscribes to the new plan.
A PR-time Access check would pass both of these PRs. The bug surfaces in production, days or weeks later. The check that catches it is a scheduled one that runs against live state.
Why the lint contracts are PR gates, not scheduled
Config, migration, boundary, and webhook contracts inspect files in your repository. They catch issues in the diff you're about to merge. Running them on a schedule re-checks the same files that haven't changed. Running them on every PR catches the issue before it lands.
Restore is the exception — it verifies that backups actually restore, which is a property of your backup pipeline, not your code. Run it daily or weekly.
Recommended setup
Two workflows:
# 1. Scheduled drift detection (Access)
npx prodverdict scheduled --frequency hourly --install
# writes .github/workflows/prodverdict-hourly.yml
# 2. PR lint checks (Config + Migration + Boundary + Webhook)
# → see examples/workflows/prodverdict-pr-config.yml in the public SDK
Or use the interactive wizard to install both:
npx prodverdict setup
See GitHub Action for full workflow YAML.
What about contract: all on a schedule?
You can run contract: all on a schedule. It works. But it re-runs the file-scanning contracts (config, migration, boundary, webhook) against unchanged files. Harmless, but wasteful. The split above is the clean shape.
What about contract: all on every PR?
You can. But Access on a PR is mostly noise — it'll pass on PRs that don't touch billing, and pass on PRs that do touch billing (because the webhook hasn't fired yet). Skip Access on PRs. Run it on a schedule.