Files
accounted/.github/workflows/test-pg-real.yml
T
Jakob Wennberg 16fbcefbbc feat(invariants): shared format contracts + upgrade-path CI (#1364)
* feat(invariants): centralise shared format contracts, reconcile the org-number paths

The same format rules were written out independently across the codebase, and
where they disagreed the disagreement was invisible until a filing failed.

Worst case, now fixed: four Skatteverket- and Bolagsverket-bound export paths
each had their own idea of a valid organisationsnummer.

  lib/skatteverket/format.ts      strip '-' only      threw on any input with a space
  lib/salary/ku/ku10-generator.ts replace('-', '')    first hyphen only, spaces survived
  lib/salary/agi/xml-generator.ts strip non-digits    stray letters passed the length check
  lib/bokslut/ixbrl/validate      /^\d{6}-?\d{4}$/    rejected the 12-digit form, no Luhn

A company stored with a space or in 12-digit form could file AGI all year and
then fail at the arsredovisning deadline with a message that did not say why.

lib/invariants/ now owns account number, ISO date, four-digit fiscal year and
org number, each with the rationale recorded next to the rule. normalizeOrgNumber
moves here from lib/company-lookup/ and isSaneDateString from lib/utils.ts; both
old paths re-export, so no caller changes. lib/api/schemas.ts builds its
primitives on the module, so ~100 schemas inherit any correction.

The arsredovisning check-digit verdict is a warn, not an error: a wrong Luhn
digit is almost certainly a typo worth surfacing, but whether every org number
Bolagsverket accepts satisfies Luhn is a Swedish domain question we have not
verified against a primary source, and an error there blocks Skicka in. We do
not block a statutory filing on an unverified assumption.

KU10 still passes a 12-digit stored org number through unfolded. That is
pre-existing, and whether the KU10 schema wants 10 or 12 digits is not covered
by the swedish-payroll skill, so it is pinned by a test rather than changed
silently.

Guard 8 (hand-rolled-invariant) tracks the remaining 114 inline copies as a
ratchet that may only go down, same mechanism as the roundOre guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(ci): add an upgrade-path job that applies new migrations against real data

The pg-real job applies all 548 migrations to an EMPTY database. Empty means
zero rows, so a migration that adds a NOT NULL, adds a CHECK, creates a unique
index or backfills passes trivially in CI and can still fail on production,
where the rows exist. CI proved that a fresh install works; nothing proved that
an existing install upgrades.

The new pg-upgrade job: apply the schema as it stands at the merge base, seed a
small real company (three posted verifikat, balanced lines, one ore-level
amount), then apply ONLY the migrations this PR adds, then assert the data
survived (entries still posted, lines intact, ledger still balances, ore
unchanged, voucher numbers sequential). A PR with no migration no-ops.

Verified locally against supabase/postgres:15.8.1.060 rather than assumed, with
three deliberately bad migrations:

  rescale money on posted lines   empty: would pass   seeded: ERROR (immutability trigger)
  CHECK violating the ore row     empty: exit 0       seeded: exit 3
  NOT NULL on a populated column  empty: exit 0       seeded: exit 3

Base migrations are read out of the merge-base git tree, not the working tree,
so a PR that edits an already-shipped migration still gets the original applied
and the edit surfaces as a failure here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: record the invariants and upgrade-CI decisions

Two entries covering what this PR changes and, more importantly, the calls that
are not obvious from the diff: why the arsredovisning check-digit verdict is a
warning rather than an error, why KU10's 12-digit passthrough is pinned instead
of fixed, and why the ROT/RUT brf org-number schemas stay on their own rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(test): mark the upgrade fixture as CI-only, never a production template

The fixture writes posted journal_entries and their lines directly, bypassing
the engine and the atomic commit RPC. That is the only way to hand a migration
pre-existing posted rows to break, and it is safe against a throwaway CI
database, but it reads like a sanctioned pattern to anyone who finds it later.

Says so explicitly, with the reason it is confined here (no voucher sequence to
keep gapless, no retention obligation on a database destroyed with the job) and
a pointer back to Hard Rule 2 for anything touching a real database.

Raised by the Swedish compliance review bot on #1364.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 16:30:14 +02:00

203 lines
7.7 KiB
YAML

name: pg-real tests
on: [pull_request]
# Neither job writes anything back: they read the repo, stand up a throwaway
# Postgres, and run tests. Without this block both inherit the repository's
# default token permissions, which are broader than that.
permissions:
contents: read
concurrency:
group: pg-real-${{ github.ref }}
cancel-in-progress: true
jobs:
coverage-gate:
# Enforces the database.md rule: a migration touching a trigger/RPC/RLS/
# DEFERRABLE must come with a *.pg.test.ts change. Previously instruction-
# only. Escape hatch: `-- pg-test: covered-by <path>` / `-- pg-test: skip
# (<reason>)` comments inside the migration.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
# Full history so the merge-base with the PR base branch exists.
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
with:
node-version: 20
- name: Require pg-real coverage for trigger/RPC/RLS migrations
env:
PG_GATE_BASE: origin/${{ github.base_ref }}
run: node scripts/check-pg-test-coverage.mjs
# Proves that an EXISTING database survives the PR's migrations, which the
# pg-real job below cannot: it applies every migration to an empty database,
# so a NOT NULL, a CHECK, a unique index or a backfill passes trivially
# against zero rows and can still fail (or silently corrupt) on production.
#
# Shape: schema at the merge-base -> seed real rows -> apply ONLY the new
# migrations -> assert the rows are intact. A PR that adds no migration skips
# straight past the apply step and costs one cheap no-op run.
pg-upgrade:
runs-on: ubuntu-latest
services:
postgres:
image: supabase/postgres:15.8.1.060
env:
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
PGPASSWORD: postgres
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
# Full history so the merge-base with the PR base branch exists.
fetch-depth: 0
persist-credentials: false
- name: Install psql client
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
- name: Identify the migrations this PR adds
id: newmig
env:
BASE_REF: origin/${{ github.base_ref }}
run: |
set -euo pipefail
MERGE_BASE=$(git merge-base "$BASE_REF" HEAD)
echo "merge_base=$MERGE_BASE" >> "$GITHUB_OUTPUT"
echo "Merge base: $MERGE_BASE"
# Migrations present at HEAD but not at the merge base. Uses the git
# tree, not the filesystem, so a rebase or a merge commit cannot make
# an already-shipped migration look new.
git ls-tree -r --name-only HEAD -- supabase/migrations \
| grep '\.sql$' | sort > /tmp/head-migrations.txt
git ls-tree -r --name-only "$MERGE_BASE" -- supabase/migrations \
| grep '\.sql$' | sort > /tmp/base-migrations.txt
comm -23 /tmp/head-migrations.txt /tmp/base-migrations.txt > /tmp/new-migrations.txt
COUNT=$(wc -l < /tmp/new-migrations.txt | tr -d ' ')
echo "count=$COUNT" >> "$GITHUB_OUTPUT"
echo "New migrations ($COUNT):"
cat /tmp/new-migrations.txt
- name: Bootstrap storage schema
if: steps.newmig.outputs.count != '0'
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
- name: Apply the schema as it stands at the merge base
if: steps.newmig.outputs.count != '0'
env:
MERGE_BASE: ${{ steps.newmig.outputs.merge_base }}
run: |
set -euo pipefail
# Read each migration out of the merge-base tree rather than the
# working tree: a PR that EDITS a shipped migration (forbidden, but
# this job must not be the thing that hides it) still gets the
# original applied here, so the edit shows up as a failure below.
while read -r f; do
echo "Applying (base) $f"
git show "$MERGE_BASE:$f" | psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q
done < /tmp/base-migrations.txt
- name: Seed a real company with posted verifikat
if: steps.newmig.outputs.count != '0'
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/upgrade/seed.sql
- name: Apply ONLY the new migrations, against existing data
if: steps.newmig.outputs.count != '0'
run: |
set -euo pipefail
while read -r f; do
echo "Applying (new) $f"
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
done < /tmp/new-migrations.txt
- name: Assert the existing data survived
if: steps.newmig.outputs.count != '0'
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f tests/pg/upgrade/assert.sql
- name: No migrations in this PR
if: steps.newmig.outputs.count == '0'
run: echo "This PR adds no migration; nothing to upgrade-test."
pg-real:
runs-on: ubuntu-latest
services:
postgres:
# Supabase image ships the auth schema, auth.uid(), and the extensions
# (uuid-ossp, pg_cron, btree_gist, vector) this repo's migrations need.
# Plain postgres:15 would require manual bootstrap SQL.
image: supabase/postgres:15.8.1.060
env:
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
PGPASSWORD: postgres
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
# Deliberately no `cache: npm` here. This workflow runs on pull_request
# only, so the cache is never written on main, and GitHub scopes caches
# by ref: every PR uploaded its own 284 MB copy under an identical key
# that no other PR could ever restore. Fourteen dead copies (4 GB, 40% of
# the repo quota) accumulated in a single day. If this is ever worth
# caching again, it has to be actions/cache/save on main plus
# actions/cache/restore here, which is the only shape that gets hits.
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
with:
node-version: 20
- run: npm ci
- name: Install psql client
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
- name: Bootstrap storage schema
# The supabase/postgres image ships a partial storage schema; the rest
# is provisioned by the storage-api service at runtime, which we do
# not run in CI. This aligns the schema with what migrations expect.
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
- name: Apply migrations
run: |
set -euo pipefail
shopt -s nullglob
files=(supabase/migrations/*.sql)
if [ ${#files[@]} -eq 0 ]; then
echo "No migration files found"
exit 1
fi
for f in "${files[@]}"; do
echo "Applying $f"
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
done
- run: npm run test:pg