feat(invariants): shared format contracts + upgrade-path CI (#1364)
* feat(invariants): centralise shared format contracts, reconcile the org-number paths
The same format rules were written out independently across the codebase, and
where they disagreed the disagreement was invisible until a filing failed.
Worst case, now fixed: four Skatteverket- and Bolagsverket-bound export paths
each had their own idea of a valid organisationsnummer.
lib/skatteverket/format.ts strip '-' only threw on any input with a space
lib/salary/ku/ku10-generator.ts replace('-', '') first hyphen only, spaces survived
lib/salary/agi/xml-generator.ts strip non-digits stray letters passed the length check
lib/bokslut/ixbrl/validate /^\d{6}-?\d{4}$/ rejected the 12-digit form, no Luhn
A company stored with a space or in 12-digit form could file AGI all year and
then fail at the arsredovisning deadline with a message that did not say why.
lib/invariants/ now owns account number, ISO date, four-digit fiscal year and
org number, each with the rationale recorded next to the rule. normalizeOrgNumber
moves here from lib/company-lookup/ and isSaneDateString from lib/utils.ts; both
old paths re-export, so no caller changes. lib/api/schemas.ts builds its
primitives on the module, so ~100 schemas inherit any correction.
The arsredovisning check-digit verdict is a warn, not an error: a wrong Luhn
digit is almost certainly a typo worth surfacing, but whether every org number
Bolagsverket accepts satisfies Luhn is a Swedish domain question we have not
verified against a primary source, and an error there blocks Skicka in. We do
not block a statutory filing on an unverified assumption.
KU10 still passes a 12-digit stored org number through unfolded. That is
pre-existing, and whether the KU10 schema wants 10 or 12 digits is not covered
by the swedish-payroll skill, so it is pinned by a test rather than changed
silently.
Guard 8 (hand-rolled-invariant) tracks the remaining 114 inline copies as a
ratchet that may only go down, same mechanism as the roundOre guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ci): add an upgrade-path job that applies new migrations against real data
The pg-real job applies all 548 migrations to an EMPTY database. Empty means
zero rows, so a migration that adds a NOT NULL, adds a CHECK, creates a unique
index or backfills passes trivially in CI and can still fail on production,
where the rows exist. CI proved that a fresh install works; nothing proved that
an existing install upgrades.
The new pg-upgrade job: apply the schema as it stands at the merge base, seed a
small real company (three posted verifikat, balanced lines, one ore-level
amount), then apply ONLY the migrations this PR adds, then assert the data
survived (entries still posted, lines intact, ledger still balances, ore
unchanged, voucher numbers sequential). A PR with no migration no-ops.
Verified locally against supabase/postgres:15.8.1.060 rather than assumed, with
three deliberately bad migrations:
rescale money on posted lines empty: would pass seeded: ERROR (immutability trigger)
CHECK violating the ore row empty: exit 0 seeded: exit 3
NOT NULL on a populated column empty: exit 0 seeded: exit 3
Base migrations are read out of the merge-base git tree, not the working tree,
so a PR that edits an already-shipped migration still gets the original applied
and the edit surfaces as a failure here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: record the invariants and upgrade-CI decisions
Two entries covering what this PR changes and, more importantly, the calls that
are not obvious from the diff: why the arsredovisning check-digit verdict is a
warning rather than an error, why KU10's 12-digit passthrough is pinned instead
of fixed, and why the ROT/RUT brf org-number schemas stay on their own rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(test): mark the upgrade fixture as CI-only, never a production template
The fixture writes posted journal_entries and their lines directly, bypassing
the engine and the atomic commit RPC. That is the only way to hand a migration
pre-existing posted rows to break, and it is safe against a throwaway CI
database, but it reads like a sanctioned pattern to anyone who finds it later.
Says so explicitly, with the reason it is confined here (no voucher sequence to
keep gapless, no retention obligation on a database destroyed with the job) and
a pointer back to Hard Rule 2 for anything touching a real database.
Raised by the Swedish compliance review bot on #1364.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
@@ -33,6 +33,108 @@ jobs:
|
||||
PG_GATE_BASE: origin/${{ github.base_ref }}
|
||||
run: node scripts/check-pg-test-coverage.mjs
|
||||
|
||||
# Proves that an EXISTING database survives the PR's migrations, which the
|
||||
# pg-real job below cannot: it applies every migration to an empty database,
|
||||
# so a NOT NULL, a CHECK, a unique index or a backfill passes trivially
|
||||
# against zero rows and can still fail (or silently corrupt) on production.
|
||||
#
|
||||
# Shape: schema at the merge-base -> seed real rows -> apply ONLY the new
|
||||
# migrations -> assert the rows are intact. A PR that adds no migration skips
|
||||
# straight past the apply step and costs one cheap no-op run.
|
||||
pg-upgrade:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: supabase/postgres:15.8.1.060
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd "pg_isready -U postgres"
|
||||
--health-interval 5s
|
||||
--health-timeout 5s
|
||||
--health-retries 20
|
||||
|
||||
env:
|
||||
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
|
||||
PGPASSWORD: postgres
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
# Full history so the merge-base with the PR base branch exists.
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install psql client
|
||||
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
|
||||
|
||||
- name: Identify the migrations this PR adds
|
||||
id: newmig
|
||||
env:
|
||||
BASE_REF: origin/${{ github.base_ref }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
MERGE_BASE=$(git merge-base "$BASE_REF" HEAD)
|
||||
echo "merge_base=$MERGE_BASE" >> "$GITHUB_OUTPUT"
|
||||
echo "Merge base: $MERGE_BASE"
|
||||
|
||||
# Migrations present at HEAD but not at the merge base. Uses the git
|
||||
# tree, not the filesystem, so a rebase or a merge commit cannot make
|
||||
# an already-shipped migration look new.
|
||||
git ls-tree -r --name-only HEAD -- supabase/migrations \
|
||||
| grep '\.sql$' | sort > /tmp/head-migrations.txt
|
||||
git ls-tree -r --name-only "$MERGE_BASE" -- supabase/migrations \
|
||||
| grep '\.sql$' | sort > /tmp/base-migrations.txt
|
||||
comm -23 /tmp/head-migrations.txt /tmp/base-migrations.txt > /tmp/new-migrations.txt
|
||||
|
||||
COUNT=$(wc -l < /tmp/new-migrations.txt | tr -d ' ')
|
||||
echo "count=$COUNT" >> "$GITHUB_OUTPUT"
|
||||
echo "New migrations ($COUNT):"
|
||||
cat /tmp/new-migrations.txt
|
||||
|
||||
- name: Bootstrap storage schema
|
||||
if: steps.newmig.outputs.count != '0'
|
||||
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
|
||||
|
||||
- name: Apply the schema as it stands at the merge base
|
||||
if: steps.newmig.outputs.count != '0'
|
||||
env:
|
||||
MERGE_BASE: ${{ steps.newmig.outputs.merge_base }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# Read each migration out of the merge-base tree rather than the
|
||||
# working tree: a PR that EDITS a shipped migration (forbidden, but
|
||||
# this job must not be the thing that hides it) still gets the
|
||||
# original applied here, so the edit shows up as a failure below.
|
||||
while read -r f; do
|
||||
echo "Applying (base) $f"
|
||||
git show "$MERGE_BASE:$f" | psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q
|
||||
done < /tmp/base-migrations.txt
|
||||
|
||||
- name: Seed a real company with posted verifikat
|
||||
if: steps.newmig.outputs.count != '0'
|
||||
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/upgrade/seed.sql
|
||||
|
||||
- name: Apply ONLY the new migrations, against existing data
|
||||
if: steps.newmig.outputs.count != '0'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
while read -r f; do
|
||||
echo "Applying (new) $f"
|
||||
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
|
||||
done < /tmp/new-migrations.txt
|
||||
|
||||
- name: Assert the existing data survived
|
||||
if: steps.newmig.outputs.count != '0'
|
||||
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f tests/pg/upgrade/assert.sql
|
||||
|
||||
- name: No migrations in this PR
|
||||
if: steps.newmig.outputs.count == '0'
|
||||
run: echo "This PR adds no migration; nothing to upgrade-test."
|
||||
|
||||
pg-real:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
|
||||
Reference in New Issue
Block a user