16fbcefbbc
* feat(invariants): centralise shared format contracts, reconcile the org-number paths
The same format rules were written out independently across the codebase, and
where they disagreed the disagreement was invisible until a filing failed.
Worst case, now fixed: four Skatteverket- and Bolagsverket-bound export paths
each had their own idea of a valid organisationsnummer.
lib/skatteverket/format.ts strip '-' only threw on any input with a space
lib/salary/ku/ku10-generator.ts replace('-', '') first hyphen only, spaces survived
lib/salary/agi/xml-generator.ts strip non-digits stray letters passed the length check
lib/bokslut/ixbrl/validate /^\d{6}-?\d{4}$/ rejected the 12-digit form, no Luhn
A company stored with a space or in 12-digit form could file AGI all year and
then fail at the arsredovisning deadline with a message that did not say why.
lib/invariants/ now owns account number, ISO date, four-digit fiscal year and
org number, each with the rationale recorded next to the rule. normalizeOrgNumber
moves here from lib/company-lookup/ and isSaneDateString from lib/utils.ts; both
old paths re-export, so no caller changes. lib/api/schemas.ts builds its
primitives on the module, so ~100 schemas inherit any correction.
The arsredovisning check-digit verdict is a warn, not an error: a wrong Luhn
digit is almost certainly a typo worth surfacing, but whether every org number
Bolagsverket accepts satisfies Luhn is a Swedish domain question we have not
verified against a primary source, and an error there blocks Skicka in. We do
not block a statutory filing on an unverified assumption.
KU10 still passes a 12-digit stored org number through unfolded. That is
pre-existing, and whether the KU10 schema wants 10 or 12 digits is not covered
by the swedish-payroll skill, so it is pinned by a test rather than changed
silently.
Guard 8 (hand-rolled-invariant) tracks the remaining 114 inline copies as a
ratchet that may only go down, same mechanism as the roundOre guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ci): add an upgrade-path job that applies new migrations against real data
The pg-real job applies all 548 migrations to an EMPTY database. Empty means
zero rows, so a migration that adds a NOT NULL, adds a CHECK, creates a unique
index or backfills passes trivially in CI and can still fail on production,
where the rows exist. CI proved that a fresh install works; nothing proved that
an existing install upgrades.
The new pg-upgrade job: apply the schema as it stands at the merge base, seed a
small real company (three posted verifikat, balanced lines, one ore-level
amount), then apply ONLY the migrations this PR adds, then assert the data
survived (entries still posted, lines intact, ledger still balances, ore
unchanged, voucher numbers sequential). A PR with no migration no-ops.
Verified locally against supabase/postgres:15.8.1.060 rather than assumed, with
three deliberately bad migrations:
rescale money on posted lines empty: would pass seeded: ERROR (immutability trigger)
CHECK violating the ore row empty: exit 0 seeded: exit 3
NOT NULL on a populated column empty: exit 0 seeded: exit 3
Base migrations are read out of the merge-base git tree, not the working tree,
so a PR that edits an already-shipped migration still gets the original applied
and the edit surfaces as a failure here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: record the invariants and upgrade-CI decisions
Two entries covering what this PR changes and, more importantly, the calls that
are not obvious from the diff: why the arsredovisning check-digit verdict is a
warning rather than an error, why KU10's 12-digit passthrough is pinned instead
of fixed, and why the ROT/RUT brf org-number schemas stay on their own rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(test): mark the upgrade fixture as CI-only, never a production template
The fixture writes posted journal_entries and their lines directly, bypassing
the engine and the atomic commit RPC. That is the only way to hand a migration
pre-existing posted rows to break, and it is safe against a throwaway CI
database, but it reads like a sanctioned pattern to anyone who finds it later.
Says so explicitly, with the reason it is confined here (no voucher sequence to
keep gapless, no retention obligation on a database destroyed with the job) and
a pointer back to Hard Rule 2 for anything touching a real database.
Raised by the Swedish compliance review bot on #1364.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
203 lines
7.7 KiB
YAML
203 lines
7.7 KiB
YAML
name: pg-real tests
|
|
|
|
on: [pull_request]
|
|
|
|
# Neither job writes anything back: they read the repo, stand up a throwaway
|
|
# Postgres, and run tests. Without this block both inherit the repository's
|
|
# default token permissions, which are broader than that.
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: pg-real-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
coverage-gate:
|
|
# Enforces the database.md rule: a migration touching a trigger/RPC/RLS/
|
|
# DEFERRABLE must come with a *.pg.test.ts change. Previously instruction-
|
|
# only. Escape hatch: `-- pg-test: covered-by <path>` / `-- pg-test: skip
|
|
# (<reason>)` comments inside the migration.
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Full history so the merge-base with the PR base branch exists.
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
|
|
with:
|
|
node-version: 20
|
|
- name: Require pg-real coverage for trigger/RPC/RLS migrations
|
|
env:
|
|
PG_GATE_BASE: origin/${{ github.base_ref }}
|
|
run: node scripts/check-pg-test-coverage.mjs
|
|
|
|
# Proves that an EXISTING database survives the PR's migrations, which the
|
|
# pg-real job below cannot: it applies every migration to an empty database,
|
|
# so a NOT NULL, a CHECK, a unique index or a backfill passes trivially
|
|
# against zero rows and can still fail (or silently corrupt) on production.
|
|
#
|
|
# Shape: schema at the merge-base -> seed real rows -> apply ONLY the new
|
|
# migrations -> assert the rows are intact. A PR that adds no migration skips
|
|
# straight past the apply step and costs one cheap no-op run.
|
|
pg-upgrade:
|
|
runs-on: ubuntu-latest
|
|
|
|
services:
|
|
postgres:
|
|
image: supabase/postgres:15.8.1.060
|
|
env:
|
|
POSTGRES_PASSWORD: postgres
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U postgres"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 20
|
|
|
|
env:
|
|
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
|
|
PGPASSWORD: postgres
|
|
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Full history so the merge-base with the PR base branch exists.
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Install psql client
|
|
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
|
|
|
|
- name: Identify the migrations this PR adds
|
|
id: newmig
|
|
env:
|
|
BASE_REF: origin/${{ github.base_ref }}
|
|
run: |
|
|
set -euo pipefail
|
|
MERGE_BASE=$(git merge-base "$BASE_REF" HEAD)
|
|
echo "merge_base=$MERGE_BASE" >> "$GITHUB_OUTPUT"
|
|
echo "Merge base: $MERGE_BASE"
|
|
|
|
# Migrations present at HEAD but not at the merge base. Uses the git
|
|
# tree, not the filesystem, so a rebase or a merge commit cannot make
|
|
# an already-shipped migration look new.
|
|
git ls-tree -r --name-only HEAD -- supabase/migrations \
|
|
| grep '\.sql$' | sort > /tmp/head-migrations.txt
|
|
git ls-tree -r --name-only "$MERGE_BASE" -- supabase/migrations \
|
|
| grep '\.sql$' | sort > /tmp/base-migrations.txt
|
|
comm -23 /tmp/head-migrations.txt /tmp/base-migrations.txt > /tmp/new-migrations.txt
|
|
|
|
COUNT=$(wc -l < /tmp/new-migrations.txt | tr -d ' ')
|
|
echo "count=$COUNT" >> "$GITHUB_OUTPUT"
|
|
echo "New migrations ($COUNT):"
|
|
cat /tmp/new-migrations.txt
|
|
|
|
- name: Bootstrap storage schema
|
|
if: steps.newmig.outputs.count != '0'
|
|
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
|
|
|
|
- name: Apply the schema as it stands at the merge base
|
|
if: steps.newmig.outputs.count != '0'
|
|
env:
|
|
MERGE_BASE: ${{ steps.newmig.outputs.merge_base }}
|
|
run: |
|
|
set -euo pipefail
|
|
# Read each migration out of the merge-base tree rather than the
|
|
# working tree: a PR that EDITS a shipped migration (forbidden, but
|
|
# this job must not be the thing that hides it) still gets the
|
|
# original applied here, so the edit shows up as a failure below.
|
|
while read -r f; do
|
|
echo "Applying (base) $f"
|
|
git show "$MERGE_BASE:$f" | psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q
|
|
done < /tmp/base-migrations.txt
|
|
|
|
- name: Seed a real company with posted verifikat
|
|
if: steps.newmig.outputs.count != '0'
|
|
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/upgrade/seed.sql
|
|
|
|
- name: Apply ONLY the new migrations, against existing data
|
|
if: steps.newmig.outputs.count != '0'
|
|
run: |
|
|
set -euo pipefail
|
|
while read -r f; do
|
|
echo "Applying (new) $f"
|
|
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
|
|
done < /tmp/new-migrations.txt
|
|
|
|
- name: Assert the existing data survived
|
|
if: steps.newmig.outputs.count != '0'
|
|
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f tests/pg/upgrade/assert.sql
|
|
|
|
- name: No migrations in this PR
|
|
if: steps.newmig.outputs.count == '0'
|
|
run: echo "This PR adds no migration; nothing to upgrade-test."
|
|
|
|
pg-real:
|
|
runs-on: ubuntu-latest
|
|
|
|
services:
|
|
postgres:
|
|
# Supabase image ships the auth schema, auth.uid(), and the extensions
|
|
# (uuid-ossp, pg_cron, btree_gist, vector) this repo's migrations need.
|
|
# Plain postgres:15 would require manual bootstrap SQL.
|
|
image: supabase/postgres:15.8.1.060
|
|
env:
|
|
POSTGRES_PASSWORD: postgres
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U postgres"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 20
|
|
|
|
env:
|
|
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
|
|
PGPASSWORD: postgres
|
|
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
persist-credentials: false
|
|
|
|
# Deliberately no `cache: npm` here. This workflow runs on pull_request
|
|
# only, so the cache is never written on main, and GitHub scopes caches
|
|
# by ref: every PR uploaded its own 284 MB copy under an identical key
|
|
# that no other PR could ever restore. Fourteen dead copies (4 GB, 40% of
|
|
# the repo quota) accumulated in a single day. If this is ever worth
|
|
# caching again, it has to be actions/cache/save on main plus
|
|
# actions/cache/restore here, which is the only shape that gets hits.
|
|
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
|
|
with:
|
|
node-version: 20
|
|
|
|
- run: npm ci
|
|
|
|
- name: Install psql client
|
|
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
|
|
|
|
- name: Bootstrap storage schema
|
|
# The supabase/postgres image ships a partial storage schema; the rest
|
|
# is provisioned by the storage-api service at runtime, which we do
|
|
# not run in CI. This aligns the schema with what migrations expect.
|
|
run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
|
|
|
|
- name: Apply migrations
|
|
run: |
|
|
set -euo pipefail
|
|
shopt -s nullglob
|
|
files=(supabase/migrations/*.sql)
|
|
if [ ${#files[@]} -eq 0 ]; then
|
|
echo "No migration files found"
|
|
exit 1
|
|
fi
|
|
for f in "${files[@]}"; do
|
|
echo "Applying $f"
|
|
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
|
|
done
|
|
|
|
- run: npm run test:pg
|