Files
accounted/supabase/migrations/20260817120000_skattekonto_file_imports.sql
Jakob Wennberg 4921d1da5e feat(import): import skattekontoutdrag files into the skattekonto pipeline (#1637)
* feat(import): import skattekontoutdrag files into the skattekonto pipeline

Users can now upload the kontohändelse export from Skatteverket's
skattekonto e-service (current CSV layout, verified against a real
2026-08 export, plus legacy .skv files) instead of needing the paid API
connection. Parsed rows land in skattekonto_transactions as booked
file_import rows and inherit the existing 1630 rules engine, bulk
booking, match-to-verifikat and both UIs unchanged.

- Core parser lib/import/skattekonto-file/ with strict detection
  (orgnr header + saldo markers, or two distinct SKV vocabulary terms
  plus row shape), sum-integrity check (opening + rows must equal
  closing) and a wrong-company guard against company_settings.
- computeDedupKey moves to core (lib/skatteverket/skattekonto-dedup);
  the extension re-imports it. File rows hash-key; content-signature
  partitioning skips rows already booked (either key form) and promotes
  matching upcoming rows in place.
- syncSkattekonto gains a takeover step: an id-keyed API row adopts a
  matching hash-keyed imported row in place, so journal links survive
  connecting the API after a file import. Upcoming rows can no longer
  clobber a booked row on hash collision.
- New skattekonto_file_imports table (company-scoped file-hash dedup)
  plus source/file_import_id provenance columns on
  skattekonto_transactions.
- /import gains a Skattekontoutdrag wizard (upload/preview/result,
  deep link ?mode=skattekonto); the bank-file flow detects skattekonto
  files and redirects instead of importing them as bank rows.
- /skattekonto renders imported rows for unconnected companies (attn
  line + import CTA) instead of discarding them behind the StartCard.
- Free for everyone: the local-data booking/match routes were already
  ungated; only API sync/saldo stay capability-gated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(skattekonto): align the EF F-skatt rule with the 2012 -> 2013 decision

20260810120000 established that 2012 is not standard BAS and moved the
booking templates to 2013 (owner taxes in an enskild firma are an eget
uttag), but the skattekonto_rules seed still booked EF preliminarskatt
against 2012. The file importer makes this rule fire for every EF
F-skatt row, so bring it onto 2013 too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(import): apply review findings on the skattekonto file import

- Fix the takeover candidate comparator: the single-argument sort was an
  inconsistent relation and could adopt a stale upcoming row ahead of the
  booked file row in a 3+ candidate queue (regression test added), and
  page the candidate scan with fetchAllRows so a multi-year window is not
  silently capped at 1000 rows.
- Fail parsing when a statement HAS saldo markers but not both readable
  balances: a file cut off before "Utgående saldo" previously skipped the
  sum check entirely. sum_valid stays null only for marker-less legacy
  files.
- Count a promotion only when the UPDATE matched a row, so a concurrent
  sync cannot inflate promoted_count; log a failed finalize of the import
  record instead of discarding the error.
- Migration (unshipped, edited in place): user_id is nullable with
  ON DELETE SET NULL so import records and their file-hash dedup survive
  user deletion, and the INSERT policy binds user_id to auth.uid() so a
  member cannot attribute an import to a colleague. pg tests cover both.
- Make the upload drop zone keyboard-reachable (role, tabIndex, Enter/
  Space) and give the six count-bearing strings ICU plural forms in both
  locales.

Skipped with reasons on the PR: binding execute rows to file bytes and
re-checking orgnr in execute (same client-trust model as the shipped
bank-file execute; Zod + RLS scope writes to the caller's own company),
a 404 test (the route has no not-found path), event-bus clearing in the
route test (the route touches no events), and FK NOT VALID (new column
referencing a brand-new empty table).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 13:18:32 +02:00

93 lines
3.8 KiB
SQL

-- Skattekontoutdrag file import support
--
-- Users can download their kontohändelser from Skatteverket's skattekonto
-- e-service (CSV; legacy exports were semicolon .skv text files) and import
-- them manually. Imported rows land in skattekonto_transactions and inherit
-- the existing booking rules, matching and UI. This serves companies without
-- the paid API connection and history beyond the API's ~555-day lookback.
--
-- Two parts:
-- 1. skattekonto_file_imports: one row per uploaded file, keyed on
-- (company_id, file_hash) for whole-file duplicate rejection
-- (company-scoped from day one; see 20260707130000 for why the
-- user-scoped variant on bank_file_imports had to be fixed later).
-- 2. Provenance on skattekonto_transactions: source distinguishes
-- API-synced rows from file-imported ones (the sync takeover logic
-- needs it), file_import_id links back to the uploaded file.
CREATE TABLE public.skattekonto_file_imports (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
company_id UUID NOT NULL REFERENCES public.companies ON DELETE CASCADE,
-- Importer provenance. Nullable + SET NULL so the import record (and the
-- file-hash dedup it provides) survives the importing user's deletion;
-- the INSERT policy below binds it to auth.uid() so a member cannot
-- attribute an import to a colleague.
user_id UUID REFERENCES auth.users ON DELETE SET NULL,
filename TEXT NOT NULL,
file_hash TEXT NOT NULL,
file_variant TEXT NOT NULL CHECK (file_variant IN ('csv', 'skv')),
row_count INTEGER NOT NULL DEFAULT 0,
imported_count INTEGER NOT NULL DEFAULT 0,
duplicate_count INTEGER NOT NULL DEFAULT 0,
promoted_count INTEGER NOT NULL DEFAULT 0,
date_from DATE,
date_to DATE,
-- "Utgående saldo" from the statement's final marker row. Not yet used
-- for the balance snapshot (that write stays API-owned for now) but
-- stored so an import-history or avstämning view can surface it.
closing_saldo NUMERIC(14, 2),
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'processing', 'completed', 'failed')),
error_message TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (company_id, file_hash)
);
CREATE INDEX skattekonto_file_imports_company_created_idx
ON public.skattekonto_file_imports (company_id, created_at DESC);
ALTER TABLE public.skattekonto_file_imports ENABLE ROW LEVEL SECURITY;
CREATE POLICY "Users see skattekonto file imports for their companies"
ON public.skattekonto_file_imports FOR SELECT
USING (company_id IN (SELECT public.user_company_ids()));
CREATE POLICY "Users insert skattekonto file imports for their companies"
ON public.skattekonto_file_imports FOR INSERT
WITH CHECK (
company_id IN (SELECT public.user_company_ids())
AND user_id = auth.uid()
);
CREATE POLICY "Users update skattekonto file imports for their companies"
ON public.skattekonto_file_imports FOR UPDATE
USING (company_id IN (SELECT public.user_company_ids()))
WITH CHECK (company_id IN (SELECT public.user_company_ids()));
CREATE POLICY "Users delete skattekonto file imports for their companies"
ON public.skattekonto_file_imports FOR DELETE
USING (company_id IN (SELECT public.user_company_ids()));
CREATE TRIGGER update_skattekonto_file_imports_updated_at
BEFORE UPDATE ON public.skattekonto_file_imports
FOR EACH ROW
EXECUTE FUNCTION public.update_updated_at_column();
-- Provenance columns. Existing rows were all written by the API sync, so
-- the 'api' default backfills them correctly.
ALTER TABLE public.skattekonto_transactions
ADD COLUMN source TEXT NOT NULL DEFAULT 'api'
CHECK (source IN ('api', 'file_import')),
ADD COLUMN file_import_id UUID
REFERENCES public.skattekonto_file_imports ON DELETE SET NULL;
NOTIFY pgrst, 'reload schema';