Files
accounted/lib/import/skattekonto-file/import-service.ts
T
4921d1da5e feat(import): import skattekontoutdrag files into the skattekonto pipeline (#1637)
* feat(import): import skattekontoutdrag files into the skattekonto pipeline

Users can now upload the kontohändelse export from Skatteverket's
skattekonto e-service (current CSV layout, verified against a real
2026-08 export, plus legacy .skv files) instead of needing the paid API
connection. Parsed rows land in skattekonto_transactions as booked
file_import rows and inherit the existing 1630 rules engine, bulk
booking, match-to-verifikat and both UIs unchanged.

- Core parser lib/import/skattekonto-file/ with strict detection
  (orgnr header + saldo markers, or two distinct SKV vocabulary terms
  plus row shape), sum-integrity check (opening + rows must equal
  closing) and a wrong-company guard against company_settings.
- computeDedupKey moves to core (lib/skatteverket/skattekonto-dedup);
  the extension re-imports it. File rows hash-key; content-signature
  partitioning skips rows already booked (either key form) and promotes
  matching upcoming rows in place.
- syncSkattekonto gains a takeover step: an id-keyed API row adopts a
  matching hash-keyed imported row in place, so journal links survive
  connecting the API after a file import. Upcoming rows can no longer
  clobber a booked row on hash collision.
- New skattekonto_file_imports table (company-scoped file-hash dedup)
  plus source/file_import_id provenance columns on
  skattekonto_transactions.
- /import gains a Skattekontoutdrag wizard (upload/preview/result,
  deep link ?mode=skattekonto); the bank-file flow detects skattekonto
  files and redirects instead of importing them as bank rows.
- /skattekonto renders imported rows for unconnected companies (attn
  line + import CTA) instead of discarding them behind the StartCard.
- Free for everyone: the local-data booking/match routes were already
  ungated; only API sync/saldo stay capability-gated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(skattekonto): align the EF F-skatt rule with the 2012 -> 2013 decision

20260810120000 established that 2012 is not standard BAS and moved the
booking templates to 2013 (owner taxes in an enskild firma are an eget
uttag), but the skattekonto_rules seed still booked EF preliminarskatt
against 2012. The file importer makes this rule fire for every EF
F-skatt row, so bring it onto 2013 too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(import): apply review findings on the skattekonto file import

- Fix the takeover candidate comparator: the single-argument sort was an
  inconsistent relation and could adopt a stale upcoming row ahead of the
  booked file row in a 3+ candidate queue (regression test added), and
  page the candidate scan with fetchAllRows so a multi-year window is not
  silently capped at 1000 rows.
- Fail parsing when a statement HAS saldo markers but not both readable
  balances: a file cut off before "Utgående saldo" previously skipped the
  sum check entirely. sum_valid stays null only for marker-less legacy
  files.
- Count a promotion only when the UPDATE matched a row, so a concurrent
  sync cannot inflate promoted_count; log a failed finalize of the import
  record instead of discarding the error.
- Migration (unshipped, edited in place): user_id is nullable with
  ON DELETE SET NULL so import records and their file-hash dedup survive
  user deletion, and the INSERT policy binds user_id to auth.uid() so a
  member cannot attribute an import to a colleague. pg tests cover both.
- Make the upload drop zone keyboard-reachable (role, tabIndex, Enter/
  Space) and give the six count-bearing strings ICU plural forms in both
  locales.

Skipped with reasons on the PR: binding execute rows to file bytes and
re-checking orgnr in execute (same client-trust model as the shipped
bank-file execute; Zod + RLS scope writes to the caller's own company),
a 404 test (the route has no not-found path), event-bus clearing in the
route test (the route touches no events), and FK NOT VALID (new column
referencing a brand-new empty table).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 13:18:32 +02:00

150 lines
4.7 KiB
TypeScript

/**
* Server-side execution of a skattekontoutdrag file import.
*
* Writes to skattekonto_transactions directly (RLS-scoped): imported rows
* then flow through the exact same booking/matching pipeline as API-synced
* rows. The API sync's takeover step (skattekonto-sync.ts) reconciles these
* hash-keyed rows if the company later connects Skatteverket.
*/
import type { SupabaseClient } from '@supabase/supabase-js'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import {
assignFileDedupKeys,
partitionFileRows,
type ExistingSkattekontoRow,
type SkattekontoFileRowIdentity,
} from '@/lib/skatteverket/skattekonto-dedup'
/**
* Existing rows in the statement's date span, for partitioning. Paged: a
* statement can cover years and PostgREST caps unpaged reads at 1000 rows.
*/
export async function fetchExistingSkattekontoRows(
supabase: SupabaseClient,
companyId: string,
dateFrom: string,
dateTo: string,
): Promise<ExistingSkattekontoRow[]> {
return fetchAllRows<ExistingSkattekontoRow>(({ from, to }) =>
supabase
.from('skattekonto_transactions')
.select('id, dedup_key, status, transaktionsdatum, transaktionstext, belopp_skatteverket')
.eq('company_id', companyId)
.gte('transaktionsdatum', dateFrom)
.lte('transaktionsdatum', dateTo)
.order('id', { ascending: true })
.range(from, to),
)
}
export interface SkattekontoFileImportOutcome {
imported: number
duplicates: number
promoted: number
/** Rows that failed to write for reasons other than being duplicates. */
errors: number
first_error: string | null
}
const INSERT_BATCH_SIZE = 200
/** Postgres unique_violation */
const UNIQUE_VIOLATION = '23505'
/**
* Partition the confirmed rows against the table and write them:
* inserts for new rows, in-place status promotions for upcoming rows the
* statement proves settled, skips for rows already booked.
*
* Batched inserts fall back to per-row on unique violations so one residual
* conflict (e.g. a concurrent sync) degrades to a counted duplicate instead
* of failing the whole import.
*/
export async function executeSkattekontoFileImport(
supabase: SupabaseClient,
companyId: string,
fileImportId: string,
rows: SkattekontoFileRowIdentity[],
): Promise<SkattekontoFileImportOutcome> {
const dates = rows.map((r) => r.transaktionsdatum).sort()
const existing = await fetchExistingSkattekontoRows(
supabase,
companyId,
dates[0],
dates[dates.length - 1],
)
const partition = partitionFileRows(assignFileDedupKeys(rows), existing)
const outcome: SkattekontoFileImportOutcome = {
imported: 0,
duplicates: partition.duplicates.length,
promoted: 0,
errors: 0,
first_error: null,
}
const toRow = (entry: { row: SkattekontoFileRowIdentity; dedupKey: string }) => ({
company_id: companyId,
transaktionsidentitet: null,
dedup_key: entry.dedupKey,
transaktionsdatum: entry.row.transaktionsdatum,
forfallodatum: null,
ranteberakningsdatum: null,
transaktionstext: entry.row.transaktionstext,
belopp_skatteverket: entry.row.belopp,
belopp_kronofogden: null,
status: 'booked' as const,
source: 'file_import' as const,
file_import_id: fileImportId,
})
for (let i = 0; i < partition.toInsert.length; i += INSERT_BATCH_SIZE) {
const batch = partition.toInsert.slice(i, i + INSERT_BATCH_SIZE)
const { error } = await supabase
.from('skattekonto_transactions')
.insert(batch.map(toRow))
if (!error) {
outcome.imported += batch.length
continue
}
if (error.code !== UNIQUE_VIOLATION) {
outcome.errors += batch.length
outcome.first_error = outcome.first_error ?? error.message
continue
}
for (const entry of batch) {
const { error: rowError } = await supabase
.from('skattekonto_transactions')
.insert(toRow(entry))
if (!rowError) {
outcome.imported++
} else if (rowError.code === UNIQUE_VIOLATION) {
outcome.duplicates++
} else {
outcome.errors++
outcome.first_error = outcome.first_error ?? rowError.message
}
}
}
for (const promotion of partition.promotions) {
// Return the updated rows: when a concurrent sync already flipped the
// row, zero rows match and the promotion must not be counted.
const { data: updated, error } = await supabase
.from('skattekonto_transactions')
.update({ status: 'booked' })
.eq('id', promotion.existingId)
.eq('company_id', companyId)
.eq('status', 'upcoming')
.select('id')
if (error) {
outcome.errors++
outcome.first_error = outcome.first_error ?? error.message
} else if (updated && updated.length > 0) {
outcome.promoted++
}
}
return outcome
}