feat(reconciliation): match migrated bank history against imported SIE verifikat (#1598)
* feat(reconciliation): match migrated bank history against imported SIE verifikat A first-class Fortnox/SIE migrator path: after SIE import plus bank connect or bank CSV upload, historical bank rows are auto-matched (>= 0.9) or suggestion-matched (0.75-0.89, persisted for review) against the imported verifikat, with a guided review surface, instead of landing as anonymous "Att bokfora" rows. Phase 0: per-cash-account unattended sweep (fixes #1298 cross-account pooling); widen payment_match_log action CHECK with linked_to_existing_voucher (silently unlogged since March). Phase 1: potential_journal_entry_id/method/confidence on transactions with CHECK + invalidation triggers; persistSuggestions in runReconciliation; sweep after bank CSV import with SIE overlap (suppressing auto-categorization); sweep summaries stamped on bank_connections and bank_file_imports; POST /api/reconciliation/bank/confirm-suggestions with per-pair server-side revalidation (voucher consumption + bank-leg amount and direction). Phase 2: "Granska forslag" review tab on Transactions with chunked bulk confirm, per-row fallbacks, "Kor matchning igen" (all_accounts sweep mode, mutually exclusive with dry_run), attn line, pre-migration row marker. Phase 3: ImportResultStep dual CTA (bank connect + CSV), migrator variant of the account-picker #917 nudge, sweep outcome on the onboarding checklist bank step. Non-selection apply runs on /api/reconciliation/bank/run now floor at 0.9 and persist the review band instead of auto-committing fuzzy matches. Migrations already applied to staging under the same versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): resolve PR review findings in one pass Swedish accounting review (both previously-deferred holes closed): - runReconciliation's >= 0.9 auto-apply now writes 'matched' to payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); the bus event alone lands in the 30-day event_log and is not an audit record. - The three match-route storno-conflict branches detach reconciliation links via unlinkReconciliation instead of storno-reversing the linked verifikat: a reconciliation link points at an independent verifikat that may evidence other affarshandelser, and a wholesale reversal is an over-broad rattelse (BFL 5 kap 5 §). - Historical gap quantified on prod (read-only, recorded in DECISIONS): 762 unlogged manual links across 52 companies since 2026-03-23. CodeRabbit: - confirm-suggestions route: maxDuration 300 for full 500-item batches. - AccountPickerDialog: migrator-nudge buttons set lookbackTouched so the async gap-fill probe cannot override an explicit choice. - enable-banking post-backfill sweep: persistSuggestions so the review band is not dropped. - bank-file execute: sie_sweep stamp errors are logged, not swallowed. - ImportResultStep: sandbox keeps the CSV CTA (file import works there). - payment_match_log CHECK swap: NOT VALID + VALIDATE, no table scan under ACCESS EXCLUSIVE. - logMatchEvent calls awaited (serverless can freeze unawaited work). - DECISIONS.md stale version reference annotated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): defer reconciliation-link detach until the match commits Round-2 review findings: - CodeRabbit: the eager unlinkReconciliation call could orphan a transaction if the match flow failed after it. All three match routes now persist NOTHING up front: the final transaction update overwrites journal_entry_id and clears reconciliation_method in the same write, so any failure in between leaves the existing link intact. The release is logged as 'unmatched' after the commit. - Swedish review: the auto_suggested logMatchEvent in runReconciliation is now awaited like every other audit write. - DECISIONS entry split into compliance/CodeRabbit lines and updated to describe the deferred detach. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): literal reconciliation_method payloads for the phantom-column scanner The conditional spreads introduced with the deferred detach pushed the scanner's unresolvable-expression count past its ceiling (380 > 378). reconciliation_method: null is correct unconditionally on a confirmed invoice/supplier match (null is already the value on every row that was not reconciliation-linked), so the payloads become plain literals the guard can verify. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
07e89d9b52
commit
08440fed94
@@ -901,6 +901,13 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-08-13] Underlag failures resolve the response where it fails (getResponseErrorMessage + status) and an expired session announces itself on the existing session-timeout BroadcastChannel, instead of a global authenticated-fetch wrapper: `throw new Error(json.error)` printed "[object Object]" for the structured envelope, so the middleware 401 on a backgrounded mobile tab surfaced as the generic "Något gick fel" with no way back; a wrapper would have to be threaded through every call site to buy the same thing here. Upload failures also post metadata (status, size, mime, resolved reason) to /api/log, the one API path exempt from the timeout gate, because a request answered before the route runs leaves nothing in the function logs and the user-reported failures were invisible there.
|
||||
[2026-08-13] Oversized phone photos are re-encoded in the browser (2400px long edge, JPEG q0.85 stepping down) rather than raising a platform limit or streaming straight to storage: hosted rejects any request body over 4.5 MB itself (measured against prod: 4.4 MB reaches the route, 4.6 MB returns a plain-text FUNCTION_PAYLOAD_TOO_LARGE), before the route runs and therefore invisibly in the function logs, while the route advertises 10 MB it can never receive. A downscaled photo is still a faithful, durably readable reproduction (BFL 7 kap), which a refusal is not. What cannot be shrunk (PDF, or HEIC where the browser will not decode it) is refused client-side with its actual size named, and 413 was added to the HTTP status map so a rejection in transit still says what happened. Direct-to-storage upload, which would remove the ceiling for PDFs too, is the follow-up, not this fix: it moves sha256/WORM integrity off the server.
|
||||
[2026-08-13] The book-route underlag fix landed as a pinned-document leg inside propagateUnderlagForBookedTransaction rather than the planned "extract categorize-core's propagation block into a shared helper": PR #1547 had already done that extraction overnight and wired /book and bulk-book to the shared helper, but the helper only walked matched inbox items, so a document pinned via transactions.document_id with no unconsumed inbox item (direct upload, or item consumed elsewhere) still booked to "Underlag saknas". Anchoring the pin inside the helper fixes /book, categorize, bulk-book and attach-after-book in one place; the pin is read fresh (not from the caller's pre-booking snapshot) so a concurrent attach still anchors, and the bulk-book RPC's own atomic doc-linking makes the leg a no-op there.
|
||||
[2026-08-13] Bank↔SIE match suggestions stored as three columns on transactions (potential_journal_entry_id/method/confidence, mirroring potential_invoice_id) with DB-trigger invalidation, not a suggestions table: one candidate per row matches the greedy matcher's contract, the enrichment/consume/clear paths are the exact shape the invoice hints already use, and triggers (self-clear on link/ignore, sibling-clear on consumption, clear on storno reversal) cover every write path including MCP and RPCs where app-side clearing cannot.
|
||||
[2026-08-13] /api/reconciliation/bank/run apply runs WITHOUT selected_matches now apply at the 0.9 unattended floor and persist the 0.75-0.89 band as suggestions, a deliberate behavior change from apply-everything-including-fuzzy: nobody reviewed those pairs, and auto-committing 0.75-fuzzy matches on an unreviewed run is exactly what the unattended threshold exists to prevent. Reviewed applies (selected_matches) keep the legacy no-floor behavior.
|
||||
[2026-08-13] "Granska migrerad historik" is a conditional third tab on the existing Transactions page (renders only while suggestion rows exist), not a new page: the plan called for a filtered view, the rows are transactions, and a permanent surface would advertise a migrator-only flow to every company. Its attn line yields to the SKV-reconnect line (max one per page, convention 6).
|
||||
[2026-08-13] The pre-migration row marker keys on sie_imports.fiscal_year_end, deliberately period-based: it labels which period a row belongs to. This is NOT the #917 trap (suggesting a sync-skip date from fiscal_year_end); the marker suggests nothing, and the account picker's migrator nudge uses fiscal_year_start to pull MORE history, never less.
|
||||
[2026-08-13] Staging migrations (20260813120000 [file since renamed to 20260813210000, see the rename entry below], 20260813121000) applied via execute_sql with explicit supabase_migrations.schema_migrations inserts under the exact file versions, instead of MCP apply_migration: apply_migration stamps its own apply-time version, which would leave staging's history diverging from the repo files mid-reconcile (the drift Emil is currently draining). Prod untouched; it gets both files at merge.
|
||||
[2026-08-13] /skeptic on bank-and-sie-match returned BLOCK (correctness + compliance skeptics refuted; regression could not). All four blocking defects fixed in the same worktree: bulk confirm now chunks at the schema's 500-id cap; all_accounts rejects (400) any combination with dry_run/account_number/selected_matches instead of silently applying on a requested preview; the sweep summary JSONB carries `errors` and the checklist note stays silent on an incomplete sweep; confirm revalidates the voucher's bank-leg amount AND direction at click time (closes the inline-rattelse staleness hole, since strike-and-replace keeps status posted and fires no invalidation trigger) and resolves NULL-cash_account_id rows via the PRIMARY cash account instead of hard-coded 1930.
|
||||
[2026-08-13] Two pre-existing holes the skeptic surfaced were deliberately NOT fixed here (scope containment): (a) the >= 0.9 auto-apply in runReconciliation has never written payment_match_log ('matched' is only logged by the invoice-match routes), which fails the log's own BFL 7:1 charter and predates this feature; (b) the match-invoice routes' storno-conflict branch reverses ANY linked journal entry without checking reconciliation_method, so a reconciliation link to a multi-event SIE verifikat can trigger a whole-entry reversal. Both need their own issues/PRs; this feature widens their exposed population, so they should land before or with the migrator rollout.
|
||||
[2026-08-13] Correction-chain depth guard measures depth by walking correction_of_id/reverses_id links (new lib/core/bookkeeping/correction-chain.ts), never by matching "Rättelse:" prefixes in descriptions: custom verifikationstext (allowed since #1031) would dodge any text-based check. Threshold 3, advisory with an explicit allow_deep_chain bypass on every surface (service option, REST body, MCP tool arg, UI confirm) per the standing soft-guard rule; Christoffer's agent chains hit 10 deep. The guard also fires at MCP staging time, not only at commit, so the agent reconsiders in the same turn instead of after approval.
|
||||
[2026-08-13] The reverse (storno) UI on the journal detail page got the same catch-and-confirm bypass as the correction dialog even though the plan scoped UI work to the correction dialog: reverseEntry carries the same guard, and without a working "Återför ändå" the guard would dead-end in a toast, which the soft-guard rule forbids.
|
||||
[2026-08-13] tools/list payload ceiling bumped 59K to 59.5K for the two allow_deep_chain schema properties (trimmed to one sentence first): the bypass is wire contract agents must discover to act on CORRECTION_CHAIN_TOO_DEEP, not trimmable prose.
|
||||
@@ -941,3 +948,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-08-13] Per-account VAT treatment is class-aware; explicit values extend custom accounts while canonical accounts keep their static BAS momsdeklaration mapping. SIE #SRU and #KTYP never supply it because they encode tax-return fields and account class, not momsdeklaration treatment. VMB carries no default account rate because its VAT base is the margin, not gross sales.
|
||||
[2026-08-13] Issue #1408 separates evidence classification from lock and VAT overlays, and treats import and correction sources as false positives for live-template provenance: 1,316 of 1,361 production signature matches came from those sources, so a broad account signature must not become a correction queue.
|
||||
[2026-08-13] Issue #1408 does not route locked candidates through the current correctEntry service or an unlock: the service creates its storno in the original period, so a compliant locked-period correction needs a separately tested open-period storno and replacement path that preserves the original and full rättelse trail.
|
||||
[2026-08-13] payment_match_log CHECK migration renamed 20260813120000 -> 20260813210000 at merge time: main gained its own 20260813120000 (fix_franvaro_audit_trigger_definer) while this branch was in flight, and identical versions abort the Supabase apply with a schema_migrations_pkey duplicate. Staging's tracking row was updated to the new version in the same step, freeing the 20260813120000 slot for main's migration when staging reconciles.
|
||||
[2026-08-13] PR #1598, compliance findings closed with the rollout after the Swedish accounting review escalated them from follow-up to fix-with-rollout: (a) runReconciliation's >= 0.9 auto-apply now writes 'matched' to payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); (b) the three match-route storno-conflict branches no longer storno-reverse a reconciliation-linked verifikat: a reconciliation link points at an independent verifikat that may evidence other affarshandelser, and a wholesale reversal is an over-broad rattelse (BFL 5 kap 5 §). The detach is DEFERRED (round 2, CodeRabbit): nothing is persisted up front; the final transaction update overwrites the pointer and clears reconciliation_method in the same write, so a failure anywhere in the match flow leaves the existing link intact, and the release is logged as 'unmatched' after the commit.
|
||||
[2026-08-13] PR #1598, CodeRabbit findings: confirm-suggestions maxDuration 300; lookbackTouched on the migrator nudge buttons; persistSuggestions on main's post-backfill sweep; sie_sweep stamp errors logged; sandbox keeps the CSV CTA (file import works there); payment_match_log CHECK swap now NOT VALID + VALIDATE (no table-scan under ACCESS EXCLUSIVE); every logMatchEvent call awaited (serverless can freeze unawaited work).
|
||||
[2026-08-13] Historical audit gap quantified on prod (read-only): 762 manual-method links across 52 companies since 2026-03-23 have no payment_match_log row (upper bound: includes linked_to_existing_voucher drops AND older unlogged manual paths). Not backfillable (the inserts never landed); the links themselves are intact on transactions. Recorded here as the explicit ops note the compliance review asked for.
|
||||
|
||||
Reference in New Issue
Block a user