fix(documents): make the nightly integrity-verify cron finish and surface missing objects (#965)

The nightly verify cron was killed by the platform every run: with a
500-document batch at ~0.8s/doc it hit the function timeout around item
250, so the tail of the queue (1506 current documents) was never checked.
Worse, a document whose storage object could not be downloaded threw
before last_integrity_check_at was stamped, so it sorted back to the head
of the nulls-first queue and re-failed every night without ever surfacing
as an incident.

- Declare maxDuration = 300 and lower the default batch to 200 (named
  constant, env-overridable) so a full run fits the budget with headroom.
- On download failure, write an INTEGRITY_FAILURE audit row marked
  DOCUMENT_OBJECT_MISSING (description prefix + new_state.reason; the DB
  check constraint audit_log_action_check allows only a fixed action set,
  so a brand-new action value is not possible without a migration), then
  stamp last_integrity_check_at so the row stops head-blocking the queue.
  If the audit insert fails the stamp is skipped so the incident write is
  retried next run.
- Fix the stale route comment: the schedule is nightly 03:00 UTC per
  vercel.json, not weekly Sunday.
- seed-demo-account.ts now uploads a tiny valid PDF for the AWS inbox
  demo document and stores its real SHA-256 and byte size, instead of
  inserting a fabricated hash with no storage object (the seeded row that
  tripped the cron every night).
- Add route tests: cron auth 401, happy-path stamping, hash mismatch,
  missing-object incident + stamp, audit-failure retry, batch size, and
  maxDuration.

From the 2026-07-09 production log triage.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-07-10 11:03:54 +02:00
committed by GitHub
co-authored by Claude Fable 5
parent b4a21b1029
commit 7c739529d6
3 changed files with 333 additions and 10 deletions
+24 -6
View File
@@ -22,6 +22,7 @@
import { createClient } from '@supabase/supabase-js'
import { config as dotenv } from 'dotenv'
import { createHash } from 'node:crypto'
import { resolve } from 'node:path'
import { encryptPersonnummer } from '@/lib/salary/personnummer'
@@ -1890,19 +1891,36 @@ async function seedInboxAndUncategorized(
): Promise<void> {
console.log('[6] inbox AWS PDF + 5 uncategorized + voucher gaps')
// Synthetic AWS PDF storage row (no actual file upload: storage path
// exists for demo, file content can be uploaded later via UI)
const fakeHash = 'demo' + Math.random().toString(36).slice(2, 18).padEnd(60, '0')
// Synthetic AWS invoice PDF: upload a tiny but valid PDF so the nightly
// integrity-verify cron can download the object and match its real
// SHA-256, instead of failing forever on a fabricated hash with no file.
const awsPdfBuffer = Buffer.from(
[
'%PDF-1.4',
'1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj',
'2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj',
'3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 595 842] >> endobj',
'trailer << /Root 1 0 R >>',
'%%EOF',
].join('\n'),
'utf8'
)
const awsPdfPath = `${ctx.userId}/${ctx.companyId}/inbox/aws-2026-05-05.pdf`
const { error: uploadErr } = await sb.storage
.from('documents')
.upload(awsPdfPath, awsPdfBuffer, { contentType: 'application/pdf', upsert: true })
if (uploadErr) throw new Error(`storage upload AWS PDF: ${uploadErr.message}`)
const awsPdfHash = createHash('sha256').update(awsPdfBuffer).digest('hex')
const { data: doc, error: docErr } = await sb
.from('document_attachments')
.insert({
user_id: ctx.userId,
company_id: ctx.companyId,
storage_path: `${ctx.userId}/${ctx.companyId}/inbox/aws-2026-05-05.pdf`,
storage_path: awsPdfPath,
file_name: 'aws-2026-05-05.pdf',
file_size_bytes: 124567,
file_size_bytes: awsPdfBuffer.length,
mime_type: 'application/pdf',
sha256_hash: fakeHash,
sha256_hash: awsPdfHash,
version: 1,
is_current_version: true,
uploaded_by: ctx.userId,