fix(documents): make the nightly integrity-verify cron finish and surface missing objects (#965)
The nightly verify cron was killed by the platform every run: with a 500-document batch at ~0.8s/doc it hit the function timeout around item 250, so the tail of the queue (1506 current documents) was never checked. Worse, a document whose storage object could not be downloaded threw before last_integrity_check_at was stamped, so it sorted back to the head of the nulls-first queue and re-failed every night without ever surfacing as an incident. - Declare maxDuration = 300 and lower the default batch to 200 (named constant, env-overridable) so a full run fits the budget with headroom. - On download failure, write an INTEGRITY_FAILURE audit row marked DOCUMENT_OBJECT_MISSING (description prefix + new_state.reason; the DB check constraint audit_log_action_check allows only a fixed action set, so a brand-new action value is not possible without a migration), then stamp last_integrity_check_at so the row stops head-blocking the queue. If the audit insert fails the stamp is skipped so the incident write is retried next run. - Fix the stale route comment: the schedule is nightly 03:00 UTC per vercel.json, not weekly Sunday. - seed-demo-account.ts now uploads a tiny valid PDF for the AWS inbox demo document and stores its real SHA-256 and byte size, instead of inserting a fabricated hash with no storage object (the seeded row that tripped the cron every night). - Add route tests: cron auth 401, happy-path stamping, hash mismatch, missing-object incident + stamp, audit-failure retry, batch size, and maxDuration. From the 2026-07-09 production log triage. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
b4a21b1029
commit
7c739529d6
@@ -22,6 +22,7 @@
|
||||
|
||||
import { createClient } from '@supabase/supabase-js'
|
||||
import { config as dotenv } from 'dotenv'
|
||||
import { createHash } from 'node:crypto'
|
||||
import { resolve } from 'node:path'
|
||||
import { encryptPersonnummer } from '@/lib/salary/personnummer'
|
||||
|
||||
@@ -1890,19 +1891,36 @@ async function seedInboxAndUncategorized(
|
||||
): Promise<void> {
|
||||
console.log('[6] inbox AWS PDF + 5 uncategorized + voucher gaps')
|
||||
|
||||
// Synthetic AWS PDF storage row (no actual file upload: storage path
|
||||
// exists for demo, file content can be uploaded later via UI)
|
||||
const fakeHash = 'demo' + Math.random().toString(36).slice(2, 18).padEnd(60, '0')
|
||||
// Synthetic AWS invoice PDF: upload a tiny but valid PDF so the nightly
|
||||
// integrity-verify cron can download the object and match its real
|
||||
// SHA-256, instead of failing forever on a fabricated hash with no file.
|
||||
const awsPdfBuffer = Buffer.from(
|
||||
[
|
||||
'%PDF-1.4',
|
||||
'1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj',
|
||||
'2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj',
|
||||
'3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 595 842] >> endobj',
|
||||
'trailer << /Root 1 0 R >>',
|
||||
'%%EOF',
|
||||
].join('\n'),
|
||||
'utf8'
|
||||
)
|
||||
const awsPdfPath = `${ctx.userId}/${ctx.companyId}/inbox/aws-2026-05-05.pdf`
|
||||
const { error: uploadErr } = await sb.storage
|
||||
.from('documents')
|
||||
.upload(awsPdfPath, awsPdfBuffer, { contentType: 'application/pdf', upsert: true })
|
||||
if (uploadErr) throw new Error(`storage upload AWS PDF: ${uploadErr.message}`)
|
||||
const awsPdfHash = createHash('sha256').update(awsPdfBuffer).digest('hex')
|
||||
const { data: doc, error: docErr } = await sb
|
||||
.from('document_attachments')
|
||||
.insert({
|
||||
user_id: ctx.userId,
|
||||
company_id: ctx.companyId,
|
||||
storage_path: `${ctx.userId}/${ctx.companyId}/inbox/aws-2026-05-05.pdf`,
|
||||
storage_path: awsPdfPath,
|
||||
file_name: 'aws-2026-05-05.pdf',
|
||||
file_size_bytes: 124567,
|
||||
file_size_bytes: awsPdfBuffer.length,
|
||||
mime_type: 'application/pdf',
|
||||
sha256_hash: fakeHash,
|
||||
sha256_hash: awsPdfHash,
|
||||
version: 1,
|
||||
is_current_version: true,
|
||||
uploaded_by: ctx.userId,
|
||||
|
||||
Reference in New Issue
Block a user