fix(ci): replace the integration anti-skip guard that could not pass
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 47s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
CI (SIAX Cloud) / quality (pull_request) Successful in 38s

The guard grepped vitest's HUMAN output:

    grep -qE "skipped" && ! grep -qE "Tests  [0-9]+ passed"

vitest colours that summary. The real line emitted on these runners is

    ESC[2m      Tests ESC[22m ESC[1mESC[32m97 passedESC[39m | ...

so ANSI escapes sit between "Tests " and the digits and the
two-spaces-then-digits pattern can never match. The first grep still
matched "skipped", so the condition was true whenever a single test
skipped -- no matter how many passed.

Measured on run 13911 job 31183 (this branch, 2026-09-08): 13 files /
97 tests PASSED, 1 file / 2 tests skipped, and the step still printed
"FATAL: integration suites skipped instead of running" and exited 1.
That message was false; the suites ran.

It survived review because it is correct locally -- redirecting vitest
to a file disables its colour, so the pattern matches on a developer
machine and misfires only under CI.

Replaced with vitest's JSON reporter: structured, colourless, and
fail-closed if the report is missing. A/B measured on real reports in
BOTH directions, so the replacement is not merely green:

    all skipped      (0 passed / 99 pending)  -> exit 1  (protection kept)
    945 passed + 184 skipped                  -> exit 0  (false red gone)
    report file absent                        -> exit 1  (fail closed)

The anti-skip guarantee the original guard was written for is preserved:
a run where every suite skips still fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Simon Axelsson
2026-09-08 01:23:55 +02:00
co-authored by Claude Opus 5
parent 1322d26fac
commit be5e598646
+43 -7
View File
@@ -141,17 +141,53 @@ jobs:
[ -n "$FOUND" ] || { echo "FATAL: no ClickHouse behind the alias accepts this run's credentials"; exit 1; }
echo "CLICKHOUSE_URL=http://$FOUND:8123" >> "$GITHUB_ENV"
echo "CLICKHOUSE_AUTH=opt_ci:$PASS" >> "$GITHUB_ENV"
# The anti-skip guard here used to grep vitest's HUMAN output:
#
# grep -qE "skipped" && ! grep -qE "Tests [0-9]+ passed"
#
# vitest colours that summary, so the real line on this runner is
# ESC[2m Tests ESC[22m ESC[1mESC[32m97 passedESC[39m ... — ANSI escapes
# sit between "Tests " and the digits, so the two-spaces-then-digit
# pattern can never match. The first grep still matched "skipped", so
# the guard fired on ANY run containing a single skipped test while
# printing "integration suites skipped instead of running" — a false
# statement. Measured 2026-09-08, run 13911 job 31183: 13 files / 97
# tests PASSED, 1 file / 2 tests skipped → FATAL anyway.
#
# It survived review because it is correct locally: redirecting to a
# file disables vitest's colour, so the pattern matches on a developer
# machine and only misfires under CI.
#
# Replaced with the JSON reporter — structured and colourless. A/B
# measured on real reports, BOTH directions:
# all-skipped (0 passed / 99 pending) → RED (protection preserved)
# 945 passed + 184 skipped → GREEN (false red removed)
- name: Integration tests against LIVE OpenSearch + ClickHouse (blocking)
run: |
set -eu
# Guard: env MUST be present here — the suites self-skip without it,
# and a skipped suite must never masquerade as a green integration
# gate (Master-DoD §56 status honesty).
# Env MUST be present here — the suites self-skip without it, and a
# skipped suite must never masquerade as a green integration gate
# (Master-DoD §56 status honesty).
[ -n "${OPENSEARCH_URL:-}" ] && [ -n "${CLICKHOUSE_URL:-}" ]
pnpm vitest run tests/integration 2>&1 | tee /tmp/itest.log
if grep -qE "skipped" /tmp/itest.log && ! grep -qE "Tests [0-9]+ passed" /tmp/itest.log; then
echo "FATAL: integration suites skipped instead of running"; exit 1
fi
pnpm vitest run tests/integration \
--reporter=default \
--reporter=json --outputFile.json=/tmp/itest.json
node -e '
const fs = require("fs");
const p = "/tmp/itest.json";
if (!fs.existsSync(p)) {
console.error("FATAL: vitest produced no JSON report"); process.exit(1);
}
const r = JSON.parse(fs.readFileSync(p, "utf8"));
console.log(`integration: ${r.numPassedTests} passed, ${r.numPendingTests} skipped, ${r.numFailedTests} failed, ${r.numTotalTests} total`);
if (r.numFailedTests > 0) {
console.error("FATAL: integration tests failed"); process.exit(1);
}
if (r.numPassedTests === 0) {
console.error("FATAL: no integration test actually executed — every suite skipped");
process.exit(1);
}
'
# BLOCKING. This job publishes the protected-branch status check context
# "CI (SIAX Cloud) / security (push)", which branch protection on `main`