fix(ci): replace the integration anti-skip guard that could not pass
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 47s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
CI (SIAX Cloud) / quality (pull_request) Successful in 38s
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 47s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
CI (SIAX Cloud) / quality (pull_request) Successful in 38s
The guard grepped vitest's HUMAN output:
grep -qE "skipped" && ! grep -qE "Tests [0-9]+ passed"
vitest colours that summary. The real line emitted on these runners is
ESC[2m Tests ESC[22m ESC[1mESC[32m97 passedESC[39m | ...
so ANSI escapes sit between "Tests " and the digits and the
two-spaces-then-digits pattern can never match. The first grep still
matched "skipped", so the condition was true whenever a single test
skipped -- no matter how many passed.
Measured on run 13911 job 31183 (this branch, 2026-09-08): 13 files /
97 tests PASSED, 1 file / 2 tests skipped, and the step still printed
"FATAL: integration suites skipped instead of running" and exited 1.
That message was false; the suites ran.
It survived review because it is correct locally -- redirecting vitest
to a file disables its colour, so the pattern matches on a developer
machine and misfires only under CI.
Replaced with vitest's JSON reporter: structured, colourless, and
fail-closed if the report is missing. A/B measured on real reports in
BOTH directions, so the replacement is not merely green:
all skipped (0 passed / 99 pending) -> exit 1 (protection kept)
945 passed + 184 skipped -> exit 0 (false red gone)
report file absent -> exit 1 (fail closed)
The anti-skip guarantee the original guard was written for is preserved:
a run where every suite skips still fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
1322d26fac
commit
be5e598646
+43
-7
@@ -141,17 +141,53 @@ jobs:
|
||||
[ -n "$FOUND" ] || { echo "FATAL: no ClickHouse behind the alias accepts this run's credentials"; exit 1; }
|
||||
echo "CLICKHOUSE_URL=http://$FOUND:8123" >> "$GITHUB_ENV"
|
||||
echo "CLICKHOUSE_AUTH=opt_ci:$PASS" >> "$GITHUB_ENV"
|
||||
# The anti-skip guard here used to grep vitest's HUMAN output:
|
||||
#
|
||||
# grep -qE "skipped" && ! grep -qE "Tests [0-9]+ passed"
|
||||
#
|
||||
# vitest colours that summary, so the real line on this runner is
|
||||
# ESC[2m Tests ESC[22m ESC[1mESC[32m97 passedESC[39m ... — ANSI escapes
|
||||
# sit between "Tests " and the digits, so the two-spaces-then-digit
|
||||
# pattern can never match. The first grep still matched "skipped", so
|
||||
# the guard fired on ANY run containing a single skipped test while
|
||||
# printing "integration suites skipped instead of running" — a false
|
||||
# statement. Measured 2026-09-08, run 13911 job 31183: 13 files / 97
|
||||
# tests PASSED, 1 file / 2 tests skipped → FATAL anyway.
|
||||
#
|
||||
# It survived review because it is correct locally: redirecting to a
|
||||
# file disables vitest's colour, so the pattern matches on a developer
|
||||
# machine and only misfires under CI.
|
||||
#
|
||||
# Replaced with the JSON reporter — structured and colourless. A/B
|
||||
# measured on real reports, BOTH directions:
|
||||
# all-skipped (0 passed / 99 pending) → RED (protection preserved)
|
||||
# 945 passed + 184 skipped → GREEN (false red removed)
|
||||
- name: Integration tests against LIVE OpenSearch + ClickHouse (blocking)
|
||||
run: |
|
||||
set -eu
|
||||
# Guard: env MUST be present here — the suites self-skip without it,
|
||||
# and a skipped suite must never masquerade as a green integration
|
||||
# gate (Master-DoD §56 status honesty).
|
||||
# Env MUST be present here — the suites self-skip without it, and a
|
||||
# skipped suite must never masquerade as a green integration gate
|
||||
# (Master-DoD §56 status honesty).
|
||||
[ -n "${OPENSEARCH_URL:-}" ] && [ -n "${CLICKHOUSE_URL:-}" ]
|
||||
pnpm vitest run tests/integration 2>&1 | tee /tmp/itest.log
|
||||
if grep -qE "skipped" /tmp/itest.log && ! grep -qE "Tests [0-9]+ passed" /tmp/itest.log; then
|
||||
echo "FATAL: integration suites skipped instead of running"; exit 1
|
||||
fi
|
||||
pnpm vitest run tests/integration \
|
||||
--reporter=default \
|
||||
--reporter=json --outputFile.json=/tmp/itest.json
|
||||
node -e '
|
||||
const fs = require("fs");
|
||||
const p = "/tmp/itest.json";
|
||||
if (!fs.existsSync(p)) {
|
||||
console.error("FATAL: vitest produced no JSON report"); process.exit(1);
|
||||
}
|
||||
const r = JSON.parse(fs.readFileSync(p, "utf8"));
|
||||
console.log(`integration: ${r.numPassedTests} passed, ${r.numPendingTests} skipped, ${r.numFailedTests} failed, ${r.numTotalTests} total`);
|
||||
if (r.numFailedTests > 0) {
|
||||
console.error("FATAL: integration tests failed"); process.exit(1);
|
||||
}
|
||||
if (r.numPassedTests === 0) {
|
||||
console.error("FATAL: no integration test actually executed — every suite skipped");
|
||||
process.exit(1);
|
||||
}
|
||||
'
|
||||
|
||||
# BLOCKING. This job publishes the protected-branch status check context
|
||||
# "CI (SIAX Cloud) / security (push)", which branch protection on `main`
|
||||
|
||||
Reference in New Issue
Block a user