feat(api): agent-substrate quick wins: worked examples in the spec, honest Retry-After, and a payload guard that covers the namespace new installs get (#1974)
* feat(api): surface the registry's worked examples in the OpenAPI spec and generated skill EndpointDefinition.example is required and every one of the 125 v1 endpoints populates example.response, but generateOpenApiSpec() never emitted it. The examples reached only the docs markdown builder, so /api/v1/openapi.json carried none and the generated skills/accounted-api had zero json blocks in all 12 reference files: every agent reading the spec or installing the skill got schemas with no concrete body. Emit example on the application/json media types (request body and 200 response) and teach the portable renderOperationMd to print it as a fenced json block. 178 worked examples now reach the skill. SKILL.md is unchanged: the examples land in the on-demand reference files, not the entry file. Attached to JSON media types only, so a multipart body and a binary application/pdf response do not advertise an example they cannot send. Adds the one missing example.request (currency-revaluation) so the new exhaustive coverage assertions hold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(api): emit Retry-After on a v1 429 so the documented contract is real The published accounted-api skill has told agents to honor Retry-After on a 429 since it shipped, but no /api/v1 route ever sent one: the wrapper's auth failure path early-returns through v1ErrorResponseFromCode, whose finalize() set only X-Request-Id and Gnubok-Version. Unattended clients had nothing to pace against and had to back off blindly. 60 seconds is an exact upper bound rather than a guess: the rate limiter is a fixed one-minute tumbling window per key row and the limited branch does not slide it. The value moves into an exported constant next to that limiter, so the MCP server's hardcoded '60' now reads from the same place. Also corrects the withApiV1 doc comment, which claimed step 8 stamps X-RateLimit-Limit. It never did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(mcp): guard the tools/list payload for the namespace new installs get The payload ratchet only ever serialized the gnubok_* projection. The accounted_* projection is inherently larger (every tool reference gains 3 chars, ~209 tokens across the default catalog) and CLAUDE.md points new MCP installs at exactly that namespace, so the payload a new user's client receives was never measured. It had already drifted ~90 tokens past the 63.4K ceiling while the guarded number sat comfortably under it. Measure both and assert on the larger. The ceiling moves to 63.6K to cover the real worst case; this buys no new catalog surface. A second test pins the direction of the delta so Math.max cannot silently stop describing reality. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
Jakob Wennberg
parent
c0ecb34a2b
commit
3447da027a
@@ -1,26 +1,47 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import { tools, deriveToolMeta, isDefaultCatalogTool } from '../server'
|
||||
import { projectToolInputSchema } from '../company-routing'
|
||||
import { projectToolReferences } from '../tool-namespace'
|
||||
|
||||
// Mirror the real tools/list serializer, including the derived staging _meta
|
||||
// (requires_approval / approve_tool / preflight) merged over any literal
|
||||
// _meta: otherwise the guard under-measures the wire payload.
|
||||
const canonicalToolNames = new Set(tools.map((t) => t.name))
|
||||
|
||||
function serializeCatalog(namespace: 'gnubok' | 'accounted'): string {
|
||||
const projection = tools.filter(isDefaultCatalogTool).map((t) => {
|
||||
const meta = { ...(deriveToolMeta(t) ?? {}), ...(t._meta ?? {}) }
|
||||
const projected = {
|
||||
name: t.name,
|
||||
...(t.title ? { title: t.title } : {}),
|
||||
description: t.description,
|
||||
inputSchema: projectToolInputSchema(t),
|
||||
...(t.outputSchema ? { outputSchema: t.outputSchema } : {}),
|
||||
annotations: t.annotations,
|
||||
...(Object.keys(meta).length > 0 ? { _meta: meta } : {}),
|
||||
}
|
||||
return namespace === 'accounted'
|
||||
? projectToolReferences(projected, namespace, canonicalToolNames)
|
||||
: projected
|
||||
})
|
||||
return JSON.stringify({ tools: projection })
|
||||
}
|
||||
|
||||
const tokensFor = (namespace: 'gnubok' | 'accounted') =>
|
||||
Math.round(serializeCatalog(namespace).length / 4)
|
||||
|
||||
describe('tools/list payload size guard', () => {
|
||||
it('keeps the projected tools/list payload under the context-budget ceiling', () => {
|
||||
// Mirror the real tools/list serializer, including the derived staging
|
||||
// _meta (requires_approval / approve_tool / preflight) merged over any
|
||||
// literal _meta: otherwise the guard under-measures the wire payload.
|
||||
const projection = tools.filter(isDefaultCatalogTool).map((t) => {
|
||||
const meta = { ...(deriveToolMeta(t) ?? {}), ...(t._meta ?? {}) }
|
||||
return {
|
||||
name: t.name,
|
||||
...(t.title ? { title: t.title } : {}),
|
||||
description: t.description,
|
||||
inputSchema: projectToolInputSchema(t),
|
||||
...(t.outputSchema ? { outputSchema: t.outputSchema } : {}),
|
||||
annotations: t.annotations,
|
||||
...(Object.keys(meta).length > 0 ? { _meta: meta } : {}),
|
||||
}
|
||||
})
|
||||
const payload = JSON.stringify({ tools: projection })
|
||||
const approxTokens = Math.round(payload.length / 4)
|
||||
// Measure BOTH namespaces and guard the larger.
|
||||
//
|
||||
// `?tool_namespace=accounted` rewrites every gnubok_* reference to
|
||||
// accounted_* (+3 chars each), so the accounted projection is inherently
|
||||
// ~200 tokens larger than the gnubok one. CLAUDE.md points new MCP
|
||||
// installs at the accounted-mcp package and the accounted_* aliases, so
|
||||
// that larger payload is what a NEW user's client actually receives,
|
||||
// while this guard measured only the legacy namespace and let the real
|
||||
// worst case drift untested.
|
||||
const approxTokens = Math.max(tokensFor('gnubok'), tokensFor('accounted'))
|
||||
// Ceiling progression: 20K to 25K to 30K to 31K to 31.5K to 32K to 36K.
|
||||
// * 20K → 25K when item 8 of the agent-native API plan landed
|
||||
// (additionalProperties: false on all inputSchemas + period_status in the
|
||||
@@ -242,9 +263,25 @@ describe('tools/list payload size guard', () => {
|
||||
// catalog: the onboarding efterkontroll instructs matching bank rows
|
||||
// against SIE verifikat, and a search-only tool is uncallable on
|
||||
// Claude.ai (E2E #12 punted to the web app over it).
|
||||
// * 63.4K to 63.6K with NO new tool: the guard started measuring the
|
||||
// accounted_* namespace as well as gnubok_* and asserting on the
|
||||
// larger. The accounted projection was already ~90 tokens over the
|
||||
// 63.4K line (the namespace rewrite costs ~209 tokens on its own) and
|
||||
// nothing tested it, so this bump buys no new catalog surface. It
|
||||
// re-points an existing ceiling at the payload new installs actually
|
||||
// receive; the gnubok projection still sits ~320 tokens under it.
|
||||
// Long-term answer to growth is leaning harder on gnubok_search_tools: if this
|
||||
// fires again, prefer trimming descriptions or making a tool opt-in via search
|
||||
// before bumping further.
|
||||
expect(approxTokens).toBeLessThan(63_400)
|
||||
expect(approxTokens).toBeLessThan(63_600)
|
||||
})
|
||||
|
||||
it('keeps the accounted_* namespace as the measured worst case', () => {
|
||||
// Pins the DIRECTION of the delta, not its size. If a future change ever
|
||||
// made gnubok_* the larger projection, the Math.max above would silently
|
||||
// keep passing while this guard stopped describing reality.
|
||||
expect(serializeCatalog('accounted').length).toBeGreaterThan(
|
||||
serializeCatalog('gnubok').length,
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
validateApiKey,
|
||||
createServiceClientNoCookies,
|
||||
hasScope,
|
||||
RATE_LIMIT_RETRY_AFTER_SECONDS,
|
||||
TOOL_SCOPE_MAP,
|
||||
type ApiKeyMode,
|
||||
type ApiKeyScope,
|
||||
@@ -19384,7 +19385,10 @@ export async function handleMcpRequest(request: Request): Promise<Response> {
|
||||
if (status === 429) {
|
||||
return new Response(authResult.error, {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'text/plain', 'Retry-After': '60' },
|
||||
headers: {
|
||||
'Content-Type': 'text/plain',
|
||||
'Retry-After': String(RATE_LIMIT_RETRY_AFTER_SECONDS),
|
||||
},
|
||||
})
|
||||
}
|
||||
return unauthorized()
|
||||
|
||||
Reference in New Issue
Block a user