Put a JSON Contract in Front of a Free Model Route Before Your CI Ever Calls It

07:42. The GitLab job fails at risk not found. I open the raw log. The model returned { "output": "low" }. My parser expected { "summary": "...", "risk": "low" }.

No code changed. No pipeline change. The model endpoint changed.

A free model route is cheap. It is also an interface that can drift. The fix isn't another retry or a string slice. It's a gate between the model and the caller.

I'm going to show a small bouncer that sits in front of a model route and fails closed before GitLab CI ever parses a bad shape.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I rely on MonkeyCode's free model access and free server option as the environment for this pattern.

The failure mode I keep seeing

Free model routes change in three ways:

• Field names: output becomes summary.
• Enum values: "LOW" replaces "low".
• Response size: verbose output eats the job's timeout.

A CI job that calls the endpoint directly fails late. Worse, it can pass once and fail next run. You then spend ten minutes on the model, not the code.

My rule is simple: no downstream code may trust a free model response directly. Trust has to be checked.

Three things that didn't work

I tried:

1. A retry on parse failure. It just repeats the wrong shape.
2. A || 'low' fallback. It hides real drift.
3. A validator inside CI after the model call. It still ran the model and spent pipeline time before failing.

Retries and fallbacks treat a broken contract as an occasional glitch. Drift is not noise. It's a signal.

What the contract gateway does

The gateway sits between the caller and the model route:

1. Accept the same POST payload the pipeline would send.
2. Forward it to the model endpoint with a timeout.
3. Enforce a response size cap.
4. Require JSON.
5. Enforce required fields, allowed fields, types, and enum values.
6. Return 200 on a valid shape, or 502 with a named problem list.

On mismatch, it logs a one-line sample: latency, problem list, and the first 200 characters of the response. It does not log the prompt body.

The gateway is independent of the provider. It does not care whether the endpoint is MonkeyCode, local tooling, or another free route. That is the point.

The minimal artifact

I keep it dependency-light so it runs on a small free server. The contract is a plain module.

contract.mjs

export const contract = {
  required: ['summary', 'risk'],
  allowed: ['summary', 'risk'],
  properties: {
    summary: { type: 'string', min: 10, max: 800 },
    risk: { values: ['low', 'medium', 'high'] },
  },
};

check-payload.mjs

import { contract } from './contract.mjs';

export function checkPayload(payload) {
  const problems = [];

  for (const key of contract.required) {
    if (!(key in payload)) problems.push(`missing ${key}`);
  }

  for (const key of Object.keys(payload)) {
    if (!contract.allowed.includes(key)) problems.push(`unexpected ${key}`);
  }

  const summary = payload.summary;
  if (summary !== undefined) {
    if (typeof summary !== 'string') {
      problems.push('summary must be a string');
    } else if (
      summary.length < contract.properties.summary.min ||
      summary.length > contract.properties.summary.max
    ) {
      problems.push('summary length out of range');
    }
  }

  const risk = payload.risk;
  if (risk !== undefined && !contract.properties.risk.values.includes(risk)) {
    problems.push('risk has an unknown value');
  }

  return problems;
}

gateway.mjs

import http from 'node:http';
import { checkPayload } from './check-payload.mjs';

const upstream = process.env.MODEL_URL;
const PORT = process.env.PORT ?? 3210;
const MAX_LATENCY_MS = Number(process.env.MAX_LATENCY_MS ?? 12000);
const MAX_BODY_CHARS = Number(process.env.MAX_BODY_CHARS ?? 2000);
const MAX_PROMPT_CHARS = Number(process.env.MAX_PROMPT_CHARS ?? 8000);

const server = http.createServer(async (req, res) => {
  if (req.method !== 'POST') {
    res.writeHead(405).end('POST only');
    return;
  }

  let body = '';
  for await (const chunk of req) body += chunk;
  if (body.length > MAX_PROMPT_CHARS) {
    res.writeHead(413).end('request too large');
    return;
  }

  const started = Date.now();
  let upstreamRes;
  try {
    upstreamRes = await fetch(upstream, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body,
      signal: AbortSignal.timeout(MAX_LATENCY_MS),
    });
  } catch (err) {
    res.writeHead(504).end(`upstream unavailable: ${err.name}`);
    return;
  }

  const raw = await upstreamRes.text();
  if (raw.length > MAX_BODY_CHARS) {
    res.writeHead(502).end('model response too large');
    return;
  }

  let payload;
  try {
    payload = JSON.parse(raw);
  } catch {
    res.writeHead(502).end('model response is not JSON');
    return;
  }

  const problems = checkPayload(payload);
  const elapsedMs = Date.now() - started;

  if (problems.length > 0) {
    console.log(JSON.stringify({
      elapsed_ms: elapsedMs,
      problems,
      sample: raw.slice(0, 200),
    }));
    res.writeHead(502, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ ok: false, error: 'contract_mismatch', problems }));
    return;
  }

  res.writeHead(200, { 'content-type': 'application/json' });
  res.end(JSON.stringify({ ok: true, elapsed_ms: elapsedMs, payload }));
});

server.listen(PORT, () => {
  console.log(`contract gateway on :${PORT}`);
});

The bouncer is intentionally boring. It is not a reviewer. It only asks: "Does this response match the shape the code already expects?"

Test the failure you are trying to catch

The happy path proves almost nothing. The useful tests simulate a silent reshape.

gateway.test.mjs

import test from 'node:test';
import assert from 'node:assert/strict';
import { checkPayload } from './check-payload.mjs';

test('accepts the expected shape', () => {
  const payload = {
    summary: 'All tests passed and coverage is stable.',
    risk: 'low',
  };
  assert.deepEqual(checkPayload(payload), []);
});

test('rejects a renamed field', () => {
  const payload = {
    output: 'All tests passed and coverage is stable.',
    risk: 'low',
  };
  const problems = checkPayload(payload);
  assert.ok(problems.includes('missing summary'));
  assert.ok(problems.includes('unexpected output'));
});

test('rejects a disguised enum value', () => {
  const payload = {
    summary: 'All tests passed and coverage is stable.',
    risk: 'LOW',
  };
  assert.ok(checkPayload(payload).includes('risk has an unknown value'));
});

Run with Node's built-in test runner:

node --test gateway.test.mjs

Why run it on a free server instead of inside CI

If the bouncer runs only inside a pipeline job, it has several limits:

• It runs only when the pipeline runs.
• It can't be shared across repos.
• It can't be called from a local editor.
• It doesn't catch drift between pipeline runs.

A small free server keeps the gate always available. The GitLab CI job then points at the bouncer instead of the raw model route.

model_check:
  image: node:20
  script:
    - >-
      curl --fail-with-body -sS
      -H 'content-type: application/json'
      -d '{"text":"review this change"}'
      http://gateway:3210/analyze

The path doesn't matter to the bouncer. Any POST is checked the same way.

Decisions the contract can't make

A shape check is not an eval.

The model can return:

• A wrong summary that is still a string.
• A confident but incorrect risk label.
• A grammatically perfect answer that misses the point.

The bouncer won't catch those. That's fine. It isn't supposed to. It removes one class of failure so a human or a second model can review the content.

When not to use this pattern

Skip it if:

• The model call is user-facing or latency-critical.
• The data is regulated, health, financial, or otherwise private.
• You need high availability and cannot tolerate a restarted free server.
• The output shape must change dynamically per request.

This is a development-time tool for CI jobs, issue triage, test generation, and free-tier experiments. It is not a production API gateway.

Also treat every free endpoint as public. The gateway avoids logging prompt bodies, but the upstream provider may keep its own logs. Do not send secrets or unreviewed private data.

The workflow I now follow

1. Pin the contract before the first model call.
2. Add the happy path and two failure-path tests.
3. Start the gateway on a free server.
4. Point the CI job at the gateway, not the raw model.
5. When a 502 appears, read the problem list and update the contract only if the change was intentional.

The last step is the important one. The gateway never tells me what to do. It tells me something moved.

I've written before about readiness contracts for free model endpoints in CI. This bouncer is the runtime side of that same idea. A readiness check asks "is the endpoint alive?" The bouncer asks "does the response match what the code expects?"

If you already have a free model route, copy the bouncer and run those three tests before your next merge. You'll stop debugging field names and start seeing drift as a fast, normal failure.