Build a Budget Watchdog for a Free Model Endpoint Before It Goes Quiet

Why this is worth reading: a free model endpoint and a free server are easy to adopt, but the failure that actually wakes you up is not the first request. It is the 3 a.m. cron job that starts returning a 429, quietly falls back to garbage, and makes your dashboard lie. This guide gives you a small watchdog that treats quota exhaustion as a normal state, degrades to a cached answer, and writes a ledger you can read before you trust the word "free".

To make this concrete, I used MonkeyCode's free model access and free server option as the deployment target. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The broader MonkeyCode project is open source, but this guide treats the free options as deployment constraints, not as a review of the repository. The 30 million token allowance is the operator-supplied current number; treat quotas as moving targets and verify them in the console before you rely on them.

The useful part here is not another endpoint probe. You already have the canary harness and the disposable sandbox. This watchdog assumes the endpoint is connected and asks a different question: what happens when the free tokens run out during a normal Tuesday run?

Design for the edge, not the happy path

Your service only has two jobs. First, it must return a useful summary or a clearly labeled fallback instead of a raw upstream error. Second, it must record which path it took so you can decide whether the free tier is still worth it. If you only instrument the success path, you will discover the quota limit from a user.

A 70-line budget watchdog

The watchdog has three parts: an isolated model call, an in-memory cache, and a JSONL ledger. The most important branch is the catch block, because that is where the free tier actually shows up.

import { createServer } from 'node:http';
import { readFile, writeFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';

const MODEL_URL = process.env.MONKEYCODE_MODEL_URL ?? '';
const API_KEY = process.env.MONKEYCODE_API_KEY ?? '';
const LEDGER = process.env.LEDGER_FILE ?? './ledger.jsonl';
const cache = new Map<string, { at: number; text: string }>();

class QuotaError extends Error {
  constructor(public status: number) {
    super(`quota signal ${status}`);
  }
}

async function callModel(input: string): Promise<string> {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 8000);
  try {
    const res = await fetch(MODEL_URL, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ prompt: input, max_tokens: 120 }),
      signal: controller.signal,
    });
    if (res.status === 429 || res.status === 402) throw new QuotaError(res.status);
    if (!res.ok) throw new Error(`model HTTP ${res.status}`);
    const data = await res.json();
    return data.text ?? data.choices?.[0]?.text ?? '';
  } finally {
    clearTimeout(timer);
  }
}

function fallbackSummary(input: string): string {
  const cleaned = input.replace(/\s+/g, ' ').trim();
  return cleaned.length > 200 ? `${cleaned.slice(0, 200)}...` : cleaned;
}

async function appendLedger(record: Record<string, unknown>): Promise<void> {
  await writeFile(LEDGER, JSON.stringify(record) + '\n', { flag: 'a' });
}

const server = createServer(async (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  if (req.method !== 'POST' || req.url !== '/summarize') {
    res.statusCode = 404;
    res.end(JSON.stringify({ error: 'not found' }));
    return;
  }
  let body = '';
  for await (const chunk of req) body += chunk;
  let input: string;
  try {
    input = JSON.parse(body).input;
  } catch {
    res.statusCode = 400;
    res.end(JSON.stringify({ error: 'invalid json' }));
    return;
  }

  const key = createHash('sha1').update(input).digest('hex');
  const cached = cache.get(key);
  if (cached && Date.now() - cached.at < 24 * 60 * 60 * 1000) {
    await appendLedger({ ts: Date.now(), source: 'cache', key });
    res.end(JSON.stringify({ source: 'cache', text: cached.text }));
    return;
  }

  try {
    const text = await callModel(input);
    cache.set(key, { at: Date.now(), text });
    await appendLedger({ ts: Date.now(), source: 'model', key, tokens: 120 });
    res.end(JSON.stringify({ source: 'model', text }));
  } catch (error) {
    const text = fallbackSummary(input);
    cache.set(key, { at: Date.now(), text });
    await appendLedger({
      ts: Date.now(),
      source: 'degraded',
      key,
      reason: error instanceof QuotaError ? `quota_${error.status}` : 'timeout_or_error',
    });
    res.end(JSON.stringify({ source: 'degraded', text }));
  }
});

server.listen(Number(process.env.PORT ?? 3000));

Run it locally before you put it on a server. This snippet keeps the model call isolated in callModel, so when MonkeyCode's current request shape differs from the generic JSON here, you only replace that function.

npm init -y
npm i -D typescript tsx @types/node
npx tsx index.ts

Then send a request with a realistically long body:

curl -s -X POST http://localhost:3000/summarize \
  -H 'content-type: application/json' \
  -d '{"input":"paste the issue body here"}'

Run it on the free server without turning it into a pet

The free server option needs one exposed port and no database. Keep the cache in memory on purpose: when the server restarts, the cache resets, and the first request goes back to the model or the fallback. That is the behavior you want for a side project.

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY index.ts ./
EXPOSE 3000
CMD npx tsx index.ts

Push the directory to whatever deployment target your MonkeyCode server console gives you, then send a warm-up request and watch the ledger. Do not add a database yet: the point is to keep the service disposable enough that deleting it costs nothing.

Read the ledger before you trust the word "free"

Each request appends one JSON line. After a day, these two commands tell you whether the service is quietly degrading:

tail -n 20 ledger.jsonl
jq -s 'group_by(.source) | map({source: .[0].source, count: length})' ledger.jsonl

That table is the useful part. It turns a vague "free tier is flaky" feeling into a decision you can make from a log file.

Limits and who should skip this

• The 30 million token figure and the free server boundary can change. The watchdog does not make free capacity unlimited; it makes the limit visible.
• The cache lives in memory and resets on redeploy. If you need a longer-lived cache, use a file or object store, but check whether the free server allows persistent disk before you assume it does.
• callModel is intentionally generic; MonkeyCode's exact API shape may differ. Swap only that function and leave the degradation branch alone.
• This is not for production health checks, private data, or high-concurrency workloads.

If you need guaranteed latency or want a free tier to behave like a paid SLA, skip this. A stale-but-honest answer is fine for an internal dashboard, but it is not a production fix.

Try the watchdog against your current MonkeyCode free server and keep the ledger file. The ledger is the only part worth keeping when you move off the free tier, because it tells you when a paid key starts failing silently. If you have already hit a free-tier limit, what did the endpoint return: a 429, a 402, or a JSON error field? That detail determines the first branch in callModel, and I would like to know before I generalize it.