Qualify a Free AI Model and Server Offer With a 4-Probe Acceptance Harness

A 30-million-token allowance is not capacity. It is an experiment budget that only matters if two other numbers hold: the endpoint's rate-limit behavior and the team's cost to exit. Teams often test the model and the server as one object, then discover one half works while the other half creates a production dependency.

This article uses MonkeyCode's currently advertised free model access and free server option as the worked example. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source project, but the only product claims treated here are the free model access and free server option. The operator states that the current offer includes a free token allowance advertised as 30,000,000 tokens plus a free server option. Treat that as a time-limited claim to verify, not as a permanent capacity plan. Model names, hardware, duration, and other product details should be rechecked in the current documentation before any commitment.

The two promises fail at different speeds

A model API fails fast: it returns a bad schema, a wrong answer, or an HTTP error. A server promise fails slowly: it works during a pilot, then degrades under load, changes its rate-limit posture, or becomes expensive to leave after code has grown around it.

The practical workflow below separates those two promises into four probes. The goal is not to prove that MonkeyCode is good. The goal is to prove that a free offer can be tested, pinned, and exited without handing over the team's architecture first.

Probe 0: pin the offer identity

Before running a single request, write a short manifest into the repository. The manifest makes the experiment reproducible and prevents the team from arguing later about what was actually tested.

{
  "offer": "MonkeyCode free model + free server",
  "recorded_on": "2026-08-14",
  "endpoint": "https://<openai-compatible-endpoint>/v1",
  "model": "<model-name-from-current-docs>",
  "advertised_token_allowance": 30000000,
  "server_option": "free-server",
  "docs_url": "<link-to-current-offer-page>",
  "owner": "platform-lead",
  "expires": "2026-08-28"
}

If the model name, endpoint, token allowance, docs URL, owner, and expiry are not all present, the probe has not started.

Probe 1: capability baseline

The first probe checks whether the endpoint can perform three small, deterministic tasks from your own domain. Do not use marketing examples; use text from your own issue tracker, documentation, or support tickets.

import json
import os
import sys
import time
import urllib.error
import urllib.request
from concurrent.futures import ThreadPoolExecutor, as_completed

BASE = os.environ["PROBE_BASE_URL"].rstrip("/")
KEY = os.environ["PROBE_API_KEY"]
MODEL = os.environ["PROBE_MODEL"]

TASKS = {
    "summarize": "Summarize in 3 bullets:\n\n<insert a paragraph from your own docs>",
    "schema": "Return JSON with keys category, severity, reason. Input: <paste your ticket text>",
    "edge": "Explain a retry loop failure when every non-200 is treated as transient. No markdown.",
}

def chat(prompt, timeout=30.0):
    payload = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
    }).encode("utf-8")
    request = urllib.request.Request(
        f"{BASE}/chat/completions",
        data=payload,
        headers={
            "Authorization": f"Bearer {KEY}",
            "Content-Type": "application/json",
        },
        method="POST",
    )
    started = time.perf_counter()
    try:
        with urllib.request.urlopen(request, timeout=timeout) as response:
            body = response.read().decode("utf-8")
            data = json.loads(body)
            return {
                "status": response.status,
                "elapsed_s": round(time.perf_counter() - started, 2),
                "model": data.get("model"),
                "content": data["choices"][0]["message"]["content"],
                "headers": {k.lower(): v for k, v in response.headers.items()},
            }
    except urllib.error.HTTPError as error:
        return {
            "status": error.code,
            "elapsed_s": round(time.perf_counter() - started, 2),
            "headers": {k.lower(): v for k, v in error.headers.items()},
            "body": error.read().decode("utf-8", errors="replace")[:200],
        }

def run_capability():
    for name, prompt in TASKS.items():
        result = chat(prompt)
        has_text = isinstance(result.get("content"), str) and len(result["content"]) > 0
        print(f"{name}: status={result.get('status')} elapsed={result.get('elapsed_s')} model={result.get('model')} text={has_text}")

def run_rate_limit(parallel=8):
    with ThreadPoolExecutor(max_workers=parallel) as pool:
        futures = [pool.submit(chat, TASKS["summarize"]) for _ in range(parallel)]
        for future in as_completed(futures):
            result = future.result()
            retry_after = result.get("headers", {}).get("retry-after", "")
            print(f"status={result.get('status')} elapsed={result.get('elapsed_s')} retry_after={retry_after}")

if __name__ == "__main__":
    mode = sys.argv[1] if len(sys.argv) > 1 else "capability"
    if mode == "capability":
        run_capability()
    elif mode == "rate_limit":
        run_rate_limit()
    else:
        print("usage: python free_tier_probe.py capability|rate_limit")
        sys.exit(1)

Run it with environment variables, not with hardcoded secrets:

export PROBE_BASE_URL="https://<openai-compatible-endpoint>/v1"
export PROBE_API_KEY="<key>"
export PROBE_MODEL="<model-name-from-current-docs>"

python free_tier_probe.py capability

The pass condition is simple: all three tasks return non-empty output, and the JSON task returns a parseable shape. A model that can summarize but cannot follow a constrained schema should not pass the capability gate.

Probe 2: rate-limit envelope

A free server is usually shared. That is acceptable for a pilot; what matters is whether the server tells the client how to behave when the limit is reached.

python free_tier_probe.py rate_limit

With eight parallel requests, record three things:

• How many requests return 200.
• Whether a 429 response includes a Retry-After header.
• Whether the endpoint fails fast or hangs until the client timeout.

A missing Retry-After header is a signal that the client cannot build reliable backoff. That is an infrastructure problem, not a model-quality problem.

An example threshold, not an objective truth: at least seven of eight parallel requests should return 200. If more than two fail, or if a 429 arrives without Retry-After, do not build on the free server. These thresholds are conversation tools; the right values depend on the team's latency and throughput requirements.

Probe 3: reproducibility check

Run the same capability probe twice, fifteen minutes apart. Compare the model field and the response schema.

python free_tier_probe.py capability > run-a.log
sleep 900
python free_tier_probe.py capability > run-b.log

diff -u <(cut -d' ' -f1 run-a.log) <(cut -d' ' -f1 run-b.log)

A silent model change can alter behavior without changing the API. If the model field disappears or changes, the team needs a pinning policy before it builds a regression harness around that endpoint.

Probe 4: exit test

The most important probe is the one most teams skip. Revoke the key on the provider side, then run the probe again.

PROBE_API_KEY="revoked-key" \
PROBE_BASE_URL="https://<openai-compatible-endpoint>/v1" \
PROBE_MODEL="<model-name-from-current-docs>" \
python free_tier_probe.py capability

Expected result: the endpoint returns an authentication error, such as 401, and the client does not write partial output. If a revoked key still returns 200, or if the client caches and serves stale generations, the integration has an exit problem.

Decision table

This table is an example to adapt, not a universal verdict.

Break-even view

A free allowance should be measured in hours-to-exit, not token count. Suppose the advertised allowance is 30,000,000 tokens and the team's blended cost is $120 per engineer-hour. If an eight-hour failed integration costs the equivalent of the value of the free budget, the offer has not bought a cheaper build; it has bought a cheaper experiment. That can be useful, but only if the experiment is designed to end.

The formula is:

experiment_hours <= time-box_days × usable_engineer_hours_per_day

If the team cannot state both sides of that inequality, the free tier is driving the schedule instead of the decision criteria driving the pilot.

Who should not use this approach

This probe does not replace a security review, a legal review of data handling, or a production capacity plan. Teams that cannot send their domain text to a third-party server should not start the probe at all. Teams that need a contractual uptime guarantee should not treat a free server as a production control plane. Teams that cannot write a repository manifest should not consume a free API from shared infrastructure.

Exit criteria

The offer is worth moving past the probe only when all four probes pass and the team writes down the one condition that would reverse the decision. If no condition can reverse the decision, the team is not evaluating a server; it is looking for permission to adopt one.

If your team is evaluating MonkeyCode, run this probe against the endpoint and model listed in the current documentation rather than assuming the placeholders in this article still apply. The useful output is not an opinion about the product; it is a written pass/fail record that survives the next free-tier model release.