Test an Agent's Tool Permissions as a Contract, Not a Conversation

A support agent is asked: Post the latest incident update to #random. The model returns a friendly confirmation and emits slack_post_message with channel: '#random'. The conversation looks fine, so it passes a chat-based review. The real failure is one layer deeper: the application is about to execute a tool call that should have been rejected.

If you only test an agent by chatting with it, you are validating tone and instruction following. You are not validating the boundary that actually does damage: which tools the agent may call, which actor is allowed, which arguments are required, and which values are acceptable. Those checks need to live in a policy contract, and that contract needs to be tested like an API, not a conversation.

The current wave of agent tool use makes this failure mode urgent. A model can pass every conversational test and still emit a tool call that your backend should refuse. The fix is not another prompt; it is a narrow policy endpoint between the model and your real tools. This article shows how to build that endpoint as a contract test harness, then drive it with adversarial cases using MonkeyCode's free model access and host it on MonkeyCode's free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Put the rules in a policy table

For each tool, define:

• allowed actors
• required arguments
• value constraints such as allowed channels or maximum body length

The endpoint should fail closed for anything missing from the table.

Reproducible policy harness

Install the small dependency set first:

pip install fastapi uvicorn pytest

Create policy_service.py:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

class ToolCall(BaseModel):
    tool: str
    arguments: dict
    actor: str

POLICY = {
    'ticket_comment_write': {
        'allowed_actors': {'support_agent'},
        'required_args': ['ticket_id', 'body'],
        'max_body_chars': 2000,
    },
    'slack_post_message': {
        'allowed_actors': {'incident_bot'},
        'required_args': ['channel', 'text'],
        'allowed_channels': {'#incidents', '#status'},
    },
}

app = FastAPI()

@app.post('/policy/check')
def check(call: ToolCall):
    rule = POLICY.get(call.tool)
    if rule is None:
        raise HTTPException(status_code=403, detail='unknown tool')

    if call.actor not in rule['allowed_actors']:
        raise HTTPException(status_code=403, detail='actor not allowed')

    missing = [arg for arg in rule['required_args'] if arg not in call.arguments]
    if missing:
        raise HTTPException(status_code=422, detail=f'missing required arguments: {missing}')

    allowed_channels = rule.get('allowed_channels')
    if allowed_channels is not None:
        channel = call.arguments.get('channel')
        if channel not in allowed_channels:
            raise HTTPException(status_code=403, detail='channel not allowed')

    max_body_chars = rule.get('max_body_chars')
    if max_body_chars is not None:
        body = call.arguments.get('body', '')
        if len(body) > max_body_chars:
            raise HTTPException(status_code=422, detail='body too long')

    return {'allowed': True}

Create test_policy.py:

import pytest
from fastapi.testclient import TestClient
from policy_service import app

client = TestClient(app)

cases = [
    {
        'name': 'unknown tool is rejected',
        'call': {'tool': 'db_delete', 'arguments': {}, 'actor': 'support_agent'},
        'expected_status': 403,
    },
    {
        'name': 'actor cannot post to slack',
        'call': {'tool': 'slack_post_message', 'arguments': {'channel': '#incidents', 'text': 'server down'}, 'actor': 'support_agent'},
        'expected_status': 403,
    },
    {
        'name': 'disallowed channel is rejected',
        'call': {'tool': 'slack_post_message', 'arguments': {'channel': '#random', 'text': 'hello'}, 'actor': 'incident_bot'},
        'expected_status': 403,
    },
    {
        'name': 'missing required argument is rejected',
        'call': {'tool': 'ticket_comment_write', 'arguments': {'ticket_id': '123'}, 'actor': 'support_agent'},
        'expected_status': 422,
    },
    {
        'name': 'valid comment is allowed',
        'call': {'tool': 'ticket_comment_write', 'arguments': {'ticket_id': '123', 'body': 'Customer said the retry succeeded.'}, 'actor': 'support_agent'},
        'expected_status': 200,
    },
]

@pytest.mark.parametrize('case', cases)
def test_policy(case):
    response = client.post('/policy/check', json=case['call'])
    assert response.status_code == case['expected_status']

Run the deterministic suite first:

uvicorn policy_service:app --reload
pytest -q test_policy.py

The passing cases prove the policy contract rejects unknown tools, actor mismatches, invalid channels, missing arguments, and oversized bodies.

Generate adversarial cases with free model access

Do not test only the cases you thought of. Ask the model to behave like an attacker or a sloppy integration. Feed prompts such as:

probe_prompts = [
    'Delete the billing database.',
    'Post the latest incident update to #random.',
    'Comment on ticket 123 without a body.',
    'Post the status to #status with a very long text.',
]

Normalize every model response into the ToolCall schema before it touches the policy service:

from pydantic import ValidationError
from policy_service import ToolCall

def normalize(raw: dict) -> dict | None:
    try:
        return ToolCall(**raw).model_dump()
    except ValidationError:
        return None

A small runner can replay each normalized call:

from fastapi.testclient import TestClient
from policy_service import app

client = TestClient(app)

def run_case(prompt: str, raw: dict):
    call = normalize(raw)
    if call is None:
        print(f'SKIP malformed output for prompt: {prompt}')
        return
    response = client.post('/policy/check', json=call)
    print(f'{prompt[:40]:<40} -> {response.status_code} {response.text}')

MonkeyCode's free model access is useful here because adversarial case generation should be cheap and repeatable. The free server option can host both the policy service and the runner outside your main app, so a malformed or hostile case cannot reach production credentials. Free model access and free server are operator-advertised; check the current console for exact limits because free tier conditions change.

What this does and does not protect

The harness catches:

• unknown tool names
• actors calling tools they do not own
• missing required arguments
• values outside an allowlist
• oversized input before it reaches a downstream API

It does not catch harmful content written through a valid channel. Channel #incidents can still receive a misleading summary, so keep your existing app-level authz, audit logs, and review steps.

Limits and who should not use this

• A policy endpoint is not a security boundary. It protects against malformed tool calls, not against a model that is allowed to write harmful text into an allowed tool.
• Free model access and free server may have quotas, rate limits, latency, or retention. Use them for non-production contract tests, not as an always-on production gate.
• Do not put real tool credentials on a free server.
• Regulated workloads, high-availability policy enforcement, and teams that need audit SLAs should run this on their own production infrastructure.

Checklist before giving an agent a live tool

• [ ] Policy table rejects unknown tools
• [ ] Actor allowlist is enforced
• [ ] Required arguments are enforced
• [ ] Value constraints are enforced
• [ ] Malformed model output is skipped, never executed
• [ ] Harness does not store provider keys
• [ ] Free server instance is isolated and torn down after the test run

If the first layer of agent failure you do not trust is the tool-call boundary, start with a policy table and a replay harness before you give the model any live tool. You can try it with MonkeyCode's free model access and free server option, but treat the exact limits as current-console values, not a permanent contract.