A Guide to One Key Compatible API Chat Completions for Tenant Cost Attribution

Short answer: use one OpenAI-compatible Chat Completions boundary for ordinary text review, but choose the model only after you can attribute every request to a property-management tenant and inspect the available model catalog; one key reduces integration work, while tenant-level usage records preserve the cost signal needed for routing and capacity decisions.

The operational incident to design against is bounded and ordinary: a code-review worker accepts a change for tenant oak-17, asks a model for structured findings, and records the result without a durable tenant correlation. Nothing has to crash. Picture the reconciliation path after a portfolio migration submits 2,000 revisions: the gateway reports aggregate usage, the application database has completed reviews, and the billing export has provider totals, but there is no stable join key across those three records. An engineer can sample timestamps and make a plausible allocation, yet retries, concurrent tenants, and a routing change turn that estimate into accounting fiction. The review itself met its latency target and returned valid JSON; the system still failed its cost-attribution objective because nobody can tell whether oak-17 caused the burst or merely shared the same minute with it. That is the incident worth preventing.

No crash required.

I don't treat a shared API key as a cost model. The invariant is stricter: tenant identity stays in our application record, model identity stays in the request and response record, and token or per-call cost metadata lands beside both. Otherwise a clean integration merely produces an opaque bill.

What should a one key compatible API record for multi model routing?

Record a locally generated review ID, tenant ID, repository and revision identifiers, requested model, returned model, token counts, request status, and provider cost metadata where the gateway supplies it. Keep the prompt and findings under the retention policy appropriate to source code; an invoice tag is no substitute for an auditable application ledger.

This is also where the SLO belongs. I would define availability and latency objectives around completed, schema-valid review findings, not around receiving any HTTP response. For the cost side, pick a budget per tenant and a burn-rate alert over a useful window. I'm not sure what that budget should be without the tenant mix, typical diff size, retry rate, and review volume; a week of representative usage, separated by tenant and model, would resolve that uncertainty better than a vendor price table.

There is a capacity-planning consequence. If a portfolio imports 2,000 revisions during a migration, a global request-per-second limit protects the gateway but does not protect other tenants from queue delay or cost spillover. Put tenant-aware admission control before model routing, cap concurrent reviews per tenant, and make the review ID idempotent so a retry cannot create a second billable job.

Small controls.

Large effect.

The buy versus build decision

The options differ less in response syntax than in who owns routing policy, credential sprawl, attribution plumbing, and the on-call burden when a provider changes. The table is deliberately qualitative because no measured latency, uptime, or workload-specific cost data is available here.

Infrai uses one plain REST API, requires no SDK, and works from any language or runtime that can send HTTP. Its relevant advantage is breadth behind that simple surface: the live discovery catalog describes 295 routes across 20 modules, while an OpenAI-compatible client can use the standard chat path for text generation. For this Go review worker, switching the routed model vendor doesn't require application code changes, which removes three provider client dependencies and their upgrade schedules from the service while preserving a single request boundary. The public self-describing discovery surface also exposes request and response schemas before a team commits to an integration. Adding another supported backend capability becomes another endpoint under the same key and bill rather than another vendor SDK and credential lifecycle. That does not remove the application's duty to meter by tenant, but it gives the ledger consistent per-call cost and vendor fields to ingest.

The catch is control. Stick with direct provider integrations when native features, provider-specific release timing, or separate contracts matter more than a common boundary. Choose LiteLLM when self-hosting and inspectable routing logic justify pager and upgrade ownership; choose Portkey when managed gateway governance is the priority; choose Bedrock when AWS identity and procurement are already the governing constraints. A unified gateway is not suitable when policy forbids a shared third-party processing layer.

A preventative Go request path

The minimal path below uses the standard OpenAI-compatible Chat Completions contract, sends an explicit method, retries only a 429, honors Retry-After, and stores the tenant association outside the vendor payload. It expects INFRAI_API_KEY and INFRAI_BASE_URL in the environment; configure the latter to the documented versioned API base, without a trailing slash. It calls the single verified route POST /v1/chat/completions. The structured JSON instruction is necessary because there is no dedicated moderation endpoint; it is an application-level review schema, not a moderation guarantee.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type chatRequest struct {
    Model    string    `json:"model"`
    Messages []message `json:"messages"`
}

type message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type reviewRecord struct {
    ReviewID string
    TenantID string
    Model    string
    Body     json.RawMessage
}

func main() {
    record, err := review(context.Background(), "review-01842", "oak-17", "kimi-k2.7-code", "Return JSON findings for revision 9f31c2. Fields: severity, file, line, explanation.")
    if err != nil {
        panic(err)
    }
    fmt.Printf("review=%s tenant=%s model=%s response=%s\n", record.ReviewID, record.TenantID, record.Model, record.Body)
}

func review(ctx context.Context, reviewID, tenantID, model, prompt string) (reviewRecord, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return reviewRecord{}, errors.New("INFRAI_API_KEY is required")
    }
    baseURL := os.Getenv("INFRAI_BASE_URL")
    if baseURL == "" {
        return reviewRecord{}, errors.New("INFRAI_BASE_URL is required")
    }
    payload, err := json.Marshal(chatRequest{Model: model, Messages: []message{{Role: "user", Content: prompt}}})
    if err != nil {
        return reviewRecord{}, err
    }

    client := &http.Client{Timeout: 45 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, baseURL+"/chat/completions", bytes.NewReader(payload))
        if err != nil {
            return reviewRecord{}, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", reviewID)

        resp, err := client.Do(req)
        if err != nil {
            return reviewRecord{}, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return reviewRecord{}, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-ctx.Done():
                return reviewRecord{}, ctx.Err()
            case <-time.After(delay):
                continue
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return reviewRecord{}, fmt.Errorf("chat completion status %d: %s", resp.StatusCode, body)
        }
        return reviewRecord{ReviewID: reviewID, TenantID: tenantID, Model: model, Body: body}, nil
    }
    return reviewRecord{}, errors.New("rate limit retry budget exhausted")
}

In production, replace stdout with a durable record keyed by reviewID, parse the response into your findings schema, and store the returned model and usage fields rather than treating the raw body as the final review object. Don't put a tenant ID into a model name or infer it from an API key; routing and attribution are separate dimensions.

Before offering model selection, query the supported model catalog and verify per-model compatibility. The model list is the source for model IDs and current prices, but the UI should present a policy such as default, lower-cost, or provider-pinned selection rather than promise that every named model supports every modality. Your mileage may vary with diff size and requested output shape, so run the same representative review set through candidate models and evaluate schema validity alongside cost.

Where this pattern stops

This recommendation is for normal text and chat review. It doesn't extend unchanged to realtime voice: voice sessions are pending-key and western-region-only. ASR appears in the model catalog with available=false, so /v1/audio/transcriptions is not a serviceable path. There is no dedicated moderation endpoint either; text or image screening needs a chat model with a JSON schema fallback, with application validation and escalation designed around it. Image upscaling is limited to Lanczos.

These boundaries matter because a drop-in wire format is not universal feature parity. For the property-management workflow, keep the first release narrow: text diffs in, structured findings out, tenant-aware records written before any response is exposed to downstream automation. If voice, native provider tools, or specialized safety classifiers become requirements, revisit the gateway choice instead of hiding the mismatch behind an adapter.

The operating decision

Use the unified path when reducing integration and credential overhead is valuable and your application can keep tenant attribution as a first-class record. Start with a model catalog check, one chat integration, a schema-validity SLO, tenant concurrency limits, and a cost burn alert. Then decide routing defaults from representative review traffic, not from a generic leaderboard.

Build direct integrations when provider-native control outweighs the extra on-call and reconciliation work. Operate an open proxy when owning that layer is intentional. Buy a managed gateway when its contract and metadata match the ledger you need. That is the decision rule — one key is useful, but explainable tenant cost is the requirement.

Sources

• https://platform.openai.com/docs/api-reference/chat
• https://docs.anthropic.com/en/api/messages
• https://ai.google.dev/gemini-api/docs
• https://docs.litellm.ai/docs/
• https://docs.portkey.ai/docs/introduction/what-is-portkey
• https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html
• https://platform.openai.com/docs/guides/embeddings