Overview

Guardrails

Guardrails are governance policies bound to API keys. When an API key is created, the system copies the policy into a snapshot. When a model request enters api-gateway, the gateway validates against that snapshot instead of reading the latest policy each time.

This means editing a policy may not change behavior for existing keys. To apply a new policy, create a new key, validate it, then revoke the old key.

Enforced Constraints

The current api-gateway enforces these constraints on POST /openai/v1/chat/completions:

GuardrailCurrent behavior
blocked / block_reasonRejects the request when the snapshot is blocked
ip_blacklistRejects when the client IP matches the blacklist
ip_whitelistWhen configured, the client IP must match an IP or CIDR
tiers / allowed_tiersNormalized to economy / standard / premium and intersected with API key tier and request tier
model_blacklistRejects when the selected model or fixed model is blacklisted
fixed_modelWhen an API key has a fixed model, the request model must be auto or that model
routing_strategyPassed to the router; supports BALANCE, COST_FIRST, QUALITY_FIRST, SPEED_FIRST
tagPassed to the router as a lowercased scenario tag
industry_packsPassed to the router as industry scenario packs
required_feature_tagsAdded by the gateway from request content; image input requires vision

Execution Order

extract API key
  -> ValidateApiKey
  -> parse policy_snapshot_json
  -> blocked / IP checks
  -> tier intersection
  -> fixed model / model blacklist
  -> wallet balance precheck
  -> Route
  -> wallet freeze
  -> provider invoke
  -> usage record and wallet settle

Tier Constraints

The final allowed tier set is built from multiple layers:

  1. allowed_tiers from the policy snapshot, or tiers when allowed_tiers is empty.
  2. The API key’s own tier.
  3. The request body’s tier.
  4. Wallet balance checks remove tiers with zero balance.

If no tier remains, the request fails. For model: "auto", if the selected tier cannot be frozen due to insufficient wallet balance, the gateway can retry routing to another allowed tier.

IP Policy

The gateway resolves client IP from:

SourceNotes
X-Forwarded-ForUses the first IP
X-Real-IPCommon real-IP header from reverse proxies
remote addressUsed when proxy headers are absent

Rules support exact IPs and CIDR ranges:

IP policy
1{
2  "ip_whitelist": ["203.0.113.10", "10.0.0.0/24"],
3  "ip_blacklist": ["198.51.100.23"]
4}

Model Policy

PolicyBehavior
fixed_modelForces routing to one model. auto is constrained to that model; other concrete models are rejected
model_blacklistPrevents blacklisted models from being selected; fixed models are checked too
required_feature_tagsDerived from request content; image input must select a vision model

Budget and Wallet

API key validation returns limit_total, limit_daily, and limit_weekly. Policies can also include daily_limit / weekly_limit. In the current gateway path, wallet balance and freeze are the direct enforcement points:

Enforcement pointCurrent behavior
filterPolicyTiersByBalanceRemoves tiers with zero balance
freezeSelectedModelEstimates and freezes credits from model rates and token estimates
billing-service CalculateComputes actual credits after provider success
wallet-service Settle / ReleaseSettles on successful calculation and usage record; releases on failure, empty amount, or zero amount
ScenarioRecommendation
Local developmentSeparate key, low budget, relaxed IP, BALANCE or COST_FIRST
Production backendIP allowlist, explicit tier, model blacklist, budget limits
IDE / Coding agentSeparate key, low daily/weekly limit, prefer COST_FIRST
RAG indexingSeparate key, fixed embedding or batch model once those endpoints are public
High-risk agentRuntime stop conditions plus JoyToken key budget

Common Rejections

Error typeCommon cause
policy_rejectedIP, tier, fixed model, model blacklist, or blocked policy failed
insufficient_quotaWallet balance, budget, or freeze check failed
invalid_api_keyKey is invalid, revoked, expired, or not ACTIVE