Rate Limits

The current project does not enforce a traditional fixed QPS rate limit as the primary limit. The implemented limit mechanisms are API key quota, policy constraints, and wallet balance precheck. Client docs should describe those real behaviors.

Current Enforcement

Wallet Fallback

When the request uses model: "auto" or omits model, JoyToken may try a different tier when the selected tier lacks balance. Current order:

If the request specifies a concrete model, the gateway does not automatically switch to another model because of insufficient balance.

Client Handling

Exponential Backoff Example

1
const retryableStatuses = new Set([502, 503, 504]);
2
3
export async function withJoyTokenRetry<T>(fn: () => Promise<T>, attempts = 3) {
4
  let lastError: unknown;
5
6
  for (let attempt = 0; attempt < attempts; attempt += 1) {
7
    try {
8
      return await fn();
9
    } catch (error: any) {
10
      lastError = error;
11
      const status = error?.status ?? error?.response?.status;
12
      if (!retryableStatuses.has(status) || attempt === attempts - 1) {
13
        throw error;
14
      }
15
      await new Promise((resolve) => setTimeout(resolve, 300 * 2 ** attempt));
16
    }
17
  }
18
19
  throw lastError;
20
}

Infinite retries on 402 and 403 usually amplify traffic. Fix the account, policy, or request configuration first.