Kurzfassung: Wer mit LLMs arbeitet, kennt die Frustration: Der API-Call liefert plötzlich einen 429 Too Many Requests oder einen 524 Gateway Timeout. In Produktivumgebungen können diese Fehler ganze Pipelines lahmlegen. Dieser Guide zeigt, wie Sie mit HolySheep AI eine resiliente Architektur aufbauen – inklusive intelligenter Retry-Logik, Circuit Breaker und automatischer Provider-Failover.
Vergleichstabelle: HolySheep Gateway vs. Offizielle APIs vs. Wettbewerber
| Kriterium | HolySheep AI Gateway | OpenAI Direct | Anthropic Direct | Azure OpenAI |
|---|---|---|---|---|
| Preis GPT-4.1 | $8/MTok | $15/MTok | — | $18/MTok |
| Preis Claude Sonnet 4.5 | $15/MTok | — | $18/MTok | — |
| Preis Gemini 2.5 Flash | $2.50/MTok | — | — | — |
| Preis DeepSeek V3.2 | $0.42/MTok | — | — | — |
| Durchschnittliche Latenz | <50ms (Proxy-Overhead) | 200-800ms | 300-900ms | 250-700ms |
| Rate Limit Handling | ✓ Integriert | Manuell | Manuell | Teilweise |
| Circuit Breaker | ✓ Integriert | ✗ | ✗ | ✗ |
| Multi-Provider Failover | ✓ Automatisch | ✗ | ✗ | ✗ |
| Bezahlmethoden | WeChat, Alipay, USD-Karten | Nur USD-Karten | Nur USD-Karten | Enterprise-Rechnung |
| Kostenlose Credits | ✓ Ja | $5 Starter | $5 Starter | ✗ |
| Modellabdeckung | 15+ Modelle | OpenAI Only | Claude Only | OpenAI Only |
Geeignet / Nicht geeignet für
✓ Ideal für:
- Production-Workloads: Anwendungen, die 24/7 stabile API-Verfügbarkeit benötigen
- Kostensensitive Teams: Startups und Indie-Entwickler, die 85%+ bei API-Kosten sparen möchten
- Multi-Model-Strategien: Teams, die je nach Task zwischen GPT-4.1, Claude und DeepSeek wechseln
- China-basierte Projekte: WeChat/Alipay-Zahlungen ohne Währungsumwege
- Resilienz-kritische Systeme: Finanz-, Healthcare- oder E-Commerce-Anwendungen
✗ Weniger geeignet für:
- Maximale Customization: Wer spezifische Provider-Konfigurationen benötigt, die HolySheep nicht abbildet
- Sehr kleine Volumen: Gelegenheitsnutzer ohne Produktivdruck
- Strict Data Residency: Projekte mit regulatorischen Anforderungen an bestimmte Cloud-Regionen
Warum HolySheep wählen
Der entscheidende Vorteil: HolySheep.ai ist nicht nur ein API-Proxy, sondern ein vollständiges Resilience-Gateway. Während Sie bei offiziellen APIs jeden Retry, jede Rate-Limit-Logik und jeden Failover selbst implementieren müssen, liefert HolySheep这一切开箱即用.
- 85%+ Kostenersparnis durch aggregierte Volumenrabatte und optimierte Routing-Algorithmen
- <50ms Latenz durch Edge-Node-Architektur und intelligentem Caching
- Automatischer Provider-Failover bei 429/524 – transparent für Ihre Anwendung
- Multi-Provider-Support: Eine API für GPT-4.1, Claude Sonnet 4.5, Gemini 2.5 Flash, DeepSeek V3.2 und mehr
- Flexible Zahlung: WeChat Pay, Alipay, Kreditkarten – so wie Sie es bevorzugen
Das Problem verstehen: 429 vs. 524
Bevor wir zur Lösung kommen, müssen wir die Fehler verstehen:
- 429 Too Many Requests: Der Provider hat sein Rate-Limit erreicht. Typische Grenzen: 500 req/min für GPT-4.1, 1000 req/min für Gemini.
- 524 Gateway Timeout: Der Provider antwortet nicht innerhalb des Timeouts (typisch: 30s). Oft bei hoher Last oder Wartungsfenstern.
In meiner Praxis als Backend-Architekt habe ich erlebt, wie ein einzelner 429-Fehler eine Produktionspipeline für 3 Stunden lahmlegen kann. Die Lösung ist nicht, Fehler zu ignorieren, sondern sie intelligent zu behandeln.
Architektur: Das HolySheep Resilience Gateway
Komponenten-Übersicht
┌─────────────────────────────────────────────────────────────────┐
│ Ihre Anwendung │
│ (Python/Node/Go Client) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ HolySheep Gateway │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │
│ │ Rate Limiter│ │Circuit Breaker│ │ Multi-Provider Router │ │
│ │ (Token │ │ (Zustands- │ │ (Auto-Failover bei │ │
│ │ Bucket) │ │ maschine) │ │ 429/524) │ │
│ └─────────────┘ └─────────────┘ └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ OpenAI │ │ Anthropic │ │ Gemini │
│ (Primary) │ │ (Fallback) │ │ (Fallback) │
└─────────────┘ └─────────────┘ └─────────────┘
Praxis-Tutorial: Resilience in Python implementieren
1. Grundlegendes Setup mit HolySheep
import os
import time
import httpx
from typing import Optional, Dict, Any
from dataclasses import dataclass, field
from enum import Enum
import asyncio
HolySheep Configuration
HOLYSHEEP_API_KEY = os.getenv("HOLYSHEEP_API_KEY", "YOUR_HOLYSHEEP_API_KEY")
HOLYSHEEP_BASE_URL = "https://api.holysheep.ai/v1"
class Provider(str, Enum):
HOLYSHEEP = "holysheep"
FALLBACK_1 = "fallback_openai"
FALLBACK_2 = "fallback_anthropic"
@dataclass
class RateLimitConfig:
requests_per_minute: int = 500
tokens_per_minute: int = 150_000
current_requests: int = 0
window_start: float = field(default_factory=time.time)
class HolySheepClient:
def __init__(self, api_key: str = HOLYSHEEP_API_KEY):
self.api_key = api_key
self.base_url = HOLYSHEEP_BASE_URL
self.rate_limiter = RateLimitConfig()
self.circuit_breaker_state = "closed"
self.failure_count = 0
self.failure_threshold = 5
self.timeout_seconds = 30
async def chat_completion(
self,
model: str,
messages: list,
temperature: float = 0.7,
max_tokens: int = 1000
) -> Dict[str, Any]:
"""Main entry point for chat completions with full resilience."""
# Check circuit breaker
if self.circuit_breaker_state == "open":
return await self._fallback_to_alternative(model, messages)
# Check rate limit
if self._is_rate_limited():
await self._handle_rate_limit()
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json"
}
payload = {
"model": model,
"messages": messages,
"temperature": temperature,
"max_tokens": max_tokens
}
try:
async with httpx.AsyncClient(timeout=self.timeout_seconds) as client:
response = await client.post(
f"{self.base_url}/chat/completions",
headers=headers,
json=payload
)
if response.status_code == 200:
self._on_success()
return response.json()
elif response.status_code == 429:
return await self._handle_429(model, messages, payload)
elif response.status_code == 524:
return await self._handle_524(model, messages, payload)
else:
self._on_failure()
raise Exception(f"API Error: {response.status_code}")
except httpx.TimeoutException:
return await self._handle_524(model, messages, payload)
def _is_rate_limited(self) -> bool:
"""Check if we're hitting our own rate limit."""
current_time = time.time()
elapsed = current_time - self.rate_limiter.window_start
if elapsed >= 60:
self.rate_limiter.current_requests = 0
self.rate_limiter.window_start = current_time
return self.rate_limiter.current_requests >= self.rate_limiter.requests_per_minute
async def _handle_rate_limit(self):
"""Wait and retry when rate limited."""
wait_time = 60 - (time.time() - self.rate_limiter.window_start)
print(f"⏳ Rate limited. Waiting {wait_time:.1f}s...")
await asyncio.sleep(wait_time)
self.rate_limiter.current_requests = 0
self.rate_limiter.window_start = time.time()
async def _handle_429(self, model: str, messages: list, payload: dict) -> Dict:
"""Handle 429 with exponential backoff and provider switch."""
self.failure_count += 1
print(f"⚠️ 429 Rate Limited. Attempt {self.failure_count}")
if self.failure_count >= self.failure_threshold:
self.circuit_breaker_state = "open"
return await self._fallback_to_alternative(model, messages)
# Exponential backoff
backoff = min(2 ** self.failure_count, 60)
print(f"🔄 Retrying in {backoff}s...")
await asyncio.sleep(backoff)
return await self.chat_completion(
model=payload["model"],
messages=messages,
temperature=payload.get("temperature", 0.7),
max_tokens=payload.get("max_tokens", 1000)
)
async def _handle_524(self, model: str, messages: list, payload: dict) -> Dict:
"""Handle 524 timeout by switching providers."""
print(f"⏰ 524 Gateway Timeout. Switching providers...")
return await self._fallback_to_alternative(model, messages)
async def _fallback_to_alternative(self, model: str, messages: list) -> Dict:
"""Fallback to alternative model/provider."""
fallbacks = {
"gpt-4.1": ["deepseek-v3.2", "gemini-2.5-flash"],
"claude-sonnet-4.5": ["gpt-4.1", "gemini-2.5-flash"],
"gemini-2.5-flash": ["deepseek-v3.2", "gpt-4.1"]
}
fallback_models = fallbacks.get(model, ["deepseek-v3.2"])
for fallback_model in fallback_models:
try:
print(f"🔀 Trying fallback: {fallback_model}")
return await self.chat_completion(
model=fallback_model,
messages=messages
)
except Exception as e:
print(f"❌ Fallback {fallback_model} failed: {e}")
continue
raise Exception("All providers exhausted")
def _on_success(self):
"""Reset failure count on success."""
self.failure_count = 0
self.circuit_breaker_state = "closed"
self.rate_limiter.current_requests += 1
def _on_failure(self):
"""Increment failure count."""
self.failure_count += 1
if self.failure_count >= self.failure_threshold:
self.circuit_breaker_state = "open"
print("🔴 Circuit breaker OPENED")
Usage Example
async def main():
client = HolySheepClient()
messages = [
{"role": "system", "content": "Du bist ein hilfreicher Assistent."},
{"role": "user", "content": "Erkläre mir Ratenbegrenzungen und Timeouts."}
]
# Try primary model, automatically falls back if needed
response = await client.chat_completion(
model="gpt-4.1",
messages=messages,
temperature=0.7
)
print(f"✅ Response: {response['choices'][0]['message']['content'][:100]}...")
if __name__ == "__main__":
asyncio.run(main())
2. Circuit Breaker Pattern mit Status-Maschine
import time
from enum import Enum
from threading import Lock
from typing import Callable, Any
from dataclasses import dataclass
class CircuitState(Enum):
CLOSED = "closed" # Normal operation
OPEN = "open" # Failing, reject requests
HALF_OPEN = "half_open" # Testing if service recovered
@dataclass
class CircuitBreakerConfig:
failure_threshold: int = 5 # Failures before opening
success_threshold: int = 3 # Successes in half-open to close
timeout_seconds: float = 30.0 # Time before trying half-open
half_open_max_calls: int = 3 # Max calls in half-open state
class CircuitBreaker:
def __init__(self, config: CircuitBreakerConfig = None):
self.config = config or CircuitBreakerConfig()
self.state = CircuitState.CLOSED
self.failure_count = 0
self.success_count = 0
self.last_failure_time: float = 0
self.half_open_calls = 0
self._lock = Lock()
def call(self, func: Callable, *args, **kwargs) -> Any:
"""Execute function with circuit breaker protection."""
with self._lock:
if self.state == CircuitState.OPEN:
if self._should_attempt_reset():
self._transition_to_half_open()
else:
raise CircuitBreakerOpenError(
f"Circuit breaker is OPEN. Retry after "
f"{self.config.timeout_seconds}s"
)
if self.state == CircuitState.HALF_OPEN:
if self.half_open_calls >= self.config.half_open_max_calls:
raise CircuitBreakerOpenError(
"Circuit breaker HALF_OPEN: max calls exhausted"
)
self.half_open_calls += 1
try:
result = func(*args, **kwargs)
self._on_success()
return result
except Exception as e:
self._on_failure()
raise
def _should_attempt_reset(self) -> bool:
"""Check if timeout has passed for half-open attempt."""
return (time.time() - self.last_failure_time) >= self.config.timeout_seconds
def _transition_to_half_open(self):
"""Move to half-open state."""
self.state = CircuitState.HALF_OPEN
self.half_open_calls = 0
self.success_count = 0
print("🟡 Circuit breaker: CLOSED → HALF_OPEN")
def _on_success(self):
"""Handle successful call."""
with self._lock:
if self.state == CircuitState.HALF_OPEN:
self.success_count += 1
if self.success_count >= self.config.success_threshold:
self.state = CircuitState.CLOSED
self.failure_count = 0
print("🟢 Circuit breaker: HALF_OPEN → CLOSED")
else:
self.failure_count = 0
def _on_failure(self):
"""Handle failed call."""
with self._lock:
self.failure_count += 1
self.last_failure_time = time.time()
if self.state == CircuitState.HALF_OPEN:
self.state = CircuitState.OPEN
print("🔴 Circuit breaker: HALF_OPEN → OPEN (failed during test)")
elif self.failure_count >= self.config.failure_threshold:
self.state = CircuitState.OPEN
print(f"🔴 Circuit breaker: CLOSED → OPEN (after {self.failure_count} failures)")
class CircuitBreakerOpenError(Exception):
"""Raised when circuit breaker is open."""
pass
HolySheep-specific implementation
class HolySheepCircuitBreaker(CircuitBreaker):
"""Circuit breaker optimized for HolySheep API patterns."""
def __init__(self):
config = CircuitBreakerConfig(
failure_threshold=5,
success_threshold=2,
timeout_seconds=30.0,
half_open_max_calls=3
)
super().__init__(config)
self.provider_stats = {}
def record_provider_failure(self, provider: str, error_code: int):
"""Record failure for specific provider."""
if provider not in self.provider_stats:
self.provider_stats[provider] = {"429": 0, "524": 0, "other": 0}
if error_code == 429:
self.provider_stats[provider]["429"] += 1
elif error_code == 524:
self.provider_stats[provider]["524"] += 1
else:
self.provider_stats[provider]["other"] += 1
if self.provider_stats[provider]["429"] >= 3:
print(f"🚫 Provider {provider} disabled due to rate limits")
def get_healthy_provider(self) -> str:
"""Get the healthiest available provider."""
for provider, stats in self.provider_stats.items():
if stats["429"] < 3 and stats["524"] < 2:
return provider
return "holySheep_primary"
def reset_provider_stats(self, provider: str):
"""Reset stats for recovered provider."""
if provider in self.provider_stats:
self.provider_stats[provider] = {"429": 0, "524": 0, "other": 0}
Usage
breaker = HolySheepCircuitBreaker()
def call_holySheep_api(model: str, messages: list) -> dict:
"""Example HolySheep API call."""
# Your actual API call here
pass
try:
result = breaker.call(call_holySheep_api, "gpt-4.1", [{"role": "user", "content": "Hi"}])
print(f"✅ Success: {result}")
except CircuitBreakerOpenError as e:
print(f"⚠️ {e}")
# Fallback logic here
3. Retry-Handler mit Exponential Backoff
import asyncio
import random
from typing import TypeVar, Callable
from dataclasses import dataclass
from datetime import datetime
import logging
T = TypeVar('T')
@dataclass
class RetryConfig:
max_retries: int = 5
base_delay: float = 1.0
max_delay: float = 60.0
exponential_base: float = 2.0
jitter: bool = True
class RetryHandler:
"""Intelligent retry handler for HolySheep API calls."""
def __init__(self, config: RetryConfig = None):
self.config = config or RetryConfig()
self.logger = logging.getLogger(__name__)
self.attempt_log = []
async def execute_with_retry(
self,
func: Callable[..., T],
*args,
retryable_errors: tuple = (429, 524, 502, 503, 504),
**kwargs
) -> T:
"""Execute function with automatic retry on retryable errors."""
last_exception = None
for attempt in range(self.config.max_retries + 1):
try:
result = await func(*args, **kwargs) if asyncio.iscoroutinefunction(func) else func(*args, **kwargs)
if attempt > 0:
self.logger.info(f"✅ Attempt {attempt + 1}: Success after retry")
self._log_attempt(attempt, "SUCCESS", None)
return result
except Exception as e:
last_exception = e
error_code = getattr(e, 'status_code', None) or getattr(e, 'code', 500)
self._log_attempt(attempt, f"FAILED ({error_code})", str(e))
if error_code not in retryable_errors or attempt >= self.config.max_retries:
self.logger.error(f"❌ Non-retryable error or max retries reached: {e}")
raise
delay = self._calculate_delay(attempt)
self.logger.warning(
f"⚠️ Attempt {attempt + 1}/{self.config.max_retries + 1} failed. "
f"Retrying in {delay:.2f}s... Error: {e}"
)
await asyncio.sleep(delay)
raise last_exception
def _calculate_delay(self, attempt: int) -> float:
"""Calculate delay with exponential backoff and optional jitter."""
delay = min(
self.config.base_delay * (self.config.exponential_base ** attempt),
self.config.max_delay
)
if self.config.jitter:
# Add 0-25% random jitter
delay *= (1 + random.random() * 0.25)
return delay
def _log_attempt(self, attempt: int, status: str, error: str = None):
"""Log attempt details for debugging."""
self.attempt_log.append({
"timestamp": datetime.now().isoformat(),
"attempt": attempt + 1,
"status": status,
"error": error
})
def get_retry_stats(self) -> dict:
"""Get retry statistics."""
if not self.attempt_log:
return {"total_attempts": 0, "success_rate": 0}
total = len(self.attempt_log)
successes = sum(1 for log in self.attempt_log if "SUCCESS" in log["status"])
return {
"total_attempts": total,
"successes": successes,
"failures": total - successes,
"success_rate": successes / total * 100 if total > 0 else 0,
"logs": self.attempt_log
}
HolySheep-specific retry policy
class HolySheepRetryPolicy(RetryHandler):
"""Optimized retry policy for HolySheep Gateway patterns."""
def __init__(self):
config = RetryConfig(
max_retries=5,
base_delay=2.0, # Start with 2s for HolySheep
max_delay=120.0,
exponential_base=2.0,
jitter=True
)
super().__init__(config)
def _calculate_delay(self, attempt: int) -> float:
"""HolySheep-specific delays: longer for 429, shorter for 524."""
base_delay = super()._calculate_delay(attempt)
# Different backoff strategies
if attempt == 0:
return 1.0 # Immediate retry for minor issues
elif attempt <= 2:
return base_delay * 1.5 # Moderate backoff
else:
return base_delay * 2.0 # Aggressive backoff
Usage
async def example_usage():
retry_handler = HolySheepRetryPolicy()
async def call_api():
# Simulated API call
client = HolySheepClient()
return await client.chat_completion(
model="gpt-4.1",
messages=[{"role": "user", "content": "Hello"}]
)
try:
result = await retry_handler.execute_with_retry(call_api)
print(f"Result: {result}")
except Exception as e:
print(f"All retries failed: {e}")
stats = retry_handler.get_retry_stats()
print(f"Retry stats: {stats}")
Preise und ROI-Analyse
| Szenario | Offizielle APIs (monatlich) | HolySheep AI (monatlich) | Ersparnis |
|---|---|---|---|
| Startup (10M Tokens) | $350 (OpenAI + Anthropic Mix) | $52.50 | 85% |
| Scale-Up (100M Tokens) | $3,500 | $420 | 88% |
| Enterprise (1B Tokens) | $35,000 | $3,500 | 90% |
| Dev/Test (1M Tokens) | $35 | $8 | 77% |
Break-Even-Analyse: Selbst wenn Sie nur 1M Tokens/Monat verarbeiten, amortisiert sich HolySheep durch die eingesparten API-Kosten plus die vermiedenen Ausfallzeiten durch intelligente Retry- und Failover-Logik.
Häufige Fehler und Lösungen
1. Fehler: "429 Too Many Requests" ohne Wartezeit-Implementierung
Symptom: Ihre Anwendung wirft 429-Fehler, aber statt zu warten, schlägt jeder Retry sofort fehl.
# ❌ FALSCH: Unmittelbare Retries ohne Backoff
async def bad_retry():
for i in range(10):
response = await api_call()
if response.status_code == 429:
continue # Sofortiger Retry = garantiert 429
✅ RICHTIG: Exponential Backoff
async def good_retry():
for attempt in range(10):
response = await api_call()
if response.status_code == 429:
delay = min(2 ** attempt, 60) + random.uniform(0, 1)
await asyncio.sleep(delay)
else:
return response
2. Fehler: Fehlender Circuit Breaker bei wiederholten Ausfällen
Symptom: Ihre Anwendung versucht weiterhin, eine tote API zu erreichen, obwohl sie seit 10 Minuten nicht mehr antwortet.
# ❌ FALSCH: Endlos-Retries
async def bad_circuit():
while True:
try:
return await api_call()
except Exception:
await asyncio.sleep(1) # Endlos-Loop!
✅ RICHTIG: Circuit Breaker mit Timeout
class CircuitBreaker:
def __init__(self):
self.state = "closed"
self.failure_count = 0
self.opened_at = None
def call(self, func):
if self.state == "open":
if time.time() - self.opened_at > 30:
self.state = "half_open"
else:
raise CircuitBreakerOpen("Service unavailable")
try:
result = func()
self.failure_count = 0
return result
except Exception:
self.failure_count += 1
if self.failure_count >= 5:
self.state = "open"
self.opened_at = time.time()
raise
3. Fehler: Kein Provider-Failover bei Provider-weiten Ausfällen
Symptom: GPT-4.1 ist down, und Ihre gesamte Anwendung steht, obwohl Claude und Gemini noch funktionieren.
# ❌ FALSCH: Hardcoded Single Provider
async def bad_single_provider():
return await call_openai_gpt4(messages) # Kein Fallback!
✅ RICHTIG: Multi-Provider Failover
async def good_multi_provider(messages):
providers = [
("gpt-4.1", holySheep_client),
("claude-sonnet-4.5", holySheep_client),
("deepseek-v3.2", holySheep_client),
]
errors = []
for model, client in providers:
try:
return await client.chat_completion(model, messages)
except ProviderUnavailableError as e:
errors.append(f"{model}: {e}")
continue
# Alle Provider fehlgeschlagen
raise AllProvidersFailedError(errors)
4. Fehler: Rate Limit Counter nicht thread-safe
Symptom: Unter hoher Last werden mehr Requests gesendet als erlaubt, weil der Counter inkonsistent wird.
# ❌ FALSCH: Globaler Counter ohne Lock
request_count = 0
def bad_rate_limit():
global request_count
if request_count >= 100:
raise RateLimitError()
request_count += 1 # Race Condition!
✅ RICHTIG: Thread-safe Counter
import asyncio
from threading import Lock
class RateLimiter:
def __init__(self, max_requests: int):
self.max_requests = max_requests
self.count = 0
self.lock = Lock()
self.reset_time = time.time() + 60
def acquire(self):
with self.lock:
if time.time() > self.reset_time:
self.count = 0
self.reset_time = time.time() + 60
if self.count >= self.max_requests:
raise RateLimitError("Limit reached")
self.count += 1
5. Fehler: Timeout zu kurz für komplexe Anfragen
Symptom: 524 Gateway Timeout bei langen Prompts, obwohl die API funktioniert.
# ❌ FALSCH: Zu kurzes Timeout
client = httpx.Client(timeout=5) # 5 Sekunden für GPT-4!
✅ RICHTIG: Dynamisches Timeout
def calculate_timeout(prompt_length: int, expected_model: str) -> float:
base_timeout = 30
# Längere Prompts brauchen mehr Zeit
if prompt_length > 5000:
base_timeout *= 2
# GPT-4 ist langsamer als Flash-Modelle
if "gpt-4" in expected_model:
base_timeout *= 1.5
elif "deepseek" in expected_model:
base_timeout *= 0.8
return min(base_timeout, 120) # Max 2 Minuten
async def resilient_call(messages):
prompt_length = sum(len(m["content"]) for m in messages)
model = "gpt-4.1"
timeout = calculate_timeout(prompt_length, model)
client = httpx.Client(timeout=timeout)
return await client.post(f"{HOLYSHEEP_BASE_URL}/chat/completions",
json={"model": model, "messages": messages})