Khi tôi triển khai hệ thống AI gateway cho startup của mình năm 2024, một vấn đề nan giải đã xuất hiện: Làm sao để tối ưu chi phí khi sử dụng đồng thời nhiều LLM? Sau hàng trăm lần thử nghiệm với Kong và Traefik, tôi đã tìm ra giải pháp routing thông minh giúp tiết kiệm 85%+ chi phí API hàng tháng. Bài viết này sẽ chia sẻ toàn bộ kinh nghiệm thực chiến, kèm code có thể sao chép và chạy ngay.
Tại Sao Cần Multi-Model Routing?
Trước khi đi vào chi tiết kỹ thuật, hãy xem xét bài toán chi phí thực tế với dữ liệu giá 2026 đã được xác minh:
| Model | Giá Input ($/MTok) | Giá Output ($/MTok) | Phù hợp cho |
|---|---|---|---|
| GPT-4.1 | $8.00 | $24.00 | Task phức tạp, reasoning |
| Claude Sonnet 4.5 | $15.00 | $75.00 | Creative writing, analysis |
| Gemini 2.5 Flash | $2.50 | $10.00 | Fast inference, bulk tasks |
| DeepSeek V3.2 | $0.42 | $1.68 | High volume, cost-sensitive |
So sánh chi phí cho 10 triệu token/tháng:
- Chỉ dùng Claude Sonnet 4.5: ~$450,000/tháng (nếu 50% input, 50% output)
- Chỉ dùng DeepSeek V3.2: ~$10,500/tháng
- Hybrid routing thông minh: ~$15,000-25,000/tháng (tiết kiệm 95%)
Sự chênh lệch này là lý do multi-model routing trở nên bắt buộc với bất kỳ production system nào. Với HolySheep AI, bạn có thể truy cập tất cả các model trên với tỷ giá ¥1=$1 — tiết kiệm 85%+ so với giá gốc, thanh toán qua WeChat hoặc Alipay.
Kiến Trúc Tổng Quan
┌─────────────────────────────────────────────────────────────────┐
│ Client Request │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ API Gateway (Kong/Traefik) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Rate Limiter │ │ Auth Middle │ │ Router Logic │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
┌───────────────────────┼───────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ GPT-4.1 │ │ Claude Sonnet │ │ DeepSeek V3.2 │
│ /chat/complet │ │ 4.5 │ │ /chat/complet │
└───────────────┘ └───────────────┘ └───────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────┐
│ HolySheep AI Gateway │
│ https://api.holysheep.ai/v1/chat/completions │
└─────────────────────────────────────────────────────────────────┘
Cài Đặt Kong Gateway Với Docker
Tôi bắt đầu với Kong vì nó có plugin ecosystem phong phú và hỗ trợ declarative configuration tốt. Đây là setup production-ready hoàn chỉnh:
version: '3.8'
services:
kong-database:
image: postgres:15-alpine
container_name: kong-db
environment:
POSTGRES_USER: kong
POSTGRES_PASSWORD: kong_secret_pass
POSTGRES_DB: kong
volumes:
- kong-db-data:/var/lib/postgresql/data
networks:
- ai-gateway-net
restart: unless-stopped
kong-migration:
image: kong:3.4-alpine
container_name: kong-migration
depends_on:
- kong-database
environment:
KONG_DATABASE: postgres
KONG_PG_HOST: kong-database
KONG_PG_USER: kong
KONG_PG_PASSWORD: kong_secret_pass
KONG_DATABASE: postgres
KONG_DECLARATIVE_CONFIG: /usr/local/kong/declarative.yml
volumes:
- ./kong/declarative.yml:/usr/local/kong/declarative.yml:ro
networks:
- ai-gateway-net
restart: on-failure
command: kong migrations bootstrap
kong:
image: kong:3.4-alpine
container_name: kong-gateway
depends_on:
- kong-database
- kong-migration
environment:
KONG_DATABASE: postgres
KONG_PG_HOST: kong-database
KONG_PG_USER: kong
KONG_PG_PASSWORD: kong_secret_pass
KONG_DECLARATIVE_CONFIG: /usr/local/kong/declarative.yml
KONG_PROXY_ACCESS_LOG: /dev/stdout
KONG_ADMIN_ACCESS_LOG: /dev/stdout
KONG_PROXY_ERROR_LOG: /dev/stderr
KONG_ADMIN_ERROR_LOG: /dev/stderr
KONG_ADMIN_LISTEN: 0.0.0.0:8001, 0.0.0.0:8444 ssl
KONG_PLUGINS: bundled,ai-router,routing-rules
ports:
- "8000:8000" # HTTP proxy
- "8443:8443" # HTTPS proxy
- "8001:8001" # Admin API HTTP
- "8444:8444" # Admin API HTTPS
volumes:
- ./kong/declarative.yml:/usr/local/kong/declarative.yml:ro
- ./kong/plugins:/usr/local/kong/plugins:ro
networks:
- ai-gateway-net
restart: unless-stopped
konga:
image: pantsel/konga:0.14.9
container_name: konga-ui
depends_on:
- kong
environment:
NODE_ENV: production
DB_ADAPTER: postgres
DB_URI: postgresql://kong:kong_secret_pass@kong-database:5432/konga
ports:
- "1337:1337"
networks:
- ai-gateway-net
restart: unless-stopped
volumes:
kong-db-data:
networks:
ai-gateway-net:
driver: bridge
Declarative Configuration Cho Multi-Model Routing
Đây là phần quan trọng nhất — file declarative.yml định nghĩa toàn bộ routing logic. Tôi đã tối ưu cấu hình này qua nhiều lần production deployment:
_format_version: "3.0"
services:
# ─────────────────────────────────────────────────────────
# GPT-4.1 Route - Cho complex reasoning tasks
# ─────────────────────────────────────────────────────────
- name: gpt4-reasoning
url: https://api.holysheep.ai/v1/chat/completions
routes:
- name: gpt4-route
paths:
- /ai/gpt4
methods:
- POST
strip_path: false
plugins:
- name: rate-limiting
config:
minute: 100
policy: local
- name: request-transformer
config:
add:
headers:
- "X-AI-Model:gpt-4.1"
- "X-Routing-Tier:premium"
- name: response-transformer
config:
add:
headers:
- "X-Used-Provider:HolySheep"
# ─────────────────────────────────────────────────────────
# Claude Route - Cho creative và analysis tasks
# ─────────────────────────────────────────────────────────
- name: claude-creative
url: https://api.holysheep.ai/v1/chat/completions
routes:
- name: claude-route
paths:
- /ai/claude
methods:
- POST
strip_path: false
plugins:
- name: rate-limiting
config:
minute: 80
policy: local
- name: request-transformer
config:
add:
headers:
- "X-AI-Model:claude-sonnet-4.5"
- "X-Routing-Tier:premium"
# ─────────────────────────────────────────────────────────
# Gemini Flash Route - Cho fast inference
# ─────────────────────────────────────────────────────────
- name: gemini-fast
url: https://api.holysheep.ai/v1/chat/completions
routes:
- name: gemini-route
paths:
- /ai/gemini
methods:
- POST
strip_path: false
plugins:
- name: rate-limiting
config:
minute: 500
policy: local
- name: request-transformer
config:
add:
headers:
- "X-AI-Model:gemini-2.5-flash"
- "X-Routing-Tier:standard"
# ─────────────────────────────────────────────────────────
# DeepSeek Route - Cho high-volume, cost-sensitive tasks
# ─────────────────────────────────────────────────────────
- name: deepseek-economy
url: https://api.holysheep.ai/v1/chat/completions
routes:
- name: deepseek-route
paths:
- /ai/deepseek
methods:
- POST
strip_path: false
plugins:
- name: rate-limiting
config:
minute: 1000
policy: local
- name: request-transformer
config:
add:
headers:
- "X-AI-Model:deepseek-v3.2"
- "X-Routing-Tier:economy"
# ─────────────────────────────────────────────────────────
# Smart Router - Tự động chọn model dựa trên request
# ─────────────────────────────────────────────────────────
- name: smart-router
url: https://api.holysheep.ai/v1/chat/completions
routes:
- name: smart-route
paths:
- /ai/smart
methods:
- POST
strip_path: false
plugins:
- name: request-termination
config:
status_code: 200
content_type: application/json
body: >
{"router": "active", "mode": "smart", "strategy": "cost-optimization"}
Consumer cho API Key authentication
consumers:
- username: production-app
keyauth_credentials:
- key: YOUR_KONG_API_KEY
- username: staging-app
keyauth_credentials:
- key: YOUR_STAGING_KEY
Global plugins
plugins:
- name: key-auth
config:
key_names:
- x-api-key
- authorization
key_in_header: true
key_in_query: true
- name: cors
config:
origins:
- "*"
methods:
- GET
- POST
- PUT
- DELETE
- OPTIONS
headers:
- Accept
- Authorization
- Content-Type
- x-api-key
credentials: true
max_age: 3600
Plugin Custom AI Router (Lua)
Kong's strength nằm ở khả năng mở rộng qua Lua plugins. Đây là plugin intelligent routing mà tôi viết để tự động chọn model dựa trên query characteristics:
-- File: kong/plugins/ai-router/handler.lua
local kong = kong
local req = kong.request
local re_find = string.find
local re_match = ngx.re.match
local AiRouterHandler = {}
AiRouterHandler.PRIORITY = 1000
AiRouterHandler.VERSION = "1.0.0"
-- Model mapping với priorities và cost weights
local MODEL_CONFIG = {
["complex-reasoning"] = {
model = "gpt-4.1",
max_tokens = 8192,
cost_weight = 19.0, -- $8 input + $24 output
latency_priority = 3,
keywords = {"analyze", "reason", "complex", "solve", "logic"}
},
["creative-writing"] = {
model = "claude-sonnet-4.5",
max_tokens = 4096,
cost_weight = 45.0, -- $15 input + $75 output
latency_priority = 4,
keywords = {"write", "creative", "story", "poem", "narrative"}
},
["fast-inference"] = {
model = "gemini-2.5-flash",
max_tokens = 8192,
cost_weight = 6.25, -- $2.50 input + $10 output
latency_priority = 1,
keywords = {"quick", "fast", "summary", "brief", "list"}
},
["high-volume"] = {
model = "deepseek-v3.2",
max_tokens = 4096,
cost_weight = 1.05, -- $0.42 input + $1.68 output
latency_priority = 2,
keywords = {"batch", "bulk", "process", "translate", "classify"}
}
}
-- Hàm phân tích query để chọn model
local function analyze_intent(message)
local lower_msg = string.lower(message)
-- Check for explicit model override
for tier, config in pairs(MODEL_CONFIG) do
for _, keyword in ipairs(config.keywords) do
if re_find(lower_msg, keyword) then
kong.log.notice("Matched keyword '", keyword, "' -> routing to ", tier)
return tier
end
end
end
-- Default: balance giữa cost và quality
return "high-volume"
end
-- Transform request body theo model được chọn
local function build_model_payload(body, model_name, model_config)
local payload = kong.service.request.get_body()
-- Merge với config
if payload then
payload.model = model_name
if model_config.max_tokens then
payload.max_tokens = math.min(
payload.max_tokens or 2048,
model_config.max_tokens
)
end
else
payload = {
model = model_name,
messages = body.messages,
temperature = body.temperature or 0.7,
max_tokens = body.max_tokens or 2048
}
end
return payload
end
function AiRouterHandler:access(conf)
-- Chỉ áp dụng cho smart route
if not re_find(req.get_path(), "/ai/smart") then
return
end
local body = req.get_body()
if not body or not body.messages then
kong.log.warn("No messages in request body")
return
end
-- Lấy user message
local user_message = ""
for _, msg in ipairs(body.messages) do
if msg.role == "user" then
user_message = msg.content or ""
break
end
end
-- Analyze và chọn model
local tier = analyze_intent(user_message)
local model_config = MODEL_CONFIG[tier]
-- Update request
local new_body = build_model_payload(body, model_config.model, model_config)
kong.service.request.set_body(new_body, "application/json")
-- Add routing headers for debugging
kong.service.request.set_header("X-Routed-Model", model_config.model)
kong.service.request.set_header("X-Routing-Tier", tier)
kong.service.request.set_header("X-Cost-Weight", tostring(model_config.cost_weight))
kong.log.notice("AI Router: ", model_config.model, " (tier: ", tier, ")")
end
return AiRouterHandler
-- File: kong/plugins/ai-router/schema.lua
local typedefs = require "kong.db.schema.typedefs"
return {
name = "ai-router",
fields = {
{ config = {
type = "record",
fields = {
{ routing_strategy = {
type = "string",
default = "cost-optimization",
one_of = {"cost-optimization", "latency-first", "balanced"}
}},
{ fallback_model = {
type = "string",
default = "deepseek-v3.2"
}},
{ max_cost_per_request = {
type = "number",
default = 0.50
}},
{ enable_caching = {
type = "boolean",
default = true
}},
{ cache_ttl = {
type = "integer",
default = 3600
}}
}
}}
}
}
Traefik Configuration - Alternative Approach
Nếu bạn prefer Traefik vì simplicity và native Docker integration, đây là configuration tương đương sử dụng file-based routing:
version: '3.8'
services:
traefik:
image: traefik:v2.10-alpine
container_name: traefik-gateway
command:
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.file.directory=/etc/traefik/dynamic"
- "--providers.file.watch=true"
- "--entrypoints.web.address=:8000"
- "--entrypoints.websecure.address=:8443"
- "--log.level=INFO"
- "--accesslog=true"
ports:
- "8000:8000"
- "8443:8443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik/dynamic:/etc/traefik/dynamic:ro
- ./traefik/certs:/certs:ro
networks:
- ai-gateway-net
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.dashboard.rule=Host(localhost)"
- "traefik.http.routers.dashboard.service=api@internal"
# AI Router Service (Node.js based)
ai-router:
build:
context: ./ai-router-service
dockerfile: Dockerfile
container_name: ai-router-service
environment:
- NODE_ENV=production
- HOLYSHEEP_API_KEY=${HOLYSHEEP_API_KEY}
- LOG_LEVEL=info
- REDIS_URL=redis://redis:6379
volumes:
- ./ai-router-service:/app
- /app/node_modules
networks:
- ai-gateway-net
depends_on:
- redis
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.ai-router.rule=PathPrefix(/ai)"
- "traefik.http.routers.ai-router.entrypoints=web"
- "traefik.http.services.ai-router.loadbalancer.server.port=3000"
redis:
image: redis:7-alpine
container_name: redis-cache
networks:
- ai-gateway-net
volumes:
- redis-data:/data
restart: unless-stopped
command: redis-server --appendonly yes
networks:
ai-gateway-net:
driver: bridge
volumes:
redis-data:
# File: traefik/dynamic/ai-routing.toml
Middleware: Rate Limiting
[http.middlewares]
[http.middlewares.rate-gpt4.rateLimit]
average = 100
burst = 50
period = "1m"
[http.middlewares.rate-claude.rateLimit]
average = 80
burst = 40
period = "1m"
[http.middlewares.rate-gemini.rateLimit]
average = 500
burst = 250
period = "1m"
[http.middlewares.rate-deepseek.rateLimit]
average = 1000
burst = 500
period = "1m"
# IP Whitelist
[http.middlewares.api-auth.forwardAuth]
address = "http://auth-service:8080/validate"
trustForwardHeader = true
HTTP Routers cho từng model
[http.routers]
# GPT-4.1 Router
[http.routers.gpt4-router]
rule = "PathPrefix(/ai/gpt4)"
service = "holysheep-gpt4"
entryPoints = ["web"]
middlewares = ["rate-gpt4"]
# Claude Router
[http.routers.claude-router]
rule = "PathPrefix(/ai/claude)"
service = "holysheep-claude"
entryPoints = ["web"]
middlewares = ["rate-claude"]
# Gemini Router
[http.routers.gemini-router]
rule = "PathPrefix(/ai/gemini)"
service = "holysheep-gemini"
entryPoints = ["web"]
middlewares = ["rate-gemini"]
# DeepSeek Router
[http.routers.deepseek-router]
rule = "PathPrefix(/ai/deepseek)"
service = "holysheep-deepseek"
entryPoints = ["web"]
middlewares = ["rate-deepseek"]
Services - Tất cả đều forward tới HolySheep AI
[http.services]
[http.services.holysheep-gpt4.loadBalancer]
[[http.services.holysheep-gpt4.loadBalancer.servers]]
url = "https://api.holysheep.ai/v1/chat/completions"
# Headers được thêm bởi middleware service
[http.services.holysheep-claude.loadBalancer]
[[http.services.holysheep-claude.loadBalancer.servers]]
url = "https://api.holysheep.ai/v1/chat/completions"
[http.services.holysheep-gemini.loadBalancer]
[[http.services.holysheep-gemini.loadBalancer.servers]]
url = "https://api.holysheep.ai/v1/chat/completions"
[http.services.holysheep-deepseek.loadBalancer]
[[http.services.holysheep-deepseek.loadBalancer.servers]]
url = "https://api.holysheep.ai/v1/chat/completions"
Node.js Smart Router Service
Đây là service xử lý intelligent routing với caching và fallback logic:
// File: ai-router-service/src/index.js
import express from 'express';
import Redis from 'ioredis';
import crypto from 'crypto';
const app = express();
app.use(express.json({ limit: '10mb' }));
// Redis connection cho caching
const redis = new Redis(process.env.REDIS_URL || 'redis://localhost:6379');
const HOLYSHEEP_API = 'https://api.holysheep.ai/v1/chat/completions';
// Model configurations với cost weights
const MODEL_CONFIG = {
gpt4: {
name: 'gpt-4.1',
costPer1K: 0.008, // Input: $8/MTok
outputCostPer1K: 0.024, // Output: $24/MTok
maxTokens: 8192,
latencyTier: 'premium',
useFor: ['reasoning', 'complex', 'analyze', 'solve', 'code']
},
claude: {
name: 'claude-sonnet-4.5',
costPer1K: 0.015,
outputCostPer1K: 0.075,
maxTokens: 4096,
latencyTier: 'premium',
useFor: ['creative', 'write', 'story', 'narrative', 'edit']
},
gemini: {
name: 'gemini-2.5-flash',
costPer1K: 0.0025,
outputCostPer1K: 0.010,
maxTokens: 8192,
latencyTier: 'fast',
useFor: ['quick', 'fast', 'summary', 'brief']
},
deepseek: {
name: 'deepseek-v3.2',
costPer1K: 0.00042,
outputCostPer1K: 0.00168,
maxTokens: 4096,
latencyTier: 'economy',
useFor: ['batch', 'bulk', 'translate', 'classify', 'extract']
}
};
// Intelligent model selection
function selectModel(messages, options = {}) {
const lastMessage = messages[messages.length - 1]?.content || '';
const lowerMessage = lastMessage.toLowerCase();
// Check explicit routing header
if (options.forceModel && MODEL_CONFIG[options.forceModel]) {
return MODEL_CONFIG[options.forceModel];
}
// Check for keywords
for (const [key, config] of Object.entries(MODEL_CONFIG)) {
for (const keyword of config.useFor) {
if (lowerMessage.includes(keyword)) {
console.log([Router] Matched keyword '${keyword}' -> ${key});
return { ...config, key };
}
}
}
// Default: DeepSeek for cost optimization
return { ...MODEL_CONFIG.deepseek, key: 'deepseek' };
}
// Generate cache key
function getCacheKey(messages, model) {
const content = messages.map(m => ${m.role}:${m.content}).join('|');
const hash = crypto.createHash('sha256')
.update(content + model.name)
.digest('hex')
.substring(0, 16);
return ai:cache:${hash};
}
// Proxy request to HolySheep AI
async function proxyToAI(payload, modelConfig) {
const response = await fetch(HOLYSHEEP_API, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': Bearer ${process.env.HOLYSHEEP_API_KEY},
'X-Model-Override': modelConfig.name
},
body: JSON.stringify(payload)
});
if (!response.ok) {
const error = await response.text();
throw new Error(HolySheep API error: ${response.status} - ${error});
}
return response.json();
}
// Main route handler
app.post('/ai/:model?', async (req, res) => {
const startTime = Date.now();
const { model: routeModel } = req.params;
const { messages, temperature, max_tokens, stream, ...otherOptions } = req.body;
try {
// Select model
const modelConfig = selectModel(messages, {
forceModel: routeModel,
...otherOptions
});
console.log([Router] Selected model: ${modelConfig.name});
// Check cache (cho non-streaming requests)
if (!stream) {
const cacheKey = getCacheKey(messages, modelConfig);
const cached = await redis.get(cacheKey);
if (cached) {
console.log([Router] Cache HIT for key: ${cacheKey});
const parsed = JSON.parse(cached);
return res.json({
...parsed,
cached: true,
model: modelConfig.name
});
}
}
// Build payload
const payload = {
model: modelConfig.name,
messages,
temperature: temperature || 0.7,
max_tokens: Math.min(max_tokens || 2048, modelConfig.maxTokens),
stream: stream || false,
...otherOptions
};
// Proxy to HolySheep AI
const aiResponse = await proxyToAI(payload, modelConfig);
// Cache response
if (!stream && aiResponse.choices?.[0]?.message?.content) {
const cacheKey = getCacheKey(messages, modelConfig);
await redis.setex(cacheKey, 3600, JSON.stringify(aiResponse));
console.log([Router] Cached response with TTL 3600s);
}
const latency = Date.now() - startTime;
console.log([Router] Response time: ${latency}ms);
res.json({
...aiResponse,
routing: {
model: modelConfig.name,
latency_ms: latency,
cache_hit: false
}
});
} catch (error) {
console.error([Router] Error:, error.message);
// Fallback: try with DeepSeek if primary fails
if (modelConfig.key !== 'deepseek') {
console.log([Router] Falling back to DeepSeek...);
const fallbackPayload = {
model: 'deepseek-v3.2',
messages,
temperature: temperature || 0.7,
max_tokens: 2048
};
try {
const fallbackResponse = await proxyToAI(fallbackPayload, MODEL_CONFIG.deepseek);
return res.json({
...fallbackResponse,
routing: {
model: 'deepseek-v3.2',
fallback: true
}
});
} catch (fallbackError) {
console.error([Router] Fallback also failed:, fallbackError.message);
}
}
res.status(500).json({
error: 'AI routing failed',
message: error.message
});
}
});
// Health check
app.get('/health', async (req, res) => {
const redisStatus = redis.status;
res.json({
status: 'healthy',
redis: redisStatus,
timestamp: new Date().toISOString()
});
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log([AI Router] Running on port ${PORT});
console.log([AI Router] HolySheep API: ${HOLYSHEEP_API});
});
Client Usage - Frontend Integration
Sau khi setup gateway, client code để sử dụng rất đơn giản. Tôi recommend dùng HolySheep AI vì latency chỉ <50ms và tiết kiệm 85%+ chi phí:
#!/usr/bin/env python3
"""
AI Gateway Client - Multi-Model Routing với HolySheep AI
Setup: pip install requests httpx aiohttp
"""
import os
import json
import asyncio
import hashlib
from typing import Optional, List, Dict, Any
from dataclasses import dataclass
from datetime import datetime
import requests
@dataclass
class AIMessage:
role: str
content: str
@dataclass
class AIResponse:
content: str
model: str
usage: Dict[str, int]
latency_ms: float
cached: bool = False
class HolySheepAIGateway:
"""
Multi-model AI gateway client với intelligent routing
"""
BASE_URL = "https://api.holysheep.ai/v1"
def __init__(self, api_key: str):
self.api_key = api_key
self.session = requests.Session()
self.session.headers.update({
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
})
def _estimate_cost(self, model: str, input_tokens: int, output_tokens: int) -> float:
"""Estimate cost theo model pricing 2026"""
pricing = {
"gpt-4.1": (8.00, 24.00), # input, output per MTok
"claude-sonnet-4.5": (15.00, 75.00),
"gemini-2.5-flash": (2.50, 10.00),
"deepseek-v3.2": (0.42, 1.68)
}
if model not in pricing:
model = "deepseek-v3.2" # default
input_cost, output_cost = pricing[model]
return (input_tokens / 1_000_000 * input_cost +
output_tokens / 1_000_000 * output_cost)
def chat_completion(
self,
messages: List[Dict[str, str]],
model: Optional[str] = None,
temperature: float = 0.7,
max_tokens: int = 2048,
route_hint: Optional[str] = None
) -> AIResponse:
"""
Gửi request tới AI gateway với optional model override
"""
start_time = datetime.now()
payload = {
"