Tháng 3/2026, tôi triển khai hệ thống RAG cho một doanh nghiệp thương mại điện tử với 50K người dùng hàng ngày. Đỉnh điểm Black Friday, API response time tăng từ 200ms lên 8 giây — khách hàng chờ đợi, đội ngũ hoảng loạn, và tôi nhận ra: không có monitoring = không có kiểm soát. Bài viết này chia sẻ cách tôi xây dựng production monitoring hoàn chỉnh cho HolySheep AI multi-model gateway với Datadog và Grafana, giúp bạn phát hiện vấn đề trước khi người dùng phàn nàn.

Vì sao cần giám sát API Gateway?

Khi làm việc với multi-model AI gateway, bạn đối mặt với những thách thức đặc thù:

Trong dự án thương mại điện tử đó, tôi mất 4 tiếng để identify nguyên nhân root cause vì thiếu metrics. Sau đó, với monitoring setup chuẩn, tôi phát hiện và xử lý vấn đề trong vòng 15 phút.

Kiến trúc Monitoring System

Trước khi vào code, hãy hiểu kiến trúc tổng thể:

┌─────────────────────────────────────────────────────────────────┐
│                      HOLYSHEEP AI GATEWAY                       │
│  base_url: https://api.holysheep.ai/v1                           │
├─────────────────────────────────────────────────────────────────┤
│  Request Flow:                                                   │
│  Client → API Gateway → Model Router → [Model Pool]              │
│                        ↓                                         │
│              Metrics Collector ← Prometheus/StatsD               │
│                        ↓                                         │
│              Datadog / Grafana ← Alert Manager                   │
└─────────────────────────────────────────────────────────────────┘

Model Pool:
├── GPT-4.1 (OpenAI compatible)        P95: 800ms, Cost: $8/MTok
├── Claude Sonnet 4.5 (Anthropic)      P95: 1200ms, Cost: $15/MTok
├── Gemini 2.5 Flash (Google)          P95: 200ms, Cost: $2.50/MTok
└── DeepSeek V3.2 (DeepSeek)           P95: 150ms, Cost: $0.42/MTok

1. Cài đặt Prometheus Metrics Collector

Đầu tiên, tôi tạo một middleware để expose Prometheus metrics từ API requests. Đây là phần quan trọng nhất — nếu không có metrics, thì không có gì để giám sát.

# metrics_server.py - Prometheus metrics collector

Install: pip install prometheus-client fastapi uvicorn httpx

from prometheus_client import Counter, Histogram, Gauge, start_http_server from fastapi import FastAPI, Request, Response from fastapi.responses import JSONResponse import httpx import time import os from typing import Optional

============================================

HOLYSHEEP API CONFIGURATION

============================================

HOLYSHEEP_BASE_URL = "https://api.holysheep.ai/v1" HOLYSHEEP_API_KEY = os.getenv("YOUR_HOLYSHEEP_API_KEY", "sk-holysheep-xxxxx")

============================================

METRICS DEFINITIONS

============================================

Request counter by model, status code

request_counter = Counter( 'holysheep_requests_total', 'Total API requests', ['model', 'status_code', 'endpoint'] )

Latency histogram in milliseconds

latency_histogram = Histogram( 'holysheep_request_latency_ms', 'Request latency in milliseconds', ['model', 'endpoint'], buckets=[50, 100, 200, 500, 1000, 2000, 5000, 10000, 30000] )

Token usage gauge

token_usage = Counter( 'holysheep_tokens_total', 'Total tokens used', ['model', 'token_type'] # token_type: prompt, completion )

5xx error counter

error_5xx_counter = Counter( 'holysheep_5xx_errors_total', 'Total 5xx errors', ['model', 'error_type'] )

Active requests gauge

active_requests = Gauge( 'holysheep_active_requests', 'Number of active requests', ['model'] )

Cost tracking

cost_accumulator = Counter( 'holysheep_cost_usd', 'Accumulated cost in USD', ['model'] )

============================================

MODEL PRICING (2026 rates)

============================================

MODEL_PRICING = { 'gpt-4.1': {'prompt': 0.000002, 'completion': 0.000006}, # $8/MTok 'claude-sonnet-4.5': {'prompt': 0.000003, 'completion': 0.000012}, # $15/MTok 'gemini-2.5-flash': {'prompt': 0.000000125, 'completion': 0.0000005}, # $2.50/MTok 'deepseek-v3.2': {'prompt': 0.000000021, 'completion': 0.00000007}, # $0.42/MTok }

============================================

API GATEWAY APPLICATION

============================================

app = FastAPI(title="HolySheep AI Gateway Monitored") @app.api_route("/{full_path:path}", methods=["GET", "POST", "PUT", "DELETE"]) async def proxy_to_holysheep(request: Request, full_path: str): """ Proxy requests to HolySheep AI with full metrics collection """ start_time = time.perf_counter() model = request.headers.get('X-Model', 'gpt-4.1') active_requests.labels(model=model).inc() try: # Build target URL target_url = f"{HOLYSHEEP_BASE_URL}/{full_path}" # Prepare headers headers = dict(request.headers) headers['Authorization'] = f'Bearer {HOLYSHEEP_API_KEY}' headers.pop('host', None) # Get request body body = await request.body() # Forward request to HolySheep async with httpx.AsyncClient(timeout=60.0) as client: response = await client.request( method=request.method, url=target_url, headers=headers, content=body if body else None ) # Calculate latency in milliseconds latency_ms = (time.perf_counter() - start_time) * 1000 # Record metrics status_code = str(response.status_code) request_counter.labels(model=model, status_code=status_code, endpoint=full_path).inc() latency_histogram.labels(model=model, endpoint=full_path).observe(latency_ms) # Track 5xx errors if response.status_code >= 500: error_type = f"http_{response.status_code}" error_5xx_counter.labels(model=model, error_type=error_type).inc() # Extract and record token usage from response try: response_data = response.json() usage = response_data.get('usage', {}) prompt_tokens = usage.get('prompt_tokens', 0) completion_tokens = usage.get('completion_tokens', 0) if prompt_tokens > 0: token_usage.labels(model=model, token_type='prompt').inc(prompt_tokens) cost = prompt_tokens * MODEL_PRICING.get(model, MODEL_PRICING['gpt-4.1'])['prompt'] cost_accumulator.labels(model=model).inc(cost) if completion_tokens > 0: token_usage.labels(model=model, token_type='completion').inc(completion_tokens) cost = completion_tokens * MODEL_PRICING.get(model, MODEL_PRICING['gpt-4.1'])['completion'] cost_accumulator.labels(model=model).inc(cost) except: pass # Response might not be JSON or have usage field return Response( content=response.content, status_code=response.status_code, headers=dict(response.headers) ) except httpx.TimeoutException: latency_ms = (time.perf_counter() - start_time) * 1000 error_5xx_counter.labels(model=model, error_type='timeout').inc() request_counter.labels(model=model, status_code='504', endpoint=full_path).inc() latency_histogram.labels(model=model, endpoint=full_path).observe(latency_ms) return JSONResponse(status_code=504, content={"error": "Gateway timeout"}) except Exception as e: latency_ms = (time.perf_counter() - start_time) * 1000 error_5xx_counter.labels(model=model, error_type='internal_error').inc() request_counter.labels(model=model, status_code='500', endpoint=full_path).inc() latency_histogram.labels(model=model, endpoint=full_path).observe(latency_ms) return JSONResponse(status_code=500, content={"error": str(e)}) finally: active_requests.labels(model=model).dec() @app.get("/metrics") async def prometheus_metrics(): """Prometheus metrics endpoint - automatically formatted by prometheus_client""" from prometheus_client import generate_latest, CONTENT_TYPE_LATEST return Response(content=generate_latest(), media_type=CONTENT_TYPE_LATEST) @app.get("/health") async def health_check(): """Health check endpoint""" return {"status": "healthy", "service": "holysheep-monitored-gateway"}

============================================

STARTUP

============================================

if __name__ == "__main__": import uvicorn # Start Prometheus metrics server on port 9090 start_http_server(9090) print("Prometheus metrics exposed on http://localhost:9090/metrics") # Start FastAPI on port 8000 uvicorn.run(app, host="0.0.0.0", port=8000)

Run: python metrics_server.py

Test: curl http://localhost:8000/health

Metrics: curl http://localhost:9090/metrics

2. Cấu hình Datadog Dashboard

Datadog là lựa chọn tuyệt vời cho production monitoring với alerting thông minh. Tôi sử dụng Datadog APM + Custom Metrics để track P95 latency và 5xx error rates.

# datadog_monitor.py - Datadog integration

Install: pip install datadog

from datadog import initialize, statsd import time import os

============================================

DATADOG CONFIGURATION

============================================

options = { 'api_key': os.getenv('DD_API_KEY', 'your-datadog-api-key'), 'app_key': os.getenv('DD_APP_KEY', 'your-datadog-app-key'), 'statsd_host': os.getenv('DD_AGENT_HOST', 'localhost'), 'statsd_port': 8125, } initialize(**options)

============================================

CUSTOM METRICS FUNCTIONS

============================================

def record_request_metrics(model: str, latency_ms: float, status_code: int, tokens_used: int = 0, cost_usd: float = 0.0): """ Record comprehensive request metrics to Datadog """ tags = [ f'model:{model}', f'status_code:{status_code}', 'service:holysheep-gateway' ] # Request count statsd.increment('holysheep.requests', tags=tags) # Latency histogram (Datadog will calculate P95, P99 automatically) statsd.histogram('holysheep.latency', latency_ms, tags=tags) # Token usage if tokens_used > 0: statsd.increment('holysheep.tokens', tokens_used, tags=tags + [f'type:{"prompt" if tokens_used < 1000 else "completion"}']) # Cost tracking if cost_usd > 0: statsd.gauge('holysheep.cost', cost_usd, tags=tags) # Error tracking if status_code >= 500: statsd.increment('holysheep.errors.5xx', tags=tags + [f'error_type:http_{status_code}']) elif status_code >= 400: statsd.increment('holysheep.errors.4xx', tags=tags) def record_gateway_health(model: str, is_healthy: bool, response_time_ms: float): """ Record gateway health status """ tags = [f'model:{model}', 'service:holysheep-gateway'] statsd.gauge('holysheep.gateway.health', 1 if is_healthy else 0, tags=tags) statsd.gauge('holysheep.gateway.response_time', response_time_ms, tags=tags)

============================================

ALERT THRESHOLDS CONFIGURATION

============================================

ALERT_CONFIG = { 'p95_latency_threshold_ms': 2000, # Alert if P95 > 2 seconds 'error_rate_threshold_percent': 5, # Alert if 5xx > 5% 'cost_hourly_threshold_usd': 100, # Alert if hourly cost > $100 'active_requests_threshold': 1000, # Alert if concurrent requests > 1000 }

============================================

MONITOR CLASS

============================================

class HolySheepMonitor: """ HolySheep Gateway Monitor with Datadog integration """ def __init__(self): self.metrics_buffer = [] self.buffer_size = 100 def check_gateway_health(self, model: str) -> dict: """Health check endpoint for HolySheep API""" import httpx start = time.perf_counter() try: response = httpx.get( f"https://api.holysheep.ai/v1/models", headers={"Authorization": f"Bearer {os.getenv('YOUR_HOLYSHEEP_API_KEY')}"}, timeout=10.0 ) response_time = (time.perf_counter() - start) * 1000 is_healthy = response.status_code == 200 record_gateway_health(model, is_healthy, response_time) return {'healthy': is_healthy, 'response_time_ms': response_time} except Exception as e: response_time = (time.perf_counter() - start) * 1000 record_gateway_health(model, False, response_time) return {'healthy': False, 'error': str(e), 'response_time_ms': response_time} def calculate_p95(self, latencies: list) -> float: """Calculate P95 from latency list""" if not latencies: return 0.0 sorted_latencies = sorted(latencies) index = int(len(sorted_latencies) * 0.95) return sorted_latencies[min(index, len(sorted_latencies) - 1)] def check_alerts(self, metrics: list) -> list: """Check alert conditions""" alerts = [] if not metrics: return alerts # Calculate P95 latencies = [m['latency_ms'] for m in metrics if 'latency_ms' in m] p95 = self.calculate_p95(latencies) if p95 > ALERT_CONFIG['p95_latency_threshold_ms']: alerts.append({ 'type': 'high_latency', 'severity': 'warning', 'message': f'P95 latency {p95:.0f}ms exceeds threshold {ALERT_CONFIG["p95_latency_threshold_ms"]}ms', 'p95_value': p95 }) # Error rate check error_count = sum(1 for m in metrics if m.get('status_code', 0) >= 500) total_count = len(metrics) error_rate = (error_count / total_count * 100) if total_count > 0 else 0 if error_rate > ALERT_CONFIG['error_rate_threshold_percent']: alerts.append({ 'type': 'high_error_rate', 'severity': 'critical', 'message': f'5xx error rate {error_rate:.1f}% exceeds threshold {ALERT_CONFIG["error_rate_threshold_percent"]}%', 'error_rate': error_rate }) return alerts

============================================

USAGE EXAMPLE

============================================

if __name__ == "__main__": monitor = HolySheepMonitor() # Check gateway health health = monitor.check_gateway_health('gpt-4.1') print(f"Gateway Health: {health}") # Record sample metrics record_request_metrics( model='gpt-4.1', latency_ms=150.5, status_code=200, tokens_used=500, cost_usd=0.004 ) print("Metrics sent to Datadog successfully!")

Run: python datadog_monitor.py

Environment variables needed:

DD_API_KEY, DD_APP_KEY, DD_AGENT_HOST, YOUR_HOLYSHEEP_API_KEY

3. Grafana Dashboard Configuration

Với Grafana, tôi tạo dashboard trực quan để track real-time performance. Kết hợp Prometheus làm data source.

# grafana_dashboard.json - Grafana Dashboard Configuration

Import this JSON to Grafana

{ "dashboard": { "title": "HolySheep AI Gateway Monitor", "uid": "holysheep-gateway-monitor", "timezone": "browser", "panels": [ { "title": "P95 Latency by Model", "type": "timeseries", "gridPos": {"x": 0, "y": 0, "w": 12, "h": 8}, "targets": [ { "expr": "histogram_quantile(0.95, sum(rate(holysheep_request_latency_ms_bucket[5m])) by (le, model))", "legendFormat": "P95 - {{model}}" } ], "fieldConfig": { "defaults": { "unit": "ms", "thresholds": { "mode": "absolute", "steps": [ {"color": "green", "value": null}, {"color": "yellow", "value": 1000}, {"color": "red", "value": 2000} ] } } }, "alert": { "name": "High P95 Latency Alert", "conditions": [ { "evaluator": {"params": [2000], "type": "gt"}, "operator": {"type": "and"}, "query": {"params": ["A", "5m", "now"]}, "reducer": {"type": "avg"} } ], "frequency": "1m", "handler": 1, "message": "HolySheep Gateway P95 latency exceeded 2000ms threshold" } }, { "title": "5xx Error Rate %", "type": "timeseries", "gridPos": {"x": 12, "y": 0, "w": 12, "h": 8}, "targets": [ { "expr": "sum(rate(holysheep_5xx_errors_total[5m])) by (model) / sum(rate(holysheep_requests_total[5m])) by (model) * 100", "legendFormat": "{{model}}" } ], "fieldConfig": { "defaults": { "unit": "percent", "thresholds": { "mode": "absolute", "steps": [ {"color": "green", "value": null}, {"color": "yellow", "value": 2}, {"color": "red", "value": 5} ] } } }, "alert": { "name": "High 5xx Error Rate Alert", "conditions": [ { "evaluator": {"params": [5], "type": "gt"}, "operator": {"type": "and"} } ], "frequency": "30s", "handler": 1, "message": "5xx error rate exceeded 5% threshold - Immediate action required" } }, { "title": "Request Volume by Model", "type": "bargauge", "gridPos": {"x": 0, "y": 8, "w": 8, "h": 6}, "targets": [ { "expr": "sum(rate(holysheep_requests_total[5m])) by (model)", "legendFormat": "{{model}}" } ], "fieldConfig": { "defaults": { "unit": "reqps", "color": {"mode": "palette-classic"} } } }, { "title": "Token Usage (Thousands)", "type": "timeseries", "gridPos": {"x": 8, "y": 8, "w": 8, "h": 6}, "targets": [ { "expr": "sum(rate(holysheep_tokens_total[1h])) by (model, token_type) / 1000", "legendFormat": "{{model}} - {{token_type}}" } ], "fieldConfig": { "defaults": { "unit": "short" } } }, { "title": "Hourly Cost (USD)", "type": "stat", "gridPos": {"x": 16, "y": 8, "w": 8, "h": 6}, "targets": [ { "expr": "sum(increase(holysheep_cost_usd[1h]))", "legendFormat": "Hourly Cost" } ], "fieldConfig": { "defaults": { "unit": "currencyUSD", "thresholds": { "mode": "absolute", "steps": [ {"color": "green", "value": null}, {"color": "yellow", "value": 50}, {"color": "red", "value": 100} ] } } } }, { "title": "Active Requests", "type": "gauge", "gridPos": {"x": 0, "y": 14, "w": 6, "h": 6}, "targets": [ { "expr": "sum(holysheep_active_requests)", "legendFormat": "Active" } ], "fieldConfig": { "defaults": { "unit": "short", "max": 1000, "thresholds": { "mode": "absolute", "steps": [ {"color": "green", "value": null}, {"color": "yellow", "value": 500}, {"color": "red", "value": 800} ] } } } }, { "title": "Error Breakdown", "type": "piechart", "gridPos": {"x": 6, "y": 14, "w": 6, "h": 6}, "targets": [ { "expr": "sum(holysheep_5xx_errors_total) by (error_type)", "legendFormat": "{{error_type}}" } ] } ], "time": {"from": "now-1h", "to": "now"}, "refresh": "10s" } }

============================================

PROMETHEUS ALERT RULES (prometheus_rules.yml)

============================================

groups:

- name: holysheep-alerts

rules:

- alert: HolySheepHighP95Latency

expr: histogram_quantile(0.95, sum(rate(holysheep_request_latency_ms_bucket[5m])) by (le)) > 2000

for: 5m

labels:

severity: warning

annotations:

summary: "High P95 latency detected on HolySheep Gateway"

description: "P95 latency is {{ $value }}ms (threshold: 2000ms)"

- alert: HolySheepHighErrorRate

expr: sum(rate(holysheep_5xx_errors_total[5m])) / sum(rate(holysheep_requests_total[5m])) > 0.05

for: 2m

labels:

severity: critical

annotations:

summary: "High 5xx error rate on HolySheep Gateway"

description: "Error rate is {{ $value | humanizePercentage }} (threshold: 5%)"

- alert: HolySheepCostSpike

expr: sum(increase(holysheep_cost_usd[1h])) > 100

for: 5m

labels:

severity: warning

annotations:

summary: "Hourly cost exceeded $100"

description: "Current hourly cost: ${{ $value }}"

4. Test Script - End-to-End Verification

Trước khi deploy lên production, tôi luôn chạy test script này để verify mọi thứ hoạt động đúng.

# test_monitoring.py - Comprehensive monitoring test

Install: pip install httpx prometheus_client pytest pytest-asyncio

import asyncio import httpx import time from prometheus_client import REGISTRY HOLYSHEEP_BASE_URL = "https://api.holysheep.ai/v1" API_KEY = "YOUR_HOLYSHEEP_API_KEY" # Replace with your key

============================================

TEST CONFIGURATION

============================================

TEST_MODELS = ['gpt-4.1', 'gemini-2.5-flash', 'deepseek-v3.2'] CONCURRENT_REQUESTS = 10 TOTAL_REQUESTS = 50 async def test_single_request(client: httpx.AsyncClient, model: str) -> dict: """Test single request to HolySheep""" start_time = time.perf_counter() try: response = await client.post( f"{HOLYSHEEP_BASE_URL}/chat/completions", headers={ "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json", "X-Model": model }, json={ "model": model, "messages": [{"role": "user", "content": "Count to 3"}], "max_tokens": 50, "temperature": 0.7 }, timeout=30.0 ) latency_ms = (time.perf_counter() - start_time) * 1000 return { 'success': response.status_code == 200, 'status_code': response.status_code, 'latency_ms': latency_ms, 'model': model, 'response': response.json() if response.status_code == 200 else None, 'error': None } except Exception as e: latency_ms = (time.perf_counter() - start_time) * 1000 return { 'success': False, 'status_code': 0, 'latency_ms': latency_ms, 'model': model, 'response': None, 'error': str(e) } async def run_load_test(): """Run concurrent load test""" print("=" * 60) print("HOLYSHEEP MONITORING TEST SUITE") print("=" * 60) async with httpx.AsyncClient() as client: # Test each model results = {'gpt-4.1': [], 'gemini-2.5-flash': [], 'deepseek-v3.2': []} for model in TEST_MODELS: print(f"\n🔄 Testing {model}...") tasks = [test_single_request(client, model) for _ in range(TOTAL_REQUESTS)] model_results = await asyncio.gather(*tasks) results[model] = model_results # Calculate statistics successful = [r for r in model_results if r['success']] failed = [r for r in model_results if not r['success']] latencies = [r['latency_ms'] for r in successful] latencies.sort() print(f" ✅ Success: {len(successful)}/{TOTAL_REQUESTS}") print(f" ❌ Failed: {len(failed)}/{TOTAL_REQUESTS}") if latencies: p50 = latencies[int(len(latencies) * 0.5)] p95 = latencies[int(len(latencies) * 0.95)] p99 = latencies[int(len(latencies) * 0.99)] avg = sum(latencies) / len(latencies) print(f" 📊 Latency Stats:") print(f" P50: {p50:.2f}ms") print(f" P95: {p95:.2f}ms") print(f" P99: {p99:.2f}ms") print(f" Avg: {avg:.2f}ms") # Check P95 threshold if p95 > 2000: print(f" ⚠️ WARNING: P95 ({p95:.0f}ms) exceeds 2000ms threshold!") # Check error rate error_rate = len(failed) / TOTAL_REQUESTS * 100 if error_rate > 5: print(f" 🚨 ERROR: Error rate ({error_rate:.1f}%) exceeds 5% threshold!") # Summary print("\n" + "=" * 60) print("SUMMARY") print("=" * 60) total_success = sum(len([r for r in results[m] if r['success']]) for m in TEST_MODELS) total_requests = TOTAL_REQUESTS * len(TEST_MODELS) overall_success_rate = total_success / total_requests * 100 print(f"Total Requests: {total_requests}") print(f"Success Rate: {overall_success_rate:.1f}%") # Check Prometheus metrics print("\n📈 Prometheus Metrics Check:") for metric in REGISTRY.collect(): print(f" - {metric.name}: {len(metric.samples)} samples") return results if __name__ == "__main__": # Run test results = asyncio.run(run_load_test()) print("\n✅ Test completed! Metrics should be available at http://localhost:9090/metrics")

Run: python test_monitoring.py

Verify: curl http://localhost:9090/metrics | grep holysheep

5. Alerting Rules - P95 Latency & 5xx Error Rate

Đây là phần quan trọng nhất - alerting rules giúp bạn biết vấn đề trước khi người dùng phàn nàn.

# ============================================

ALERT RULES CONFIGURATION

============================================

DATADOG MONITOR CONFIGURATION (datadog_monitors.yml)

monitors: - name: "HolySheep P95 Latency Alert" type: "metric alert" query: | p95(holysheep.latency{service:holysheep-gateway}.as_count(), 5m) > 2000 message: | 🚨 ALERT: HolySheep Gateway High Latency P95 Latency: {{ value }}ms Threshold: 2000ms Models affected: {{ tags.model }} Action Required: 1. Check model provider status 2. Verify network connectivity 3. Check for rate limiting 4. Consider scaling gateway instances @slack-holysheep-alerts @pagerduty-holysheep tags: - "service:holysheep-gateway" - "severity:warning" - name: "HolySheep 5xx Error Rate Alert" type: "metric alert" query: | (sum(holysheep.errors.5xx{service:holysheep-gateway}.as_count()) / sum(holysheep.requests{service:holysheep-gateway}.as_count())) * 100 > 5 message: | 🚨🚨 CRITICAL: HolySheep Gateway High Error Rate 5xx Error Rate: {{ value }}% Threshold: 5% Error Types: {{#each groups}} - {{ error_type }}: {{ this }} {{/each}} IMMEDIATE ACTION REQUIRED! 1. Check HolySheep API status at https://status.holysheep.ai 2. Check model provider dashboards 3. Enable fallback routing 4. Scale up gateway if needed @pagerduty-holysheep-critical @slack-holysheep-alerts tags: - "service:holysheep-gateway" - "severity:critical"