When you build applications that talk to AI APIs, things will occasionally go wrong. Maybe the internet connection drops, maybe the API has temporary issues, or maybe you've hit a rate limit. Without proper error handling, your entire application crashes and your users see confusing error messages. That's where graceful degradation comes in.

In this tutorial, I will walk you through building robust error handling for AI API calls from absolute scratch. No prior API experience needed. By the end, you'll know exactly how to keep your application running smoothly even when things break.

What Is Graceful Degradation?

Imagine you're driving a car with a GPS. If the GPS suddenly stops working, graceful degradation means your car doesn't stop completely—you can still drive using a backup map or your memory of the route. Your application works the same way: when the primary AI service fails, it falls back to a simpler solution instead of breaking entirely.

For AI APIs, graceful degradation typically means:

Understanding API Errors: A Beginner's Overview

Before we write code, let's understand what can go wrong. When you call an AI API, several things might happen:

Setting Up Your First Error-Handled API Call

Let's start with a complete working example using Python. We'll use HolySheep AI's API, which offers rates of ¥1=$1 (saving 85%+ compared to ¥7.3 alternatives) with sub-50ms latency and free credits on signup.

Step 1: Install Required Packages

Open your terminal and install the necessary library:

pip install requests

Step 2: Your First Error-Handled Script

Create a new file called gemini_error_handling.py and paste this complete working example:

import requests
import time
import json
from datetime import datetime


Your HolySheep AI API credentials


Sign up at https://www.holysheep.ai/register to get your free credits

API_KEY = "YOUR_HOLYSHEEP_API_KEY"
BASE_URL = "https://api.holysheep.ai/v1"

class GeminiAPIError(Exception):
    """Custom exception for Gemini API errors"""
    pass

def send_prompt_to_gemini(prompt, max_retries=3):
    """
    Send a prompt to Gemini with full error handling.
    
    This function demonstrates graceful degradation - even when
    things go wrong, it handles them smoothly without crashing.
    """
    
    # Prepare the request payload
    payload = {
        "model": "gemini-2.5-flash",
        "messages": [
            {"role": "user", "content": prompt}
        ],
        "temperature": 0.7,
        "max_tokens": 1000
    }
    
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    
    # Retry logic - we'll try multiple times before giving up
    for attempt in range(max_retries):
        try:
            print(f"Attempt {attempt + 1}/{max_retries}: Sending request...")
            
            # Make the API call with a 30-second timeout
            response = requests.post(
                f"{BASE_URL}/chat/completions",
                headers=headers,
                json=payload,
                timeout=30
            )
            
            # Check for HTTP errors
            response.raise_for_status()
            
            # Parse successful response
            data = response.json()
            
            # Extract the AI's response
            if "choices" in data and len(data["choices"]) > 0:
                ai_message = data["choices"][0]["message"]["content"]
                print("✓ Success! Received response from AI")
                return {
                    "status": "success",
                    "response": ai_message,
                    "model": data.get("model", "unknown"),
                    "usage": data.get("usage", {})
                }
            else:
                raise GeminiAPIError("Unexpected response format from API")
                
        except requests.exceptions.Timeout:
            # The API took too long to respond
            print(f"⚠ Attempt {attempt + 1}: Request timed out (30 seconds)")
            if attempt < max_retries - 1:
                wait_time = 2 ** attempt  # Exponential backoff: 1, 2, 4 seconds
                print(f"   Waiting {wait_time} seconds before retry...")
                time.sleep(wait_time)
            else:
                return {
                    "status": "timeout",
                    "response": "The request took too long. Please try again.",
                    "error_type": "timeout"
                }
                
        except requests.exceptions.ConnectionError as e:
            # Network connectivity issues
            print(f"⚠ Attempt {attempt + 1}: Connection error - {str(e)}")
            if attempt < max_retries - 1:
                wait_time = 2 ** attempt
                print(f"   Waiting {wait_time} seconds before retry...")
                time.sleep(wait_time)
            else:
                return {
                    "status": "network_error",
                    "response": "Unable to connect to the AI service. Please check your internet connection.",
                    "error_type": "connection_error"
                }
                
        except requests.exceptions.HTTPError as e:
            # HTTP errors (4xx, 5xx status codes)
            status_code = e.response.status_code
            
            if status_code == 401:
                return {
                    "status": "auth_error",
                    "response": "Authentication failed. Please check your API key.",
                    "error_type": "unauthorized",
                    "status_code": status_code
                }
            elif status_code == 429:
                # Rate limit hit
                retry_after = e.response.headers.get("Retry-After", 60)
                print(f"⚠ Rate limited. Suggested wait: {retry_after} seconds")
                return {
                    "status": "rate_limited",
                    "response": "Too many requests. Please wait before trying again.",
                    "error_type": "rate_limit",
                    "retry_after": int(retry_after)
                }
            elif status_code >= 500:
                # Server error from the API provider
                print(f"⚠ Server error: {status_code}")
                if attempt < max_retries - 1:
                    wait_time = 2 ** attempt
                    print(f"   Waiting {wait_time} seconds before retry...")
                    time.sleep(wait_time)
                else:
                    return {
                        "status": "server_error",
                        "response": "The AI service is temporarily unavailable. Please try again later.",
                        "error_type": "server_error",
                        "status_code": status_code
                    }
            else:
                return {
                    "status": "http_error",
                    "response": f"Request failed with status {status_code}.",
                    "error_type": "http_error",
                    "status_code": status_code
                }
                
        except requests.exceptions.RequestException as e:
            # Catch-all for any other request-related errors
            print(f"⚠ Attempt {attempt + 1}: Request failed - {str(e)}")
            if attempt < max_retries - 1:
                wait_time = 2 ** attempt
                print(f"   Waiting {wait_time} seconds before retry...")
                time.sleep(wait_time)
            else:
                return {
                    "status": "error",
                    "response": "An unexpected error occurred. Please try again.",
                    "error_type": str(type(e).__name__)
                }
                
        except (KeyError, ValueError, json.JSONDecodeError) as e:
            # Data parsing errors
            return {
                "status": "parse_error",
                "response": "Received an unexpected response from the server.",
                "error_type": "parse_error",
                "details": str(e)
            }
    
    # This should never be reached, but just in case
    return {
        "status": "max_retries_exceeded",
        "response": "Maximum retry attempts reached. Please try again later.",
        "error_type": "max_retries"
    }


def display_response(result):
    """Display the API response in a user-friendly way"""
    print("\n" + "=" * 50)
    print("RESPONSE RESULT")
    print("=" * 50)
    print(f"Status: {result['status']}")
    print(f"Message: {result['response']}")
    if 'usage' in result and result['usage']:
        print(f"Token Usage: {result['usage']}")
    print("=" * 50 + "\n")



Example usage

if __name__ == "__main__":
    print("=" * 50)
    print("GEMINI API ERROR HANDLING DEMO")
    print("=" * 50)
    
    # Test with a simple prompt
    test_prompt = "Explain what an API is in one sentence."
    
    print(f"\nSending prompt: '{test_prompt}'\n")
    result = send_prompt_to_gemini(test_prompt)
    display_response(result)

Building a Production-Ready Error Handler

Now let's create a more advanced version that includes fallback responses, caching, and comprehensive logging. This is the kind of robust system you'd use in a real production application.

import requests
import json
import time
import logging
from datetime import datetime, timedelta
from functools import wraps
from typing import Optional, Dict, Any, Callable


Configure logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)


Configuration

API_KEY = "YOUR_HOLYSHEEP_API_KEY"
BASE_URL = "https://api.holysheep.ai/v1"


Response cache to avoid repeated API calls

response_cache: Dict[str, Dict[str, Any]] = {}
CACHE_EXPIRY_MINUTES = 60


Fallback responses when everything fails

FALLBACK_RESPONSES = [
    "I understand you're asking something. Unfortunately, I'm experiencing technical difficulties right now. Could you please try again in a few moments?",
    "Thank you for your patience. I'm currently unable to process your request due to high demand. Please try again shortly.",
    "I appreciate your question! Right now, I'm dealing with some connectivity issues. Please try again, and I'll be ready to help!"
]


class APIErrorHandler:
    """
    A comprehensive error handler for AI API calls.
    Implements graceful degradation with retries, caching, and fallbacks.
    """
    
    def __init__(self, api_key: str, base_url: str):
        self.api_key = api_key
        self.base_url = base_url
        self.request_count = 0
        self.error_log = []
        
    def _log_error(self, error_type: str, details: str, context: Dict = None):
        """Log errors with timestamps for later analysis"""
        error_entry = {
            "timestamp": datetime.now().isoformat(),
            "error_type": error_type,
            "details": details,
            "context": context or {}
        }
        self.error_log.append(error_entry)
        logger.error(f"API Error [{error_type}]: {details}")
        
    def _get_cached_response(self, prompt_hash: str) -> Optional[str]:
        """Check if we have a recent cached response"""
        if prompt_hash in response_cache:
            cached = response_cache[prompt_hash]
            if datetime.now() < cached["expires_at"]:
                logger.info("Using cached response")
                return cached["response"]
            else:
                del response_cache[prompt_hash]
        return None
    
    def _cache_response(self, prompt_hash: str, response: str):
        """Store a successful response in cache"""
        response_cache[prompt_hash] = {
            "response": response,
            "expires_at": datetime.now() + timedelta(minutes=CACHE_EXPIRY_MINUTES)
        }
        logger.info(f"Cached response for prompt hash: {prompt_hash[:8]}...")
    
    def _get_fallback_response(self) -> str:
        """Return a random fallback message"""
        import random
        return random.choice(FALLBACK_RESPONSES)
    
    def call_with_fallback(
        self,
        prompt: str,
        use_cache: bool = True,
        use_fallback: bool = True,
        max_retries: int = 3
    ) -> Dict[str, Any]:
        """
        Main method: Call API with full graceful degradation support.
        
        Features:
        1. Automatic retries with exponential backoff
        2. Response caching to reduce API calls
        3. Fallback responses when everything fails
        4. Comprehensive error logging
        """
        
        # Create a simple hash of the prompt for caching
        prompt_hash = str(hash(prompt))
        
        # Step 1: Check cache
        if use_cache:
            cached = self._get_cached_response(prompt_hash)
            if cached:
                return {
                    "status": "success",
                    "response": cached,
                    "source": "cache"
                }
        
        # Step 2: Try the actual API call with retries
        for attempt in range(max_retries):
            self.request_count += 1
            
            try:
                payload = {
                    "model": "gemini-2.5-flash",
                    "messages": [{"role": "user", "content": prompt}],
                    "temperature": 0.7,
                    "max_tokens": 1000
                }
                
                headers = {
                    "Authorization": f"Bearer {self.api_key}",
                    "Content-Type": "application/json"
                }
                
                response = requests.post(
                    f"{self.base_url}/chat/completions",
                    headers=headers,
                    json=payload,
                    timeout=30
                )
                
                # Handle HTTP errors
                if response.status_code == 429:
                    wait_time = int(response.headers.get("Retry-After", 60))
                    self._log_error("rate_limit", f"Rate limited, wait {wait_time}s")
                    
                    if attempt < max_retries - 1:
                        logger.info(f"Waiting {wait_time}s before retry...")
                        time.sleep(min(wait_time, 60))  # Cap at 60 seconds
                        continue
                    else:
                        raise Exception("Rate limit exceeded after retries")
                        
                elif response.status_code >= 500:
                    self._log_error("server_error", f"Server error: {response.status_code}")
                    wait_time = 2 ** attempt
                    if attempt < max_retries - 1:
                        time.sleep(wait_time)
                        continue
                    else:
                        raise Exception(f"Server error after {max_retries} attempts")
                        
                elif response.status_code == 401:
                    self._log_error("auth_error", "Invalid API key")
                    return {
                        "status": "error",
                        "response": "Authentication failed. Please check your API key configuration.",
                        "error_type": "unauthorized"
                    }
                
                response.raise_for_status()
                
                # Parse successful response
                data = response.json()
                
                if "choices" in data and len(data["choices"]) > 0:
                    ai_response = data["choices"][0]["message"]["content"]
                    
                    # Cache successful response
                    if use_cache:
                        self._cache_response(prompt_hash, ai_response)
                    
                    return {
                        "status": "success",
                        "response": ai_response,
                        "source": "api",
                        "model": data.get("model"),
                        "usage": data.get("usage", {})
                    }
                    
            except requests.exceptions.Timeout:
                self._log_error("timeout", f"Request timed out on attempt {attempt + 1}")
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)
                    continue
                    
            except requests.exceptions.ConnectionError as e:
                self._log_error("connection", str(e))
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)
                    continue
                    
            except Exception as e:
                self._log_error("unknown", str(e), {"attempt": attempt + 1})
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)
                    continue
        
        # Step 3: Everything failed - use fallback or cached response
        if use_fallback:
            logger.warning("All API attempts failed, using fallback response")
            return {
                "status": "fallback",
                "response": self._get_fallback_response(),
                "source": "fallback"
            }
        
        return {
            "status": "error",
            "response": "Unable to complete your request. Please try again later.",
            "error_type": "max_retries_exceeded"
        }
    
    def get_error_summary(self) -> Dict[str, Any]:
        """Get a summary of all errors that occurred"""
        error_counts = {}
        for error in self.error_log:
            error_type = error["error_type"]
            error_counts[error_type] = error_counts.get(error_type, 0) + 1
            
        return {
            "total_requests": self.request_count,
            "total_errors": len(self.error_log),
            "error_breakdown": error_counts,
            "recent_errors": self.error_log[-10:]  # Last 10 errors
        }



Decorator version for function-based usage

def graceful_degradation(
    max_retries: int = 3,
    timeout: int = 30,
    fallback_response: str = "I'm sorry, I'm having trouble right now. Please try again."
):
    """
    Decorator that adds graceful degradation to any API calling function.
    
    Usage:
        @graceful_degradation(max_retries=3, fallback_response="Default message")
        def my_api_call():
            # Your API call code here
            pass
    """
    def decorator(func: Callable) -> Callable:
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_retries):
                try:
                    return func(*args, **kwargs)
                except requests.exceptions.Timeout:
                    logger.warning(f"Timeout on attempt {attempt + 1}")
                    if attempt < max_retries - 1:
                        time.sleep(2 ** attempt)
                except requests.exceptions.ConnectionError as e:
                    logger.warning(f"Connection error on attempt {attempt + 1}: {e}")
                    if attempt < max_retries - 1:
                        time.sleep(2 ** attempt)
                except Exception as e:
                    logger.error(f"Unexpected error: {e}")
                    break
            
            # All retries exhausted - return fallback
            return {
                "status": "fallback",
                "response": fallback_response,
                "error_type": "max_retries_exceeded"
            }
        return wrapper
    return decorator



Example: Using the error handler

if __name__ == "__main__":
    # Initialize the error handler
    handler = APIErrorHandler(API_KEY, BASE_URL)
    
    # Make API calls with full error handling
    test_prompts = [
        "What is machine learning?",
        "Explain neural networks simply.",
        "How do APIs work?"
    ]
    
    print("\n" + "=" * 60)
    print("PRODUCTION-READY ERROR HANDLING DEMO")
    print("=" * 60 + "\n")
    
    for prompt in test_prompts:
        print(f"Asking: {prompt}")
        result = handler.call_with_fallback(prompt)
        print(f"Status: {result['status']} | Source: {result.get('source', 'N/A')}")
        print(f"Response: {result['response'][:80]}...\n")
    
    # Print error summary
    summary = handler.get_error_summary()
    print("\n" + "=" * 60)
    print("ERROR SUMMARY")
    print("=" * 60)
    print(f"Total Requests: {summary['total_requests']}")
    print(f"Total Errors: {summary['total_errors']}")
    if summary['error_breakdown']:
        print("Error Breakdown:", summary['error_breakdown'])

The Retry Logic Explained

One of the key concepts in graceful degradation is the retry mechanism. Notice we use something called "exponential backoff." Here's how it works:

This approach prevents overwhelming a struggling server while still giving transient errors (like temporary network hiccups) a chance to resolve themselves. The API call cost is minimal with HolySheep AI's pricing at just $2.50 per million tokens for Gemini 2.5 Flash, so retrying is financially reasonable.

When to Use Each Error Handling Strategy

Not every error needs the same response. Here's a quick guide:

Building Your Own Error Dashboard

For production applications, you want visibility into what's going wrong. Here's a simple monitoring approach you can add to any error handler:

# Add this to your error handler class
def log_error_dashboard(self, error_data: Dict):
    """
    Send errors to a monitoring system (could be Slack, email, or a logging service)
    """
    dashboard_entry = {
        "timestamp": datetime.now().isoformat(),
        "error_type": error_data.get("error_type"),
        "status_code": error_data.get("status_code"),
        "user_visible_message": error_data.get("response"),
        "severity": self._calculate_severity(error_data.get("error_type"))
    }
    
    # In production, you'd send this to Datadog, Sentry, or similar
    print(f"[DASHBOARD] {json.dumps(dashboard_entry, indent=2)}")
    
    return dashboard_entry

def _calculate_severity(self, error_type: str) -> str:
    """Determine how serious an error is"""
    critical = ["auth_error", "server_error"]
    warning = ["rate_limit", "timeout", "connection"]
    info = ["parse_error"]
    
    if error_type in critical:
        return "CRITICAL"
    elif error_type in warning:
        return "WARNING"
    return "INFO"

Common Errors and Fixes

Here are the most frequent issues beginners face when working with AI APIs, along with their solutions:

1. "401 Unauthorized" - Invalid API Key

# ❌ WRONG - Missing or incorrect API key
headers = {
    "Authorization": "Bearer YOUR_KEY_HERE"  # Make sure this matches exactly
}


✅ CORRECT - Verify your key is correct and complete

API_KEY = "hs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  # Full key from HolySheep dashboard

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}


If you're getting 401 errors, double-check:


1. The key is copied completely (no missing characters)


2. No extra spaces before or after the key


3. You're using the correct key for the environment (test vs production)

2. "429 Rate Limited" - Too Many Requests

# ❌ WRONG - Making requests as fast as possible
for i in range(1000):
    response = send_request()  # Will definitely hit rate limit


✅ CORRECT - Implement rate limiting in your code

import time
from collections import deque

class RateLimiter:
    def __init__(self, max_requests: int, time_window: int):
        self.max_requests = max_requests
        self.time_window = time_window
        self.requests = deque()
    
    def wait_if_needed(self):
        now = time.time()
        
        # Remove requests outside the time window
        while self.requests and self.requests[0] < now - self.time_window:
            self.requests.popleft()
        
        # If we've hit the limit, wait
        if len(self.requests) >= self.max_requests:
            sleep_time = self.time_window - (now - self.requests[0])
            print(f"Rate limit reached. Waiting {sleep_time:.1f} seconds...")
            time.sleep(sleep_time)
        
        # Record this request
        self.requests.append(time.time())


Usage

limiter = RateLimiter(max_requests=60, time_window=60)  # 60 requests per minute

for prompt in many_prompts:
    limiter.wait_if_needed()  # Ensures you stay within limits
    response = send_request(prompt)

3. Timeout Errors - Requests Taking Too Long

# ❌ WRONG - No timeout specified (can hang forever)
response = requests.post(url, headers=headers, json=payload)


✅ CORRECT - Set reasonable timeouts

response = requests.post(
    url,
    headers=headers,
    json=payload,
    timeout=30  # Total timeout in seconds
)


✅ EVEN BETTER - Separate connect and read timeouts

response = requests.post(
    url,
    headers=headers,
    json=payload,
    timeout=(5, 30)  # 5 seconds to connect, 30 seconds to read response
)


✅ BEST - Handle timeout gracefully

try:
    response = requests.post(url, headers=headers, json=payload, timeout=30)
except requests.exceptions.Timeout:
    print("The request took too long. Consider:")
    print("  - Reducing the max_tokens parameter")
    print("  - Using a faster model (like Gemini 2.5 Flash)")
    print("  - Adding a retry mechanism")
    return fallback_response

4. JSON Parse Errors - Invalid Response Format

# ❌ WRONG - Assuming response is always valid JSON
response = requests.post(url, headers=headers, json=payload)
data = response.json()  # Can crash if response is HTML or empty


✅ CORRECT - Validate response before parsing

response = requests.post(url, headers=headers, json=payload)
response.raise_for_status()  # Check for HTTP errors first

try:
    data = response.json()
except json.JSONDecodeError as e:
    print(f"Failed to parse response: {e}")
    print(f"Raw response: {response.text[:200]}...")  # See what we actually got
    return fallback_response


Verify expected structure

if "choices" not in data or not data["choices"]:
    print("Unexpected response structure")
    return fallback_response

content = data["choices"][0]["message"]["content"]

Best Practices for Production Applications

After working with dozens of production AI applications, here's what I've learned about error handling:

Conclusion

Error handling might seem like extra work, but it's what separates hobby projects from production applications. The patterns we covered—retries with exponential backoff, caching, fallback responses, and comprehensive logging—will keep your applications running smoothly even when the unexpected happens.

I have implemented these exact error handling strategies in applications processing thousands of daily requests, and the difference is remarkable. Users rarely see errors, and when they do, they get helpful guidance instead of cryptic technical messages.

Start with the simple error handler we built first, then gradually add features like caching, monitoring, and circuit breakers as your application grows.

👉 Sign up for HolySheep AI — free credits on registration

Related Resources

Related Articles