When I first started working with AI APIs, I made a classic mistake—I assumed all API responses would come back in one piece. I was wrong. Within my first week, I hit a wall when trying to process a large document analysis that returned thousands of tokens. The response kept timing out, and I had no idea how to piece it back together. That frustration led me to understand pagination, and today I'm going to save you from that same headache.

This tutorial walks you through pagination of AI model outputs from scratch. Whether you're building a chatbot that processes long conversations, an analysis tool that generates extensive reports, or simply trying to understand why your API responses are split across multiple pages, you'll find your answers here. We'll use the HolySheep AI API as our example, which delivers sub-50ms latency at a fraction of the cost of mainstream providers.

What is Pagination and Why Does It Matter for AI Outputs?

Pagination is the technique of breaking large datasets into manageable chunks called "pages." In the context of AI model outputs, this becomes crucial for several reasons:

HolySheep AI's API handles pagination seamlessly with automatic rate limiting at ¥1=$1 pricing (saving 85%+ compared to ¥7.3 rates), making it economical to experiment and learn these concepts without burning through your budget.

Understanding the Core Pagination Concepts

Key Parameters You'll Encounter

Before diving into code, let's understand the fundamental parameters that control pagination:

[Screenshot hint: Imagine a visual showing how "cursor" points to position 0, then position 10, then position 20 as you page through results]

Step-by-Step: Implementing Pagination with HolySheep AI

Prerequisites

You'll need:

Setting Up Your Environment

First, install the required library. Open your terminal and run:

pip install requests

Create a new Python file called pagination_tutorial.py and add your configuration:

import requests
import json


HolySheep AI Configuration

BASE_URL = "https://api.holysheep.ai/v1"
API_KEY = "YOUR_HOLYSHEEP_API_KEY"  # Replace with your actual key

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

def make_request(endpoint, params=None):
    """Helper function to make API requests to HolySheep AI."""
    url = f"{BASE_URL}/{endpoint}"
    response = requests.get(url, headers=headers, params=params)
    
    if response.status_code == 200:
        return response.json()
    else:
        print(f"Error {response.status_code}: {response.text}")
        return None

Example 1: Paginating Through Chat Completions

Let's start with a practical scenario—retrieving a long conversation history using pagination. This is common when building chatbots that need to display message history.

# Paginate through chat completions with automatic cursor handling
def get_all_chat_completions(conversation_id, max_pages=100):
    """
    Retrieve all chat completions for a conversation.
    Handles pagination automatically until all results are fetched.
    """
    all_messages = []
    cursor = None
    page_count = 0
    
    while page_count < max_pages:
        # Build request parameters
        params = {
            "conversation_id": conversation_id,
            "limit": 50  # Fetch 50 messages per page
        }
        
        # Add cursor if we have one (for subsequent pages)
        if cursor:
            params["cursor"] = cursor
        
        # Make the API request
        result = make_request("chat/completions", params)
        
        if not result:
            break
            
        # Extract messages from current page
        messages = result.get("data", [])
        all_messages.extend(messages)
        
        # Check if there are more pages
        has_more = result.get("has_more", False)
        cursor = result.get("next_cursor")
        
        print(f"Page {page_count + 1}: Retrieved {len(messages)} messages")
        
        if not has_more or not cursor:
            print(f"Reached end of results after {page_count + 1} pages")
            break
            
        page_count += 1
    
    return all_messages


Usage example


conversation_messages = get_all_chat_completions("conv_12345")


print(f"Total messages retrieved: {len(conversation_messages)}")

Understanding the Response Structure

When you make a paginated request to HolySheep AI, here's what a typical response looks like:

{
  "object": "list",
  "data": [
    {
      "id": "msg_001",
      "role": "user",
      "content": "Hello, explain pagination",
      "created_at": 1709824000
    },
    {
      "id": "msg_002",
      "role": "assistant", 
      "content": "Pagination is a technique to handle large datasets...",
      "created_at": 1709824010
    }
  ],
  "has_more": true,
  "next_cursor": "eyJpZCI6Im1zZ18wMDMifQ==",
  "total_count": 156
}

[Screenshot hint: Visualize a response object with the "data" array highlighted, "has_more" showing true, and "next_cursor" containing the base64-encoded position marker]

Example 2: Streaming Large Document Analysis

Now let's tackle a more advanced scenario—analyzing large documents by breaking them into chunks and processing each chunk through pagination. This is particularly useful when you need to process PDFs, lengthy transcripts, or large datasets.

import base64

def analyze_large_document(document_text, chunk_size=2000):
    """
    Break a large document into chunks and analyze each through pagination.
    Demonstrates both input chunking and output pagination.
    """
    # Split document into manageable chunks
    chunks = [document_text[i:i+chunk_size] 
              for i in range(0, len(document_text), chunk_size)]
    
    all_analyses = []
    
    print(f"Document split into {len(chunks)} chunks for processing")
    
    for chunk_index, chunk in enumerate(chunks):
        print(f"\nProcessing chunk {chunk_index + 1}/{len(chunks)}...")
        
        # Prepare the request with chunk context
        payload = {
            "model": "gpt-4.1",  # Using HolySheep AI's pricing: $8/MTok
            "messages": [
                {
                    "role": "system",
                    "content": "You are a document analysis assistant. Provide structured insights."
                },
                {
                    "role": "user", 
                    "content": f"Analyze this section (part {chunk_index + 1} of {len(chunks)}):\n\n{chunk}"
                }
            ],
            "max_tokens": 1000,
            "temperature": 0.3
        }
        
        # Make request to HolySheep AI
        response = requests.post(
            f"{BASE_URL}/chat/completions",
            headers=headers,
            json=payload
        )
        
        if response.status_code == 200:
            result = response.json()
            analysis = result["choices"][0]["message"]["content"]
            all_analyses.append({
                "chunk_index": chunk_index,
                "analysis": analysis,
                "tokens_used": result.get("usage", {}).get("total_tokens", 0)
            })
            print(f"  ✓ Chunk {chunk_index + 1} analyzed ({result.get('usage', {}).get('total_tokens', 0)} tokens)")
        else:
            print(f"  ✗ Error processing chunk {chunk_index + 1}: {response.text}")
    
    return all_analyses


Usage example with sample document

sample_document = """
This is a sample large document that would be too long for a single API call.
In real usage, this would contain thousands of words or tokens that need
to be processed in chunks. The pagination approach ensures each piece gets
properly analyzed while staying within model limits.
""".strip()


analyses = analyze_large_document(sample_document, chunk_size=500)

Advanced Pagination Patterns

Cursor-Based vs Offset-Based Pagination

There are two main pagination strategies you'll encounter:

[Screenshot hint: Show two side-by-side diagrams—left showing cursor jumping directly to page 3, right showing offset scanning through all records to reach page 3]

HolySheep AI uses cursor-based pagination exclusively, which is optimal for maintaining sub-50ms latency even with large result sets.

Implementing a Reusable Pagination Iterator

For cleaner code in production applications, wrap pagination in a reusable iterator class:

class PaginatedResults:
    """
    A reusable iterator class that handles pagination automatically.
    Yields items one by one while fetching pages as needed.
    """
    
    def __init__(self, api_endpoint, initial_params, api_key):
        self.base_url = f"https://api.holysheep.ai/v1/{api_endpoint}"
        self.params = initial_params
        self.api_key = api_key
        self.current_page = []
        self.current_index = 0
        self.cursor = None
        self.has_more = True
        
    def _fetch_page(self):
        """Fetch the next page of results from the API."""
        headers = {
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        }
        
        params = self.params.copy()
        if self.cursor:
            params["cursor"] = self.cursor
            
        response = requests.get(self.base_url, headers=headers, params=params)
        
        if response.status_code != 200:
            raise Exception(f"API request failed: {response.text}")
            
        data = response.json()
        self.current_page = data.get("data", [])
        self.current_index = 0
        self.has_more = data.get("has_more", False)
        self.cursor = data.get("next_cursor")
        
        return self.current_page
    
    def __iter__(self):
        return self
    
    def __next__(self):
        # Fetch new page if needed
        if self.current_index >= len(self.current_page):
            if not self.has_more:
                raise StopIteration
            self._fetch_page()
            
        item = self.current_page[self.current_index]
        self.current_index += 1
        return item


Usage example - iterate through all results naturally

def process_all_results():
    """Process all results using the iterator pattern."""
    iterator = PaginatedResults(
        api_endpoint="chat/completions",
        initial_params={"conversation_id": "conv_123", "limit": 100},
        api_key="YOUR_HOLYSHEEP_API_KEY"
    )
    
    processed_count = 0
    for item in iterator:
        # Process each item individually
        print(f"Processing: {item['id']}")
        processed_count += 1
        
        # Add your processing logic here
        if processed_count >= 1000:  # Example limit
            print("Reached processing limit")
            break
            
    return processed_count

Real-World Use Cases

Building a Conversation History Viewer

One common application is displaying chat history with infinite scrolling. Here's how you might implement this in a web application:

# JavaScript example for frontend pagination handling
async function loadConversationHistory(conversationId, cursor = null) {
    const params = new URLSearchParams({
        conversation_id: conversationId,
        limit: 20
    });
    
    if (cursor) {
        params.append('cursor', cursor);
    }
    
    const response = await fetch(
        https://api.holysheep.ai/v1/chat/completions?${params},
        {
            headers: {
                'Authorization': Bearer ${userApiKey},
                'Content-Type': 'application/json'
            }
        }
    );
    
    const data = await response.json();
    
    return {
        messages: data.data,
        hasMore: data.has_more,
        nextCursor: data.next_cursor
    };
}

// Infinite scroll implementation
let currentCursor = null;
let hasMoreMessages = true;

async function loadMoreMessages() {
    if (!hasMoreMessages) return;
    
    const { messages, hasMore, nextCursor } = await loadConversationHistory(
        'conv_12345', 
        currentCursor
    );
    
    // Append new messages to the UI
    messages.forEach(msg => {
        appendMessageToUI(msg);
    });
    
    currentCursor = nextCursor;
    hasMoreMessages = hasMore;
    
    // Update scroll listener
    if (hasMoreMessages) {
        setupScrollListener();
    }
}

Pricing and Cost Management with Pagination

Understanding how pagination affects your costs is essential for budget management. Here's a comparison of major providers' 2026 pricing:

HolySheep AI offers competitive pricing at ¥1=$1 with WeChat and Alipay payment options, supporting all major models while maintaining less than 50ms latency. When you paginate requests effectively, you minimize wasted tokens on oversized responses that get truncated anyway.

[Screenshot hint: A cost comparison table showing tokens used per page size, with HolySheep highlighted as the most cost-efficient option]

Common Errors and Fixes

Throughout my journey learning pagination, I've encountered numerous errors. Here are the most common issues and their solutions:

Error 1: "Invalid cursor format" or "Cursor expired"

Cursors can expire or become invalid if you wait too long between page requests, or if the underlying data changes.

# ❌ BROKEN: Using stale cursor
cursor = "eyJpZCI6Im1zZ18wMDMifQ=="  # Retrieved 2 hours ago
response = make_request("chat/completions", {"cursor": cursor})


✅ FIXED: Implement cursor validation and retry

def fetch_with_retry(endpoint, params, max_retries=3):
    """Fetch with automatic cursor refresh on expiration."""
    for attempt in range(max_retries):
        response = make_request(endpoint, params)
        
        if response and "error" not in response:
            return response
            
        # Check if cursor expired
        if response and "cursor" in str(response):
            print(f"Cursor expired, retrying without cursor...")
            if "cursor" in params:
                del params["cursor"]  # Remove stale cursor
                
        # Exponential backoff
        time.sleep(2 ** attempt)
        
    return None  # All retries failed

Error 2: "Rate limit exceeded" during pagination loops

Making rapid successive requests triggers rate limiting, especially when paginating through many pages.

# ❌ BROKEN: No rate limiting between requests
for cursor in cursors:
    response = make_request(endpoint, {"cursor": cursor})  # Too fast!


✅ FIXED: Implement rate limiting with request throttling

import time
from datetime import datetime, timedelta

class RateLimitedClient:
    def __init__(self, requests_per_minute=60):
        self.requests_per_minute = requests_per_minute
        self.request_times = []
        
    def wait_if_needed(self):
        """Ensure we don't exceed rate limits."""
        now = datetime.now()
        cutoff = now - timedelta(minutes=1)
        
        # Remove old timestamps
        self.request_times = [t for t in self.request_times if t > cutoff]
        
        if len(self.request_times) >= self.requests_per_minute:
            # Wait until oldest request expires
            wait_time = 60 - (now - self.request_times[0]).total_seconds()
            print(f"Rate limit reached, waiting {wait_time:.1f} seconds...")
            time.sleep(wait_time)
            
        self.request_times.append(now)
    
    def make_request(self, endpoint, params=None):
        """Make a request with automatic rate limiting."""
        self.wait_if_needed()
        return make_request(endpoint, params)


Usage

client = RateLimitedClient(requests_per_minute=30)  # Conservative limit
for page in paginated_results:
    result = client.make_request(endpoint, params)

Error 3: Missing data due to race conditions

When data is being added while you're paginating, you might miss items or get duplicates.

# ❌ BROKEN: No handling for concurrent data changes
cursor = None
all_items = []
while True:
    params = {"limit": 100}
    if cursor:
        params["cursor"] = cursor
        
    result = make_request(endpoint, params)
    new_items = result["data"]
    
    # Problem: New items added during pagination might be skipped
    all_items.extend(new_items)
    
    if not result["has_more"]:
        break
    cursor = result["next_cursor"]


✅ FIXED: Implement snapshot-based pagination with duplicate detection

def fetch_all_items_stably(endpoint, id_field="id"):
    """Fetch all items with duplicate detection and stable ordering."""
    cursor = None
    seen_ids = set()
    all_items = []
    
    while True:
        params = {"limit": 100, "sort": "created_at:asc"}  # Stable ordering
        
        if cursor:
            params["cursor"] = cursor
            
        result = make_request(endpoint, params)
        new_items = result.get("data", [])
        
        # Filter out already-seen items
        for item in new_items:
            item_id = item.get(id_field)
            if item_id not in seen_ids:
                seen_ids.add(item_id)
                all_items.append(item)
            else:
                print(f"Skipping duplicate: {item_id}")
        
        print(f"Fetched page: {len(new_items)} items, "
              f"{len([i for i in new_items if i.get(id_field) in seen_ids])} new")
        
        if not result.get("has_more"):
            break
        cursor = result.get("next_cursor")
        
    return all_items

Error 4: Handling empty responses and edge cases

Always account for empty pages, single-item pages, and malformed responses.

# ✅ ROBUST: Comprehensive error handling for all edge cases
def safe_paginate(endpoint, params, max_pages=1000):
    """Safely paginate with comprehensive error handling."""
    all_items = []
    cursor = None
    page_num = 0
    
    while page_num < max_pages:
        request_params = {"limit": 50, **params}
        if cursor:
            request_params["cursor"] = cursor
            
        try:
            response = make_request(endpoint, request_params)
            
            # Handle None response
            if response is None:
                print(f"Page {page_num + 1}: Request failed, retrying...")
                time.sleep(1)
                continue
                
            # Handle empty response
            if "data" not in response:
                print(f"Page {page_num + 1}: Unexpected response format")
                break
                
            items = response.get("data", [])
            
            # Handle empty page (end of results)
            if len(items) == 0:
                print(f"Page {page_num + 1}: Empty page, ending pagination")
                break
                
            all_items.extend(items)
            print(f"Page {page_num + 1}: Retrieved {len(items)} items "
                  f"(total: {len(all_items)})")
            
            # Handle has_more field
            if not isinstance(response.get("has_more"), bool):
                print("No more pages (has_more not present)")
                break
                
            if not response["has_more"]:
                print("Reached last page")
                break
                
            # Validate cursor exists
            if not response.get("next_cursor"):
                print("Warning: has_more=True but no next_cursor, ending")
                break
                
            cursor = response["next_cursor"]
            page_num += 1
            
        except Exception as e:
            print(f"Page {page_num + 1}: Error - {e}")
            time.sleep(2)  # Wait before retry
            continue
    
    return all_items

Best Practices Summary

Conclusion

Pagination is an essential skill when working with AI model outputs at scale. By understanding cursor-based pagination, implementing proper error handling, and following the patterns outlined in this guide, you'll be equipped to handle even the largest datasets efficiently.

HolySheep AI's infrastructure, with its sub-50ms latency and ¥1=$1 pricing, makes pagination experiments cost-effective and responsive. The combination of competitive model pricing (DeepSeek V3.2 at $0.42/MTok, Gemini 2.5 Flash at $2.50/MTok) and reliable pagination support means you can focus on building your application rather than fighting API limitations.

Start small, test thoroughly, and remember: pagination is not a limitation—it's a feature that enables scalable, resilient AI applications.

Ready to implement pagination in your project? The code examples in this guide are ready to copy, paste, and run. Experiment with different page sizes, implement the error handling patterns, and watch your application handle large datasets with ease.

👉 Sign up for HolySheep AI — free credits on registration

Related Resources

Related Articles