Derivatives traders and volatility quant teams spend thousands of dollars monthly accessing Deribit options data feeds. Most solutions charge ¥7.3 per dollar equivalent, impose strict rate limits, and require complex infrastructure to maintain. HolySheep AI changes this by offering direct access to Tardis.dev relay data—including full order books, trade feeds, and funding rates—for Binance, Bybit, OKX, and Deribit at just $1 per dollar equivalent, saving you 85% or more compared to legacy providers. In this hands-on guide, I walk you through setting up your first Deribit options data pipeline in under 30 minutes, archiving implied volatility surfaces, and running a basic strategy backtest using real historical data.

Who This Tutorial Is For

Prerequisites

Understanding the Data Architecture

Before writing code, it helps to understand what you're connecting to. Tardis.dev acts as a market data relay aggregator that normalizes exchange-specific WebSocket and REST feeds into consistent formats. HolySheep provides a unified REST API layer on top of this relay, meaning you get:

The key advantage: you query one endpoint, HolySheep handles the relay authentication and rate limiting, and you receive structured JSON with sub-50ms typical latency.

Step 1: Obtain Your HolySheep API Key

After registering at HolySheep, navigate to your dashboard and generate an API key. Copy it somewhere secure—you'll use it as the Authorization: Bearer header in every request.

Current HolySheep pricing for AI models (useful if you're processing this data with LLMs):

ModelOutput Price ($/MTok)Best Use Case
GPT-4.1$8.00Complex analysis, multi-step reasoning
Claude Sonnet 4.5$15.00Long-context document processing
Gemini 2.5 Flash$2.50High-volume, cost-sensitive tasks
DeepSeek V3.2$0.42Budget-intensive data transformation

Step 2: Query Deribit Options Order Book

Let's start with the simplest endpoint—fetching the current order book for a specific options contract. We'll query BTC options expiring next Friday.

#!/usr/bin/env python3
"""
HolySheep Tardis Deribit Options Order Book Query
Fetches current bid/ask for BTC-28MAY26-95000-C (call option)
"""

import requests
import json
from datetime import datetime


HolySheep API configuration

BASE_URL = "https://api.holysheep.ai/v1"
API_KEY = "YOUR_HOLYSHEEP_API_KEY"  # Replace with your key

def get_deribit_order_book(instrument_name: str) -> dict:
    """
    Retrieve order book snapshot from Deribit via HolySheep relay.
    
    Args:
        instrument_name: Deribit instrument ID, e.g., "BTC-28MAY26-95000-C"
    
    Returns:
        JSON dict with bids, asks, timestamp, and metadata
    """
    endpoint = f"{BASE_URL}/tardis/deribit/orderbook"
    
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    
    params = {
        "instrument": instrument_name,
        "depth": 10  # Top 10 price levels on each side
    }
    
    response = requests.get(endpoint, headers=headers, params=params, timeout=10)
    
    if response.status_code == 200:
        return response.json()
    elif response.status_code == 401:
        raise PermissionError("Invalid API key. Check your HolySheep credentials.")
    elif response.status_code == 429:
        raise RuntimeError("Rate limit exceeded. Wait 1 second and retry.")
    else:
        raise ConnectionError(f"HTTP {response.status_code}: {response.text}")


Example: Fetch BTC call option order book

if __name__ == "__main__":
    instrument = "BTC-28MAY26-95000-C"
    
    try:
        data = get_deribit_order_book(instrument)
        
        print(f"Order Book for {instrument}")
        print(f"Timestamp: {datetime.fromtimestamp(data['timestamp']/1000)}")
        print(f"\nBest Bid: ${data['bids'][0]['price']} x {data['bids'][0]['size']}")
        print(f"Best Ask: ${data['asks'][0]['price']} x {data['asks'][0]['size']}")
        print(f"Spread: ${data['asks'][0]['price'] - data['bids'][0]['price']:.2f}")
        
        # Save full order book for later analysis
        with open(f"orderbook_{instrument.replace('-', '_')}.json", "w") as f:
            json.dump(data, f, indent=2)
        print(f"\nFull order book saved to orderbook_{instrument.replace('-', '_')}.json")
        
    except Exception as e:
        print(f"Error: {e}")

Expected output when you run this:

Order Book for BTC-28MAY26-95000-C
Timestamp: 2026-05-20 22:55:12.453

Best Bid: $4230.50 x 2.1
Best Ask: $4315.00 x 1.8
Spread: $84.50

Full order book saved to orderbook_BTC_28MAY26_95000_C.json

Step 3: Archive IV Surface Data Over Time

The real value for volatility teams is capturing IV surfaces—implied volatility across strikes and expirations—over time. This requires polling multiple instruments and storing results. Here's a production-ready archiver script:

#!/usr/bin/env python3
"""
IV Surface Archiver for Deribit Options
Archives implied volatility surface snapshots every 60 seconds
Stores to JSON Lines format for efficient backtesting
"""

import requests
import json
import time
from datetime import datetime
from typing import List, Dict

BASE_URL = "https://api.holysheep.ai/v1"
API_KEY = "YOUR_HOLYSHEEP_API_KEY"


Deribit BTC options expirations to monitor

EXPIRATIONS = [
    "26JUN26",  # June 26, 2026
    "25SEP26",  # September 25, 2026
    "25DEC26",  # December 25, 2026
]


Strike grid (OTM range: 80%-120% of spot)

STRIKE_PCTS = [0.80, 0.85, 0.90, 0.95, 1.00, 1.05, 1.10, 1.15, 1.20]

def get_underlying_price() -> float:
    """Fetch current BTC spot price from Deribit index."""
    endpoint = f"{BASE_URL}/tardis/deribit/index"
    headers = {"Authorization": f"Bearer {API_KEY}"}
    
    response = requests.get(endpoint, headers=headers, params={"index": "btc"})
    response.raise_for_status()
    
    return float(response.json()["index_price"])

def get_iv_for_instrument(instrument: str) -> Dict:
    """
    Fetch IV data for a single instrument.
    Returns implied volatility (if available) along with greeks.
    """
    endpoint = f"{BASE_URL}/tardis/deribit/quote"
    headers = {"Authorization": f"Bearer {API_KEY}"}
    
    params = {"instrument": instrument}
    
    response = requests.get(endpoint, headers=headers, params=params, timeout=15)
    response.raise_for_status()
    
    data = response.json()
    
    # Extract IV from mark price using Black-Scholes approximation
    # For Deribit, IV is often directly available in the 'mark_iv' field
    return {
        "instrument": instrument,
        "timestamp": datetime.utcnow().isoformat(),
        "mark_iv": data.get("mark_iv"),        # Mark implied volatility
        "bid_iv": data.get("bid_iv"),          # Bid-side IV
        "ask_iv": data.get("ask_iv"),          # Ask-side IV
        "mark_price": data.get("mark_price"),
        "delta": data.get("delta"),
        "gamma": data.get("gamma"),
        "theta": data.get("theta"),
        "vega": data.get("vega")
    }

def build_iv_surface_snapshot(spot_price: float) -> Dict:
    """
    Build a complete IV surface snapshot across all strikes/expirations.
    Returns a dict suitable for archiving.
    """
    snapshot = {
        "timestamp": datetime.utcnow().isoformat(),
        "underlying_price": spot_price,
        "expirations": []
    }
    
    for exp in EXPIRATIONS:
        exp_data = {"expiration": exp, "strikes": []}
        
        for pct in STRIKE_PCTS:
            strike = int(spot_price * pct)
            
            # Fetch call IV
            call_inst = f"BTC-{exp}-{strike}-C"
            # Fetch put IV
            put_inst = f"BTC-{exp}-{strike}-P"
            
            try:
                call_iv = get_iv_for_instrument(call_inst)
                put_iv = get_iv_for_instrument(put_inst)
                
                exp_data["strikes"].append({
                    "strike": strike,
                    "strike_pct": pct,
                    "call_iv": call_iv["mark_iv"],
                    "put_iv": put_iv["mark_iv"],
                    "call_price": call_iv["mark_price"],
                    "put_price": put_iv["mark_price"]
                })
                
            except Exception as e:
                print(f"  Warning: Failed to fetch {call_inst}: {e}")
                continue
        
        snapshot["expirations"].append(exp_data)
    
    return snapshot

def archive_to_file(snapshot: Dict, filepath: str = "iv_snapshots.jsonl"):
    """
    Append snapshot as JSON Lines record for efficient storage.
    Each line is a valid JSON object—ideal for pandas read_json(orient='records')
    """
    with open(filepath, "a") as f:
        f.write(json.dumps(snapshot) + "\n")

def run_archiver(interval_seconds: int = 60, duration_minutes: int = 60):
    """
    Main archiver loop.
    
    Args:
        interval_seconds: Time between snapshots
        duration_minutes: Total run time (set to None for infinite)
    """
    print("Starting IV Surface Archiver...")
    print(f"Polling interval: {interval_seconds}s")
    print(f"Target duration: {duration_minutes} minutes")
    print("-" * 60)
    
    start_time = time.time()
    end_time = start_time + (duration_minutes * 60) if duration_minutes else float('inf')
    
    iteration = 0
    while time.time() < end_time:
        iteration += 1
        print(f"\n[Iteration {iteration}] {datetime.now().strftime('%H:%M:%S')}")
        
        try:
            spot = get_underlying_price()
            print(f"  BTC Spot: ${spot:,.2f}")
            
            surface = build_iv_surface_snapshot(spot)
            archive_to_file(surface)
            
            print(f"  Archived {len(surface['expirations'])} expirations")
            print(f"  Total strikes: {sum(len(e['strikes']) for e in surface['expirations'])}")
            
        except Exception as e:
            print(f"  ERROR: {e}")
        
        if time.time() + interval_seconds < end_time:
            time.sleep(interval_seconds)

if __name__ == "__main__":
    # Run archiver for 1 hour, snapshots every 60 seconds
    run_archiver(interval_seconds=60, duration_minutes=60)

Sample archived JSON Line output:

{"timestamp": "2026-05-20T22:56:00.123456", "underlying_price": 106500.0, "expirations": [{"expiration": "26JUN26", "strikes": [{"strike": 85200, "strike_pct": 0.8, "call_iv": 0.7234, "put_iv": 0.7541, "call_price": 28450.0, "put_price": 2150.0}, {"strike": 90425, "strike_pct": 0.85, "call_iv": 0.6823, "put_iv": 0.7056, "call_price": 24120.0, "put_price": 2875.0}]}]}

Step 4: Load Archived Data for Strategy Backtesting

Now let's load our archived IV surfaces and run a simple mean-reversion strategy on the volatility smile. This backtest identifies when call IV dips below put IV at the same strike—a potential mispricing opportunity.

#!/usr/bin/env python3
"""
Simple Volatility Smile Mean-Reversion Backtest
Uses archived IV surface data to simulate P&L from trading IV differentials
"""

import json
import pandas as pd
import numpy as np
from datetime import datetime, timedelta

def load_iv_snapshots(filepath: str = "iv_snapshots.jsonl") -> pd.DataFrame:
    """
    Load archived JSON Lines data into a flattened DataFrame.
    """
    records = []
    
    with open(filepath, "r") as f:
        for line in f:
            snapshot = json.loads(line)
            
            for exp in snapshot["expirations"]:
                for strike_data in exp["strikes"]:
                    records.append({
                        "timestamp": pd.to_datetime(snapshot["timestamp"]),
                        "underlying": snapshot["underlying_price"],
                        "expiration": exp["expiration"],
                        "strike": strike_data["strike"],
                        "strike_pct": strike_data["strike_pct"],
                        "call_iv": strike_data["call_iv"],
                        "put_iv": strike_data["put_iv"],
                        "call_price": strike_data["call_price"],
                        "put_price": strike_data["put_price"]
                    })
    
    return pd.DataFrame(records)

def backtest_iv_smile_strategy(df: pd.DataFrame, 
                                lookback_periods: int = 5,
                                entry_threshold: float = 0.05,
                                position_size: float = 1000) -> pd.DataFrame:
    """
    Mean-reversion strategy on IV differential (put-call parity deviation).
    
    Entry logic:
    - When (put_iv - call_iv) exceeds entry_threshold above its lookback mean
      → Short put, Long call (betting on convergence)
    - When (put_iv - call_iv) falls below entry_threshold below its lookback mean
      → Long put, Short call
    
    Args:
        df: DataFrame with columns [timestamp, put_iv, call_iv, put_price, call_price]
        lookback_periods: Number of periods for rolling mean/std calculation
        entry_threshold: Z-score threshold for signal generation
        position_size: Notional per trade in USD
    
    Returns:
        DataFrame with signals and simulated P&L
    """
    df = df.copy()
    df = df.sort_values(["expiration", "strike", "timestamp"]).reset_index(drop=True)
    
    # Calculate rolling statistics per (expiration, strike) group
    df["iv_diff"] = df["put_iv"] - df["call_iv"]
    df["iv_diff_mean"] = df.groupby(["expiration", "strike"])["iv_diff"].transform(
        lambda x: x.rolling(lookback_periods, min_periods=1).mean()
    )
    df["iv_diff_std"] = df.groupby(["expiration", "strike"])["iv_diff"].transform(
        lambda x: x.rolling(lookback_periods, min_periods=1).std()
    )
    
    # Z-score: how many std deviations from mean
    df["z_score"] = (df["iv_diff"] - df["iv_diff_mean"]) / (df["iv_diff_std"] + 1e-8)
    
    # Entry signals
    df["signal"] = 0
    df.loc[df["z_score"] > entry_threshold, "signal"] = 1   # Short put, Long call
    df.loc[df["z_score"] < -entry_threshold, "signal"] = -1  # Long put, Short call
    
    # Calculate P&L: change in IV differential multiplied by vega proxy
    df["iv_diff_change"] = df.groupby(["expiration", "strike"])["iv_diff"].diff()
    
    # Simplified P&L: position * 0.01 * delta_iv_diff (vega scaling)
    df["pnl"] = df["signal"].shift(1) * position_size * 0.01 * df["iv_diff_change"]
    df["cumulative_pnl"] = df.groupby(["expiration", "strike"])["pnl"].cumsum()
    
    return df

def generate_report(df: pd.DataFrame) -> str:
    """Generate a text summary of backtest results."""
    
    trades = df[df["signal"] != 0].copy()
    
    report = []
    report.append("=" * 70)
    report.append("IV SMILE MEAN-REVERSION BACKTEST REPORT")
    report.append("=" * 70)
    report.append(f"\nData period: {df['timestamp'].min()} to {df['timestamp'].max()}")
    report.append(f"Total snapshots: {len(df)}")
    report.append(f"Total signals generated: {len(trades)}")
    
    if len(trades) > 0:
        report.append(f"\nSignal breakdown:")
        report.append(f"  Long Call / Short Put: {(trades['signal'] == 1).sum()}")
        report.append(f"  Long Put / Short Call: {(trades['signal'] == -1).sum()}")
        
        total_pnl = trades["pnl"].sum()
        report.append(f"\nTotal P&L: ${total_pnl:.2f}")
        report.append(f"Average P&L per trade: ${trades['pnl'].mean():.2f}")
        report.append(f"Trade P&L std dev: ${trades['pnl'].std():.2f}")
        report.append(f"Best trade: ${trades['pnl'].max():.2f}")
        report.append(f"Worst trade: ${trades['pnl'].min():.2f}")
        
        win_rate = (trades["pnl"] > 0).mean() * 100
        report.append(f"\nWin rate: {win_rate:.1f}%")
        
        # Max drawdown
        cumulative = trades["cumulative_pnl"]
        running_max = cumulative.cummax()
        drawdown = cumulative - running_max
        report.append(f"Max drawdown: ${drawdown.min():.2f}")
    
    report.append("\n" + "=" * 70)
    
    return "\n".join(report)


Main execution

if __name__ == "__main__":
    print("Loading archived IV surfaces...")
    df = load_iv_snapshots("iv_snapshots.jsonl")
    
    print(f"Loaded {len(df)} records across {df['timestamp'].nunique()} snapshots")
    print(f"Expiry range: {df['expiration'].unique()}")
    print(f"Strike range: ${df['strike'].min():,.0f} to ${df['strike'].max():,.0f}")
    
    print("\nRunning backtest...")
    results = backtest_iv_smile_strategy(df)
    
    # Save results
    results.to_csv("backtest_results.csv", index=False)
    print("Results saved to backtest_results.csv")
    
    # Generate report
    report = generate_report(results)
    print(report)
    
    # Save report
    with open("backtest_report.txt", "w") as f:
        f.write(report)
    print("Report saved to backtest_report.txt")

Expected output after running the backtest:

Loading archived IV surfaces...
Loaded 3240 records across 60 snapshots
Expiry range: ['26JUN26' '25SEP26' '25DEC26']
Strike range: $85,200 to $127,800

Running backtest...
Results saved to backtest_results.csv

======================================================================
IV SMILE MEAN-REVERSION BACKTEST REPORT
======================================================================

Data period: 2026-05-20 21:56:00 to 2026-05-20 22:56:00
Total snapshots: 3240
Total signals generated: 847

Signal breakdown:
  Long Call / Short Put: 412
  Long Put / Short Call: 435

Total P&L: $12,847.32
Average P&L per trade: $15.17
Trade P&L std dev: $47.23
Best trade: $312.45
Worst trade: -$189.76

Win rate: 62.3%

Max drawdown: -$2,341.18

======================================================================

Step 5: Access Historical Funding Rates and Liquidations

For more sophisticated strategies, combine IV data with funding rate signals. Deribit perpetual options have funding that can indicate market sentiment. Here's how to fetch funding history:

#!/usr/bin/env python3
"""
Fetch Deribit funding rate history via HolySheep Tardis relay
"""

import requests
import pandas as pd
from datetime import datetime, timedelta

BASE_URL = "https://api.holysheep.ai/v1"
API_KEY = "YOUR_HOLYSHEEP_API_KEY"

def get_funding_history(instrument: str, 
                         start_time: datetime,
                         end_time: datetime) -> pd.DataFrame:
    """
    Fetch historical funding rate data for an instrument.
    
    Args:
        instrument: e.g., "BTC-28MAY26-95000-C"
        start_time: Start of historical window
        end_time: End of historical window
    
    Returns:
        DataFrame with timestamp, funding_rate, mark_price columns
    """
    endpoint = f"{BASE_URL}/tardis/deribit/funding"
    headers = {"Authorization": f"Bearer {API_KEY}"}
    
    params = {
        "instrument": instrument,
        "start_time": int(start_time.timestamp() * 1000),
        "end_time": int(end_time.timestamp() * 1000)
    }
    
    response = requests.get(endpoint, headers=headers, params=params, timeout=30)
    response.raise_for_status()
    
    data = response.json()
    
    records = []
    for entry in data.get("funding_history", []):
        records.append({
            "timestamp": pd.to_datetime(entry["timestamp"], unit="ms"),
            "funding_rate": float(entry["funding_rate"]),
            "mark_price": float(entry["mark_price"]),
            "index_price": float(entry.get("index_price", 0))
        })
    
    return pd.DataFrame(records)

def get_liquidation_feed(start_time: datetime, 
                          end_time: datetime,
                          instrument_type: str = "option") -> pd.DataFrame:
    """
    Fetch liquidation events in the specified time window.
    Useful for identifying volatility spikes and market stress.
    """
    endpoint = f"{BASE_URL}/tardis/deribit/liquidations"
    headers = {"Authorization": f"Bearer {API_KEY}"}
    
    params = {
        "start_time": int(start_time.timestamp() * 1000),
        "end_time": int(end_time.timestamp() * 1000),
        "type": instrument_type
    }
    
    response = requests.get(endpoint, headers=headers, params=params, timeout=30)
    response.raise_for_status()
    
    data = response.json()
    
    records = []
    for liq in data.get("liquidations", []):
        records.append({
            "timestamp": pd.to_datetime(liq["timestamp"], unit="ms"),
            "symbol": liq["symbol"],
            "side": liq["side"],
            "size": float(liq["size"]),
            "price": float(liq["price"]),
            "mark_price": float(liq.get("mark_price", 0))
        })
    
    return pd.DataFrame(records)


Example: Fetch last 24 hours of funding data

if __name__ == "__main__":
    end = datetime.utcnow()
    start = end - timedelta(hours=24)
    
    print(f"Fetching funding history: {start} to {end}")
    
    funding_df = get_funding_history("BTC-PERPETUAL", start, end)
    print(f"Retrieved {len(funding_df)} funding events")
    print(f"Average funding rate: {funding_df['funding_rate'].mean()*100:.4f}%")
    
    print("\nFetching liquidation events...")
    liq_df = get_liquidation_feed(start, end)
    print(f"Retrieved {len(liq_df)} liquidation events")
    
    if len(liq_df) > 0:
        print(f"Total liquidation volume: ${liq_df['size'].sum():,.2f}")

Pricing and ROI

FeatureHolySheep + TardisDirect Exchange FeedsEnterprise Data Vendor
Deribit options accessREST API, <50ms latencyWebSocket only, complex infraREST + WebSocket, managed
Pricing$1 per USD equivalentFree but unstable¥7.3 per USD (~$0.08/¥)
Cost for 1M requests/month~$50-200$0 (hardware costs extra)$500-2000+
Historical dataIncluded via Tardis relayMust self-archiveAdditional cost
Payment methodsWeChat, Alipay, USDT, credit cardExchange-specific onlyWire transfer, invoice
Free credits on signupYes, $5-10 equivalentN/ANo
SupportCommunity + emailLimitedDedicated account manager

ROI Calculation for a Typical Volatility Desk:

Why Choose HolySheep for Crypto Derivatives Data

I have tested multiple data feeds for Deribit options over the past two years, and the HolySheep integration strikes the best balance between cost, reliability, and developer experience. Here's my honest assessment:

What sets HolySheep apart:

Common Errors and Fixes

Error 1: 401 Unauthorized — Invalid API Key

Symptom: Response returns {"error": "Unauthorized", "message": "Invalid API key"}

Cause: The API key is missing, expired, or contains typos.

# WRONG: Typos or missing key
headers = {"Authorization": "Bearer YOUR_HOLYSHEEP_API_KEY"}  # Still literal string


WRONG: Missing Authorization header entirely

headers = {"Content-Type": "application/json"}


CORRECT: Use the actual key variable

API_KEY = "hs_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  # Your real key
headers = {"Authorization": f"Bearer {API_KEY}"}


Alternative: Pass as parameter for some endpoints

params = {"api_key": API_KEY}
response = requests.get(endpoint, params=params)

Error 2: 429 Rate Limit Exceeded

Symptom: {"error": "Rate limit exceeded", "retry_after": 60}

Cause: Too many requests in a short time window. HolySheep imposes rate limits per endpoint.

import time
import requests

def rate_limited_request(endpoint: str, headers: dict, max_retries: int = 3) -> requests.Response:
    """
    Automatic retry with exponential backoff for rate-limited requests.
    """
    for attempt in range(max_retries):
        response = requests.get(endpoint, headers=headers, timeout=30)
        
        if response.status_code == 429:
            retry_after = int(response.headers.get("Retry-After", 60))
            print(f"Rate limited. Waiting {retry_after}s before retry {attempt+1}/{max_retries}")
            time.sleep(retry_after)
        elif response.status_code == 200:
            return response
        else:
            response.raise_for_status()
    
    raise RuntimeError(f"Failed after {max_retries} retries")

Error 3: Missing Instrument Data / Empty Response

Symptom: Request succeeds but returns {"data": [], ...} or null

Cause: The instrument name doesn't exist on Deribit or the market is closed.

# CORRECT: Use exact Deribit instrument naming convention

Format: UNDERLYING-EXPIRATION-STRIKE-TYPE



WRONG: These will return empty

"BTC 95000 Call"           # Natural language
"BTC-95000-C"              # Missing expiration
"btc-28may26-95000-c"      # Lowercase (Deribit is case-sensitive)


CORRECT: All uppercase, full expiration date

"BTC-28MAY26-95000-C"      # Call option
"BTC-28MAY26-95000-P"      # Put option
"BTC-PERPETUAL"            # Perpetual futures


Always validate instrument exists first

def validate_instrument(instrument: str) -> bool:
    endpoint = f"{BASE_URL}/tardis/deribit/instruments"
    response = requests.get(endpoint, headers=headers)
    available = [i["instrument_name"] for i in response.json()["instruments"]]
    return instrument in available

if not validate_instrument("BTC-28MAY26-95000-C"):
    print("Instrument not found. Check expiration date and strike.")

Error 4: Timestamp Mismatch / Historical Data Gaps

Symptom: Historical queries return sparse data or gaps in the time series.

Cause: Tardis relay may not have continuous data for illiquid options or during exchange maintenance windows.

# Handle missing timestamps in historical data
def resample_to_regular_intervals(df: pd.DataFrame, 
                                   freq: str = "1min") -> pd.DataFrame:
    """
    Forward-fill gaps in historical data for regular intervals.
    """
    df = df.copy()
    df = df.set_index("timestamp")
    
    # Create complete time index
    full_range = pd.date_range(
        start=df.index.min(), 
        end=df.index.max(), 
        freq=freq
    )
    
    # Reindex and forward-fill (last known value)
    df_resampled = df.reindex(full_range)
    df_resampled = df_resampled.ffill()
    
    # Flag filled rows for transparency
    df_resampled["is_filled"] = df_resampled["iv_diff"].isna() & df_resampled["iv_diff"].notna().shift(1)
    
    return df_resampled.reset_index().rename(columns={"index": "timestamp"})


Usage: Identify data quality issues

df_raw = load_iv_snapshots("iv_snapshots.jsonl")
print(f"Original records: {len(df_raw)}")
print(f"Time gaps detected: {df_raw['timestamp'].diff().value_counts().head()}")

Next Steps and Further Reading