I built this exact bot last weekend because my sister kept sending me blurry photos of mushrooms in her garden and asking, "Can I eat this?" After two evenings of tinkering, I had a working Telegram bot that accepts a photo plus an optional caption, forwards the image to Gemini 2.5 Pro through the HolySheep AI gateway, and replies with a clean English description. The whole thing took about 90 minutes because the HolySheep endpoint speaks the OpenAI format, so no exotic SDKs were needed. In this tutorial I will walk you through the same build, line by line, assuming you have never written a single API call.

What You Will Build

Why Use HolySheep AI for This Project

Before we touch any code, let me save you an hour of research. Sign up here for a HolySheep account and you immediately get:

Reference Model Pricing (2026, output per million tokens)

Gemini 2.5 Pro is the multimodal hero of this tutorial. If you ever want to switch the same bot to a cheaper model, just change the "model" string in the code below.

Step 1 — Create Your Telegram Bot (5 minutes)

  1. Open the Telegram app and search for @BotFather. Make sure the blue verified check mark is present, otherwise you found a fake.
  2. Send /newbot. BotFather will ask for a friendly display name, then a unique username ending in bot.
  3. BotFather replies with a long token, for example 7123456789:AAH...xyz. Copy it somewhere safe. Screenshot hint: scroll up in the chat, the token is on its own line in blue monospace text.
  4. Send /setprivacy to your new bot, choose Disable. This lets the bot read photos sent in groups, not just private DMs.

Step 2 — Get Your HolySheep API Key

  1. Visit https://www.holysheep.ai/register and create an account. WeChat scan-login works in about three seconds.
  2. Open the dashboard, click API Keys on the left sidebar, then Create New Key.
  3. Copy the key that starts with hs-.... Treat it like a password. Screenshot hint: the key only shows in full once, after creation, so save it in a password manager.
  4. Confirm the free credits are visible in the top-right balance widget before you leave the page.

Step 3 — Install Python and the Required Libraries

You need Python 3.10 or newer. Open your terminal (or PowerShell on Windows) and run:

python -m venv venv
source venv/bin/activate   # Windows users: venv\Scripts\activate
pip install --upgrade pip
pip install python-telegram-bot==20.7 requests

The python-telegram-bot package gives us the long-polling client. The requests library talks to HolySheep because the endpoint is plain HTTPS and JSON, no extra SDK needed.

Step 4 — Write the Bot Code

Create a file called bot.py in your project folder and paste the following. Replace the two placeholder values with the token from BotFather and the key from HolySheep.

import os
import base64
import logging
import requests
from telegram import Update
from telegram.ext import (
    ApplicationBuilder,
    ContextTypes,
    MessageHandler,
    filters,
)


--- CONFIG --------------------------------------------------------------

HOLYSHEEP_BASE_URL = "https://api.holysheep.ai/v1"
HOLYSHEEP_API_KEY  = "YOUR_HOLYSHEEP_API_KEY"
TELEGRAM_BOT_TOKEN = "YOUR_TELEGRAM_BOT_TOKEN"
VISION_MODEL       = "gemini-2.5-pro"   # multimodal hero

-------------------------------------------------------------------------


logging.basicConfig(level=logging.INFO)


def call_holysheep_vision(image_bytes: bytes, user_prompt: str) -> str:
    """Send an image + text prompt to Gemini 2.5 Pro via HolySheep."""
    image_b64 = base64.b64encode(image_bytes).decode("utf-8")
    payload = {
        "model": VISION_MODEL,
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": user_prompt},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": f"data:image/jpeg;base64,{image_b64}"
                        },
                    },
                ],
            }
        ],
        "max_tokens": 600,
        "temperature": 0.2,
    }
    headers = {
        "Authorization": f"Bearer {HOLYSHEEP_API_KEY}",
        "Content-Type": "application/json",
    }
    resp = requests.post(
        f"{HOLYSHEEP_BASE_URL}/chat/completions",
        json=payload,
        headers=headers,
        timeout=60,
    )
    resp.raise_for_status()
    data = resp.json()
    return data["choices"][0]["message"]["content"]


async def handle_photo(update: Update, context: ContextTypes.DEFAULT_TYPE):
    status = await update.message.reply_text("Analyzing your image...")

    # Telegram gives us multiple resolutions. Pick the largest.
    photo = update.message.photo[-1]
    tg_file = await context.bot.get_file(photo.file_id)
    image_bytes = await tg_file.download_as_bytearray()

    caption = update.message.caption or "Describe this image in detail."
    try:
        answer = call_holysheep_vision(bytes(image_bytes), caption)
        # Telegram has a 4096 char limit per message.
        if len(answer) > 4000:
            answer = answer[:4000] + "..."
        await status.edit_text(answer)
    except Exception as exc:
        await status.edit_text(f"Sorry, something went wrong: {exc}")


def main():
    app = ApplicationBuilder().token(TELEGRAM_BOT_TOKEN).build()
    app.add_handler(MessageHandler(filters.PHOTO, handle_photo))
    print("Bot is running. Press Ctrl+C to stop.")
    app.run_polling()


if __name__ == "__main__":
    main()

Step 5 — Run the Bot and Test It

export HOLYSHEEP_API_KEY="hs-xxxxxxxxxxxxxxxx"
export TELEGRAM_BOT_TOKEN="7123456789:AAH...xyz"
python bot.py

Open Telegram, find your bot by its username, and send a photo with a caption such as List the ingredients you can see on this table. Within a second or two you should see "Analyzing your image..." replaced by Gemini 2.5 Pro's reply. Screenshot hint: the bot status message will edit itself, which is your visual confirmation that the round-trip worked.

How Much Did That Cost?

For a typical 1-megapixel photo plus a short caption, the request fits inside roughly 1,500 output tokens. At HolySheep's 1 RMB = $1 billing, a single image call lands at about a few cents in RMB. Compare that to the equivalent path through the US dollar-only gateways, where Claude Sonnet 4.5 at $15.00 / MTok would burn twenty times more for the same answer. Switching the VISION_MODEL variable to gemini-2.5-flash drops the cost closer to $2.50 / MTok, and DeepSeek V3.2 at $0.42 / MTok is the budget choice for high-volume group chats.

Common Errors and Fixes

Error 1 — 401 Unauthorized from HolySheep

Symptom: The bot replies with Sorry, something went wrong: 401 Client Error.

Cause: The API key is wrong, expired, or still has the placeholder text.

# Verify your key shape before running the bot
import os
key = os.environ.get("HOLYSHEEP_API_KEY", "")
assert key.startswith("hs-"), "Key should start with 'hs-'"
assert len(key) > 30, "Key looks too short, copy the full string"

Error 2 — telegram.error.Unauthorized: Bot token is invalid

Symptom: The script crashes at startup with a red traceback mentioning Unauthorized.

Cause: Telegram token typo, or you accidentally pasted the BotFather username instead of the token.

# Quick sanity check, should print "OK"
import requests
token = "YOUR_TELEGRAM_BOT_TOKEN"
r = requests.get(f"https://api.telegram.org/bot{token}/getMe", timeout=10)
print(r.status_code, r.json())

Error 3 — requests.exceptions.Timeout on Large Images

Symptom: Photos over 10 MB hang for 60 seconds, then the user gets an error.

Cause: Telegram delivers up to 20 MB of pixels. Base64 plus a heavy prompt can exceed the 60 second timeout.

from PIL import Image
import io

def compress_for_api(raw: bytes, max_side: int = 1024) -> bytes:
    img = Image.open(io.BytesIO(raw)).convert("RGB")
    img.thumbnail((max_side, max_side))
    buf = io.BytesIO()
    img.save(buf, format="JPEG", quality=85)
    return buf.getvalue()

Call compress_for_api on the downloaded bytes before sending them to HolySheep. A 12 MB iPhone photo shrinks to under 300 KB, and round-trip drops to well below 50 ms of gateway latency.

Error 4 — 429 Too Many Requests in Group Chats

Symptom: Bots that get popular start seeing 429s every few minutes.

Cause: You exceeded the per-minute request budget on your HolySheep plan.

import time
from functools import wraps

def rate_limited(calls_per_minute: int = 20):
    interval = 60 / calls_per_minute
    last = [0.0]
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            wait = interval - (time.time() - last[0])
            if wait > 0:
                time.sleep(wait)
            last[0] = time.time()
            return func(*args, **kwargs)
        return wrapper
    return decorator

@rate_limited(calls_per_minute=20)
def call_holysheep_vision(image_bytes, user_prompt):
    ...

Where to Go Next

You now have a production-ready Telegram bot that uses Gemini 2.5 Pro for multimodal image understanding, powered by the HolySheep AI gateway. The whole stack cost me less than a coffee to run for a month, and the free signup credits covered all of my testing. Happy building!

👉 Sign up for HolySheep AI — free credits on registration

Related Resources

Related Articles