From Reel to Data: Extracting Creator Economy Metrics at Scale

The creator economy generates billions of visual content pieces daily. Brands, agencies, and platforms need to understand what is working, but much of the useful data is trapped inside screenshots, thumbnails, and overlay text.

In this guide, we will build a simple two-step pipeline for short-form video analysis:

1. Use ocrskill to convert screenshots into Markdown.
2. Use a second AI model to turn that Markdown into structured JSON.

That distinction matters. ocrskill is the OCR layer. It extracts the visible content accurately and quickly, but the schema design and data normalization happen in a separate model.

The Data Problem

Short-form video platforms display critical metrics visually:

• View counts, likes, comments, shares
• Creator handles and verification badges
• Hashtags and captions burned into thumbnails
• Branded content tags and sponsorship disclosures
• Music/audio attribution text

Some of this data is unavailable or unreliable via platform APIs. But it is visible in screenshots and thumbnails, which makes it a good fit for an OCR-first workflow.

Capture Strategy

We use a combination of:

• Platform APIs (where available) for basic metadata
• Screenshot automation for visual metrics not exposed via API
• Thumbnail extraction for video preview analysis

from playwright import async_api as playwright

async def capture_reel_screenshot(url: str, output_path: str):
    async with playwright.async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page(viewport={"width": 390, "height": 844})
        await page.goto(url, wait_until="networkidle")
        await page.screenshot(path=output_path, full_page=False)
        await browser.close()

Step 1: Extract the Screenshot to Markdown

Once screenshots are captured, send them to ocrskill and ask for a faithful Markdown transcription of everything visible in the image.

from openai import OpenAI

ocr_client = OpenAI(
    base_url="https://api.ocrskill.com/v1",
    api_key="sk-your-key"
)

def extract_screenshot_markdown(image_path: str) -> str:
    import base64

    with open(image_path, "rb") as f:
        b64 = base64.b64encode(f.read()).decode()

    response = ocr_client.chat.completions.create(
        model="ocrskill-v1",
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": """Convert this social media screenshot into clean Markdown.

Preserve:
- visible text
- counts and labels
- usernames and captions
- hashtags and mentions
- sponsorship disclosures
- music attribution

Do not guess missing values. If something is unclear, mark it as uncertain in the Markdown."""},
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}
            ]
        }]
    )
    return response.choices[0].message.content

The key idea is to keep the OCR step honest. You first extract what is actually visible, in a readable format, before asking another model to interpret or normalize it.

Step 2: Use Another AI Model to Structure the Data

After OCR, pass the Markdown into a general-purpose LLM that is good at schema following. Here we use gpt-5-mini, but the same pattern also works with other structured-output-capable models.

from openai import OpenAI
import json

llm_client = OpenAI(api_key="sk-your-openai-key")

def structure_reel_metrics(markdown: str) -> dict:
    response = llm_client.chat.completions.create(
        model="gpt-5-mini",
        temperature=0,
        response_format={"type": "json_object"},
        messages=[{
            "role": "user",
            "content": f"""You are given OCR output from a social media screenshot.

OCR Markdown:
{markdown}

Extract the following fields into JSON:
- creator_handle: string | null
- verified: boolean | null
- view_count: integer | null
- like_count: integer | null
- comment_count: integer | null
- share_count: integer | null
- caption_text: string | null
- hashtags: string[]
- is_sponsored: boolean | null
- music_attribution: string | null

Rules:
- Use null when the OCR does not clearly support a value.
- Normalize numeric abbreviations like 2.4M or 18.2K into integers.
- Do not invent fields that are not present.
- Return only valid JSON."""
        }]
    )
    return json.loads(response.choices[0].message.content)

This separation gives you a more reliable system:

• ocrskill focuses on extracting the visible content from the image.
• The second model focuses on normalization, typing, and schema enforcement.
• You can swap the second model later without changing your OCR layer.

Building the Dataset

Process thousands of screenshots into a structured DataFrame:

import pandas as pd
from pathlib import Path

screenshots = list(Path("./captures").glob("*.png"))
records = []

for screenshot in screenshots:
    markdown = extract_screenshot_markdown(str(screenshot))
    structured = structure_reel_metrics(markdown)
    structured["source_file"] = screenshot.name
    structured["ocr_markdown"] = markdown
    records.append(structured)

df = pd.DataFrame(records)
df.to_parquet("creator_metrics.parquet")

Keeping both the raw OCR Markdown and the structured JSON is useful for auditing. If a downstream metric looks wrong, you can inspect the OCR output that the structuring model received.

Analysis Examples

With structured data, you can now answer questions like:

• Which creators have the highest engagement rate in the fitness niche?
• What posting times correlate with maximum views?
• How does sponsored content performance compare to organic?
• What hashtag combinations drive the most shares?

Why This Architecture Works Better

Trying to force OCR and schema reasoning into a single step can make debugging harder. A two-stage pipeline is easier to trust and improve:

• When OCR is wrong, you can fix the extraction prompt or improve screenshot quality.
• When the JSON is wrong, you can refine the second model’s schema prompt.
• You can test each stage independently.
• You can re-run the structuring step later without reprocessing the image.

For production systems, this usually leads to better observability and cleaner failure handling.

Practical Extensions

Once the pipeline is working, you can extend it with:

• Trend classification: classify the reel by niche, format, or call to action.
• Brand mention detection: flag competitor names or campaign disclosures.
• Confidence review queues: send uncertain OCR outputs to human review.
• Embedding and search: index the OCR Markdown for retrieval and analytics.

Conclusion

The creator economy is a visual-first ecosystem, and many of the most useful signals never arrive in a clean API response. A practical approach is to let ocrskill do what it does best, convert images into high-quality Markdown, and then let a second AI model transform that OCR output into structured data.

That gives you a pipeline that is easier to audit, easier to improve, and much closer to how production OCR systems are actually built.