OCR Status: Track Every Document From Queued to Searchable
OCR status is the state a document is in while text is being extracted from it: waiting, running, finished, or failed. Asynchronous OCR services report it as a job field you poll or get notified about. Synchronous APIs report it through the HTTP response itself. And for PDFs there is a second, quieter meaning: whether the file already has a text layer at all. If you are building anything that OCRs more than a handful of files, you need to track all three, because “the API call returned” and “this document is searchable and complete” are not the same thing.
This guide lists the status values the big cloud OCR services use, explains which ones are final, shows how partial results hide inside a “success”, and ends with a short Python tracker that keeps one status row per file.
What OCR status means in practice
People use the phrase for three related things:
- Job status from an asynchronous API. You submit a document, get an ID back, and ask later whether the job is still running.
- Pipeline status in your own system: received, queued, processing, done, needs review, failed. This is the column your operations team actually looks at.
- Text layer status of a file: has this PDF been OCR’d already, or is it just page images?
The first comes from the vendor. The second and third are yours to build, and they are where most lost documents hide.
Status values in the major OCR APIs
The three large cloud services use different names for the same lifecycle:
| Service | Status field | Values |
|---|---|---|
| Amazon Textract (async) | JobStatus |
IN_PROGRESS, SUCCEEDED, FAILED, PARTIAL_SUCCESS |
| Azure AI Document Intelligence | status |
notStarted, running, succeeded, failed, plus canceled and skipped in the SDK enum |
| Google Document AI (batch) | metadata.state |
WAITING, RUNNING, SUCCEEDED, FAILED, CANCELLING, CANCELLED |
Sources: the Textract GetDocumentTextDetection reference, the Azure DocumentIntelligenceOperationStatus enum, and the Document AI BatchProcessMetadata.State enum.
Split these into two groups in your code. Waiting and running states mean “ask again later”. Everything else is terminal, and your code should stop polling and record an outcome. A common bug is a polling loop that only checks for SUCCEEDED and FAILED, so a cancelled or partially successful job spins until a timeout someone set years ago.
Partial success is the status that bites
Textract’s PARTIAL_SUCCESS is the clearest example. The response still carries results, but the Warnings array lists an ErrorCode and the Pages it applies to. If your code treats any non-FAILED job as done, those pages silently vanish from your search index.
Google Document AI does something similar at the batch level. One batch operation can cover many files, and metadata.individualProcessStatuses holds a status for each input file, as the send a request guide shows. A batch that reports SUCCEEDED can still contain per-file problems, so read the per-file list, not just the top-level state.
The rule that covers both: record page or file counts next to the status, and compare what came back with what went in.
Polling versus notifications
How you learn about a status change depends on the service:
- Textract publishes job completion to an Amazon SNS topic. You subscribe an SQS queue or a Lambda function, then call the matching
Getoperation, such asGetDocumentTextDetection, for results (asynchronous operations guide). By default, Textract keeps those results for 7 days unless you set an output bucket. - Azure Document Intelligence answers the analyze request with
202 Accepted, anOperation-Locationheader to poll, and aRetry-Afterheader that tells you how long to wait between checks (Analyze Document reference). - Google Document AI returns a long-running operation. You call
operations.getuntil it reports"done": true, then read the state and output location.
Respect Retry-After when it is present. Polling every second does not make OCR finish sooner, and it can push you into rate limits that create failures of their own. When no hint exists, back off: start at a few seconds and grow the interval, with a hard ceiling on total wait time.
Synchronous OCR: the response is the status
Many OCR APIs, including OCRskill, answer in the same request by default. There is no job ID and nothing to poll, so the HTTP response is the status:
200with results means the request finished.4xxmeans the request itself is wrong and resending it unchanged will not help. On OCRskill,400covers an empty body or an undetectable file type,401an invalid or expired key, and403a URL upload attempted with a free key.504means the upstream OCR timed out. OCRskill sends aRetry-Afterheader with the number of seconds to wait before trying again.
A 200 still needs one more check. OCRskill returns an X-OCR-Pages header with pages processed and pages in the input, for example 35/35. If a key’s page limit is lower than the document length, the two numbers differ, and the honest status for that file is “partial”, not “done”. The PDF OCR in Python guide shows the request in full.
Callbacks: accepted is not finished
With a paid key you can add a callback URL instead of waiting on the connection. OCRskill validates the request, answers 202 with {"status": "accepted"}, and later sends the result to your URL with a single PUT. The OCR callbacks reference spells out two details that shape your status model:
- Failures are silent. If OCR fails after the
202, no error body is sent. Your server simply never gets aPUTfor that request. - Delivery retries for about an hour. A failed
PUTis retried up to 21 times, 3 minutes apart, then the result is discarded.
So a callback integration needs a deadline. Store each file as accepted with a timestamp when the 202 arrives, move it to done when its PUT lands, and mark anything still accepted after your expected processing time plus that hour as failed so it gets resubmitted. OCRskill adds no request ID to the PUT, so give every file its own callback path or query value, for example ?file_id=8421, and use it to find the row. The PUT also carries no X-OCR-Pages header; for JSON results, compare usage_info.pages_processed with the page count you expected.
Track OCR status per file with Python and SQLite
The script below keeps one row per file in a SQLite database: its status, how many pages came back, how many attempts it took, and the last error. It sends each PDF to OCRskill, retries on 504 and network timeouts, marks client errors as failed without retrying, and flags partial results.
import base64
import os
import sqlite3
import sys
import time
from pathlib import Path
import httpx
API_URL = "https://api.ocrskill.com/ocr"
API_KEY = os.environ["OCRSKILL_API_KEY"]
MAX_ATTEMPTS = 3
db = sqlite3.connect("ocr_status.db")
db.execute("""
CREATE TABLE IF NOT EXISTS ocr_status (
path TEXT PRIMARY KEY,
status TEXT NOT NULL,
pages_done INTEGER,
pages_total INTEGER,
attempts INTEGER NOT NULL DEFAULT 0,
error TEXT,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
)""")
def set_status(path, status, pages_done=None, pages_total=None, attempts=0, error=None):
db.execute("""
INSERT INTO ocr_status (path, status, pages_done, pages_total, attempts, error)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(path) DO UPDATE SET
status = excluded.status,
pages_done = excluded.pages_done,
pages_total = excluded.pages_total,
attempts = excluded.attempts,
error = excluded.error,
updated_at = CURRENT_TIMESTAMP
""", (str(path), status, pages_done, pages_total, attempts, error))
db.commit()
def ocr_file(path: Path):
data = base64.b64encode(path.read_bytes()).decode()
body = {"document_url": "data:application/pdf;base64," + data}
for attempt in range(1, MAX_ATTEMPTS + 1):
set_status(path, "processing", attempts=attempt)
try:
r = httpx.post(API_URL, json=body, timeout=180.0,
headers={"Authorization": f"Bearer {API_KEY}"})
except httpx.TimeoutException:
time.sleep(30 * attempt)
continue
if r.status_code == 504:
wait = r.headers.get("retry-after", "30")
time.sleep(int(wait) if wait.isdigit() else 30)
continue
if r.status_code != 200:
set_status(path, "failed", attempts=attempt,
error=f"HTTP {r.status_code}: {r.text[:200]}")
return
pages = r.json()["pages"]
header = r.headers.get("x-ocr-pages", "")
if "/" in header:
done, total = (int(n) for n in header.split("/"))
else:
done = total = len(pages)
out = path.with_suffix(".md")
out.write_text("\n\n".join(p["markdown"] for p in pages), encoding="utf-8")
status = "done" if done == total else "partial"
set_status(path, status, done, total, attempt)
return
set_status(path, "failed", attempts=MAX_ATTEMPTS, error="no result after retries")
finished = {row[0] for row in db.execute(
"SELECT path FROM ocr_status WHERE status IN ('done', 'partial')")}
for pdf in sorted(Path(sys.argv[1]).glob("*.pdf")):
if str(pdf) not in finished:
ocr_file(pdf)
for status, count in db.execute(
"SELECT status, COUNT(*) FROM ocr_status GROUP BY status ORDER BY status"):
print(f"{status:12} {count}")
Run it against a folder:
pip install httpx
export OCRSKILL_API_KEY=$(curl -fsS https://api.ocrskill.com/get-key)
python ocr_status.py ./scans
A few design choices are worth copying even if you use another OCR service:
- The status row is written before the call. If the process dies mid-request, the file stays in
processingwith anupdated_attimestamp. A row stuck inprocessingfor an hour is a crash you can find with one query instead of a file nobody noticed. partialis its own status. It is not counted as failed, because some text came back, and it is not counted as done, because some pages did not. Send these to a person or rerun them with a higher page limit.- Client errors are not retried. A
401or400will fail the same way every time. Retrying it only hides the real problem and spends quota. - Reruns are safe. Finished files are skipped, so you can run the script on a schedule and it picks up only new or failed files.
To see what needs attention:
sqlite3 ocr_status.db "SELECT path, status, pages_done, pages_total, error
FROM ocr_status WHERE status NOT IN ('done') ORDER BY updated_at;"
Check whether a PDF already has OCR
Before sending a file anywhere, it helps to know whether it already has a text layer. Poppler’s command-line tools answer that quickly:
pdffonts scan.pdf # image-only PDFs usually list no fonts
pdftotext -f 1 -l 1 scan.pdf - | wc -w # word count of page 1's text layer
A word count of zero on a page that clearly has text means the page is an image. A healthy count means someone, or some scanner, already ran OCR. That check is useful for deduplicating work, but it says nothing about quality. Old scanner OCR can be badly wrong and still produce plenty of words.
OCRmyPDF makes this explicit in its return codes, which it treats as a stable interface. Exit code 6 (already_done_ocr) means the file already appears to contain text. Its error guide lists three ways forward: --skip-text to leave text pages alone, --redo-ocr to strip old invisible OCR text and run it again, or --force-ocr to rasterize everything and start over. Map that exit code to a status such as already_has_text rather than failed, or your failure count will be full of files that were fine. If you run Paperless-ngx, the same choice shows up as its OCR mode setting, covered in Paperless-ngx OCR settings.
A finished status is not a correct result
Every status above describes the process, not the output. SUCCEEDED means the engine returned something, not that the invoice total is right. Two cheap guards close most of the gap:
- Check content, not just existence. Flag results with zero words, far fewer words than the page count suggests, or a high share of non-letter characters. Why Is OCR So Bad? walks through the usual causes when that happens.
- Validate the values you depend on. If you need fields rather than text, request them as typed JSON and let missing required fields fail loudly. With OCRskill,
/ocr.jsonreturns400when a required field is missing instead of an empty value, which turns a silent gap into a status you can route. The OCR JSON API reference lists the field catalog.
Add a needs_review status for files that pass the API but fail these checks. It is usually the most useful status in the whole table, because it is the only one that points at work a person must do.
Conclusion
Vendor status fields tell you when a job stopped, and that is only part of the story. A reliable OCR status combines that signal with page counts, a text layer check, and a content check, and stores the result per file where you can query it. Treat PARTIAL_SUCCESS, per-file batch errors, and a short X-OCR-Pages count as their own outcome, give stuck processing and accepted rows a deadline, and keep “already has text” separate from “failed”. With those few rules, the answer to “did every document get OCR’d?” becomes a single SQL query instead of a guess.
