Shipping Label OCR: Read Tracking Numbers, Addresses, and Parcel References

If you searched for shipping label OCR, you probably have parcels where the barcode will not scan, a returns desk that keys tracking numbers and addresses by hand, or a pile of label photos from drivers and customers that nobody can search. Short answer: shipping label OCR is the fallback and the audit trail, not a replacement for the barcode. Decode the barcode when it reads. Run OCR on the label photo to get the printed tracking number, the ship-to block, and any order or PO references as text. Then compare the two in code. When the printed tracking number and the barcode disagree, or the barcode routes to a different ZIP Code than the address says, the parcel goes to a person before it moves.

This post covers parcel and carton labels: the sticker on the box, not the shipping paperwork. For bills of lading, read bill of lading OCR. For commercial invoices, packing lists, and AWBs in one forwarding pipeline, read freight forwarding OCR. For reconciling carton contents against the PO at receiving, read packing list OCR.

What OCR means in shipping

In shipping and parcel logistics, OCR means optical character recognition: software that turns the printed characters on a label, a document, or a container door into machine-readable text. It sits next to barcode scanning, and the two do different jobs.

A barcode scanner decodes the bars. It is fast and precise, but it only returns what the shipper encoded, and it returns nothing when the label is torn, wet, wrinkled over a box edge, or printed too light. OCR reads the human-readable text around and under the barcode, plus everything that was never encoded at all: the recipient name, the street line, a handwritten apartment number, a return authorization number, a “fragile” note.

So the practical questions for a shipping label OCR project are narrower than “can it read text”. Which values on the label have a barcode behind them, which do not, and what should happen when the two sources disagree?

What is actually printed on a parcel label

Two standards explain most of what you will see on labels in North America and Europe.

For USPS parcels, the Domestic Mail Manual section 204 describes the Intelligent Mail package barcode (IMpb) as a data string that generally follows the GS1-128 specification and carries both tracking and routing data. A separate postal routing barcode uses Application Identifier 420 followed by the 5-digit ZIP Code or ZIP+4 of the delivery address. On a single concatenated barcode, the human-readable text suppresses the 420 and the routing ZIP, but it must include the tracking application identifier (92, 93, 94, or 95). That detail matters later: the ZIP Code the barcode routes to is not printed under the barcode, so a scan and an OCR read of the address are two independent views of the same destination. The full barcode construct, including the MOD 10 check digit on the package identification code, is in USPS Publication 199.

For pallets and cartons between businesses, the GS1 Logistic Label Guideline says the Serial Shipping Container Code (SSCC) is the single mandatory element. The label can have up to three segments: a carrier segment (ship-to postal code in AI 420, consignment number, routing), a customer segment (ship-to location, purchase order number, handling), and a supplier segment (SSCC, plus GTIN, batch, or dates when used).

Carrier-specific labels from UPS, FedEx, DHL, and national posts follow their own specs. The pattern is the same, though:

Label value	Barcode behind it?	Where OCR helps
Tracking number	Yes, as printed digits under the bars	Damaged barcode, photo-only intake, audit
Routing ZIP / postal code	Often encoded, often not printed under the bars	Cross-check against the printed address
Recipient name and street	Usually not	This is the main OCR job
Order, PO, RMA references	Sometimes, in a separate barcode	Matching returns and inbound to records
Service level, weight, ship date	Rarely	Exception triage, billing disputes
SSCC on pallet labels	Yes, GS1-128 with human-readable text	Damaged pallet labels, check-digit validation

What shipping label OCR should return

For a receiving dock or returns desk, the useful output is short:

• Tracking number as printed, digits only, ready to compare with the scan
• Ship-to block: name or company, street and number, unit, city, state or region, postal code, country when shown
• Ship-from block, kept separate so it never overwrites the recipient
• References: order number, PO, RMA, customer reference
• Service and handling: service level, weight, ship date, any handling marks

Everything else on the label is noise for this job. Keep the full text anyway, because the question in a billing dispute three weeks later is rarely the one you planned for.

Read the whole label as Markdown first

OCRskill exposes two contracts on the same Bearer-token auth. POST /ocr returns Markdown for any page. POST /ocr.json returns typed JSON for the fields you list. Both accept phone photos and scans (PNG, JPEG, WebP, GIF, BMP, TIFF) and documents such as PDF, up to 20 MB per request. A free key from get-key.json is enough to test.

curl https://api.ocrskill.com/get-key.json

export API_KEY="sk-your-key-here"

curl https://api.ocrskill.com/ocr \
  -H "Authorization: Bearer $API_KEY" \
  -F "file=@label.png" > label.md

We ran a synthetic USPS-style test label (fictional sender, recipient, and tracking number) through /ocr. The Markdown came back as a heading for the service banner and an HTML table for the label blocks:

# USPS GROUND ADVANTAGE

<table>
  <tr>
    <td>EXAMPLE OUTDOOR SUPPLY CO<br>1200 WAREHOUSE RD STE 4<br>RENO NV 89502</td>
    <td>SHIP DATE: 10/12/2026<br>WEIGHT: 2 LB 6 OZ</td>
  </tr>
  <tr>
    <td colspan="2">
      <strong>SHIP TO:</strong><br>
      JANE EXAMPLE<br>
      742 EVERGREEN TER APT 3B<br>
      SPRINGFIELD IL 62704-1234
    </td>
  </tr>
  <tr>
    <td colspan="2">
      <strong>USPS TRACKING # EP</strong><br>
      ![image](image_1.png)<br>
      9400 1000 0000 0000 0000 00
    </td>
  </tr>
  <tr>
    <td colspan="2">REF: ORDER 58213 PO 4500018823</td>
  </tr>
</table>

Two things to notice. The sender and recipient stayed in separate cells, which is what makes the ship-to block parseable. And the barcode itself came back as an image placeholder, while the printed digits underneath it came back as text. OCR gave us the human-readable tracking number, not the decoded bars. Decode the barcode with a barcode library such as ZXing or with the handheld scanner you already have, and treat the two as separate inputs.

Typed address fields for the ship-to block

The OCR JSON catalog has address fields (street_name, street_number, address_rest, town_or_city, county, region, country) and person fields (first_name, last_name). Mark them optional with a trailing ? so one smudged line does not reject the whole request:

curl "https://api.ocrskill.com/ocr.json?fields=first_name?,last_name?,street_name?,street_number?,address_rest?,town_or_city?,region?,country?" \
  -H "Authorization: Bearer $API_KEY" \
  -F "file=@label.png"

On the same test label, the response was:

{
  "first_name": "JANE",
  "last_name": "EXAMPLE",
  "street_name": null,
  "street_number": "742",
  "address_rest": "APT 3B",
  "town_or_city": "SPRINGFIELD",
  "region": "IL",
  "country": null
}

That is a useful start and an honest warning in one result. The fields came from the recipient, not the sender, on this label. street_name came back null even though EVERGREEN TER is printed, so the street line still has to come from the Markdown when the typed field is empty. There is no postal code field in the catalog today, and no tracking number field either. Do not invent postal_code or tracking_number in the fields list, because unknown field names return 400. Pull those from the Markdown in your own code, and have your parser accept both a missing key and a null value.

A label also has two addresses on it, and a typed request returns one set of address fields. If your labels come from a small number of templates, such as your own WMS output or one carrier’s returns label, crop the image to the ship-to block before calling /ocr.json. That removes the question of which address the extractor picked.

Check the printed label against the barcode

This is where shipping label OCR earns its place. The script below takes the Markdown from /ocr and the decoded barcode payload from your scanner or decoder. It finds the printed tracking number under the TRACKING banner, finds the ZIP Code in the ship-to block, and splits a USPS concatenated barcode into its routing ZIP (after AI 420) and tracking string. Then it checks that the printed tracking number matches the end of the barcode data and that the barcode routes to the same ZIP Code the address shows.

import json
import re
import sys

GS = "\x1d"  # FNC1 / group separator many scanners emit between GS1 elements


def label_text(markdown: str) -> list[str]:
    """Flatten OCR Markdown (including HTML table cells) into plain lines."""
    text = re.sub(r"<br\s*/?>", "\n", markdown, flags=re.I)
    text = re.sub(r"<[^>]+>", "\n", text)
    text = re.sub(r"!\[[^\]]*\]\([^)]*\)", "", text)  # drop image placeholders
    return [line.strip() for line in text.splitlines() if line.strip()]


def printed_tracking(lines: list[str]) -> str | None:
    """Digits of the human-readable tracking number under the barcode."""
    for i, line in enumerate(lines):
        if "TRACKING" in line.upper():
            for candidate in lines[i + 1 : i + 4]:
                digits = re.sub(r"\D", "", candidate)
                if len(digits) >= 20:
                    return digits
    return None


def ship_to_zip(lines: list[str]) -> str | None:
    """First ZIP (or ZIP+4) printed after the SHIP TO heading."""
    for i, line in enumerate(lines):
        if re.match(r"^(SHIP\s+TO|TO)\b", line, flags=re.I):
            for candidate in lines[i : i + 6]:
                match = re.search(r"\b(\d{5})(?:-\d{4})?\b", candidate)
                if match:
                    return match.group(1)
    return None


def barcode_parts(payload: str) -> dict:
    """Split a decoded USPS concatenated IMpb: AI 420 + ZIP, then the tracking string."""
    if not payload.startswith(GS + "420") and not payload.startswith("420"):
        return {"routing_zip": None, "tracking": re.sub(r"\D", "", payload)}
    body = payload.lstrip(GS)[3:]
    if GS in body:
        zip_part, tracking = body.split(GS, 1)
    else:
        zip_part, tracking = body[:5], body[5:]  # no separator: only the 5-digit ZIP is safe
    return {"routing_zip": zip_part[:5], "tracking": re.sub(r"\D", "", tracking)}


def check(markdown: str, payload: str) -> dict:
    lines = label_text(markdown)
    hri = printed_tracking(lines)
    printed_zip = ship_to_zip(lines)
    bar = barcode_parts(payload)
    issues = []
    if not hri:
        issues.append("no printed tracking number found")
    elif not bar["tracking"].endswith(hri):
        issues.append("printed tracking number does not match the barcode")
    if bar["routing_zip"] and printed_zip and bar["routing_zip"] != printed_zip:
        issues.append(f"barcode routes to {bar['routing_zip']}, ship-to says {printed_zip}")
    if not printed_zip:
        issues.append("no ship-to ZIP found")
    return {
        "tracking": hri,
        "ship_to_zip": printed_zip,
        "barcode_zip": bar["routing_zip"],
        "status": "review" if issues else "ok",
        "issues": issues,
    }


if __name__ == "__main__":
    markdown = open(sys.argv[1], encoding="utf-8").read()
    payload = sys.argv[2].replace("\\x1d", GS)
    print(json.dumps(check(markdown, payload), indent=2))

It needs Python 3.10 or newer. Run it with the Markdown file and the decoded payload, writing the group separator as \x1d. Against the test label with a barcode payload that routes to a different ZIP Code, the output is:

python3 check_label.py label.md '\x1d42062701\x1d9400100000000000000000'

{
  "tracking": "9400100000000000000000",
  "ship_to_zip": "62704",
  "barcode_zip": "62701",
  "status": "review",
  "issues": [
    "barcode routes to 62701, ship-to says 62704"
  ]
}

With a payload of \x1d42062704\x1d9400100000000000000000, the same label returns "status": "ok" and no issues. A payload whose tracking digits differ from the printed ones returns "printed tracking number does not match the barcode".

Adapt it before you trust it. The TRACKING and SHIP TO anchors match this test layout, so add the banners your carriers actually print. Other carriers use different tracking formats and different barcode payloads, so add one parser per carrier from that carrier’s own spec rather than stretching the USPS one. Non-US postal codes need their own patterns. A match on both checks tells you the label is internally consistent, not that it belongs to the order you think it does, so still look the tracking number up in your shipping or returns system.

Pallet labels and the SSCC

On pallet and carton labels built to the GS1 guideline, the value to protect is the SSCC: 18 digits, with the last one a check digit computed by the GS1 check digit method. When a pallet label is damaged and the SSCC has to come from OCR, validate the check digit before you look up the despatch advice. The packing list OCR post includes a runnable SSCC checker you can reuse. If a label pairs the SSCC with a customer segment that prints a PO number, compare that PO with the one on the packing list in the carton pocket.

A practical pipeline at the dock or returns desk

Keep each step small enough that the person who owns the exception queue can explain it:

1. Scan first. Decode the barcode with the scanner. If it reads and the parcel is expected, you may not need OCR at all for routing.
2. Photograph exceptions. No-reads, damaged labels, unexpected parcels, and returns without a prepaid label get one flat, well-lit photo of the whole label.
3. Markdown OCR. Run /ocr on the photo and store the Markdown next to the image and the scan result.
4. Optional typed address. Run /ocr.json with optional address fields when you need structured ship-to data, ideally on a crop of the ship-to block.
5. Cross-check. Compare the printed tracking number with the scan, the barcode ZIP with the printed ZIP, and any PO or RMA reference with your records.
6. Route or hold. Consistent labels move on. Anything that fails goes to review with the photo, the Markdown, and the failed check side by side. Re-labeling, redirecting, or refusing a parcel stays a human decision.

For volume, two paid-key features help. A document URL lets OCRskill fetch a photo already in your object storage, and an OCR callback returns 202 and posts the result to your endpoint later. Free keys get 403 for both. If the photos should also land in a searchable archive, the logistics paperless document workflow covers folders and Paperless-ngx.

Limitations to plan for

• OCR does not decode barcodes. In our test the barcode came back as an image placeholder. Use a barcode decoder for the bars and OCR for the text.
• Photo quality drives everything. Labels wrapped around box edges, glare from shrink wrap, thermal print that has faded, and motion blur from a handheld photo all raise error rates. A fixed capture station beats phone photos taken on the floor.
• Two addresses, one set of fields. Typed address extraction returns one address. Crop to the ship-to block or verify against the Markdown before you store it.
• Catalog gaps. There is no postal code or tracking number field. Those come from the Markdown and your parsers.
• Handwriting and overlays. Handwritten unit numbers, relabel stickers over old labels, and customer notes on returns are where most corrections happen. Measure them separately.
• Personal data. A label holds a name and a home address. Keep label photos and OCR output under the same retention and access rules as the rest of your customer data.

Measure before you promise accuracy. Take a hundred real exception labels from the last month, run them through the pipeline, and count how often a clerk still fixes the tracking number, the street line, or the postal code. That number tells you which label sources can move with light review and which still need a person on every parcel.

Where OCRskill fits

OCRskill handles the reading step: label photo in, Markdown or typed JSON out, with a clear 400 when a required field is missing or a field name is unknown. It does not decode barcodes, validate addresses against a postal database, or talk to carrier tracking systems. Matching a parcel to an order, deciding whether to re-label it, and releasing it to the next leg stay in your WMS, your carrier integrations, and your people.

Conclusion

Shipping label OCR is most useful in the places the barcode cannot cover: the no-read, the return without paperwork, the recipient line nobody encoded, and the dispute where you need to show what the label said. Treat the barcode and the printed text as two independent readings of the same parcel. Read the label as Markdown, take typed address fields as a hint, pull tracking numbers and postal codes with your own parsers, and hold any parcel where the two readings disagree. Start with one exception queue and one label source, measure what clerks still correct, and widen from there.

To try it, get a key from get-key.json, run a photo of a damaged or returned label through /ocr, and feed the Markdown and the scanner output into the checker above. Product details and pricing are at ocrskill.com.