Product Data Extraction: E-Commerce HTML to a Typed Feed

What Product Data Extraction Actually Means

If you run a catalog, a marketplace, or a data-ingestion pipeline, you have probably already solved the easy half of product data extraction: you can pull the HTML for thousands of product pages. The hard half is everything after that. Every site uses different markup, every template drifts over time, and you need the same product record out of all of them. That is where most pipelines turn into a pile of brittle, per-site selectors that break the week after you ship them.

Product data extraction is the process of converting an arbitrary e-commerce product page's HTML into a typed, validated product record (name, price, specifications, variants, and more) that conforms to a stable schema and can load straight into a downstream feed. It is not the same thing as fetching the page, and it is not the same thing as publishing schema.org markup for search engines. It is the step where messy markup becomes clean, conforming JSON.

The distinction matters because it tells you where quality is won or lost. Fetching either succeeds or fails; there is not much subtlety to it. Extraction is the part where field accuracy, schema adherence, and consistency across templates actually live, and it is the part that quietly eats your engineering time. A model built for this task gives you strict, schema-conforming JSON on every page instead of selectors you maintain forever. Schematron is the family of long-context models Inference.net trained specifically to extract clean, typed JSON from messy HTML, and it is what this guide builds on.

Figure 1: Product Data Extraction Pipeline. Schematron is the extraction layer; fetching, cleaning, validating, and feed formatting are separate, swappable steps you own.

This article walks the whole loop: design a product schema once, clean the HTML, call the model, validate the result, format your feed, and scale it across many sites. If you want the broader framing of turning messy HTML into typed JSON, that is covered in our guide on the HTML-to-JSON extraction API.

Read time: 11 minutes

Separate Crawling and Fetching from Extraction

The single most useful thing you can do before writing any code is to split your pipeline into layers and keep them independent. A product data extraction pipeline has five distinct jobs:

1. Fetch or crawl the product page with your existing stack (requests, a headless browser, or a fetch API).
2. Clean the HTML to strip scripts, styles, and boilerplate.
3. Extract the fields you care about by sending the cleaned HTML and a schema to the model.
4. Validate the typed result against your schema and business rules.
5. Format and store the record in your downstream feed.

Schematron owns exactly one of these: extraction. It is not a proxy network, a browser automation layer, or an anti-bot tool, and it does not try to be. Fetching the page is your job, with whatever tooling already works for you. Keeping fetch and extract as separate, swappable layers is the whole point: marketplace pages in 2026 are harder to retrieve than they used to be, and you want to be able to change how you fetch without touching how you structure the result.

This is also why a product scraping API built around a schema scales better than one built around selectors. The extraction layer does not care which fetcher produced the HTML, so the same extraction logic survives every change to your crawl layer. If you are working in Python, the same fetch-clean-extract loop is laid out step by step in our walkthrough on AI web scraping in Python.

Design the Product Schema

The schema is the most important artifact in the whole pipeline, because with Schematron it is the only control surface. The model does not take a system or user prompt with instructions. It reads your schema (field names, types, and descriptions) and returns JSON that conforms to it. Everything you want the model to do has to be expressed in the schema itself.

Required vs Optional Fields

The first decision is which fields are required and which are optional. A field is required only if it is present on every valid product; if a page can legitimately omit it, make it optional with a sensible default so a missing value does not fail the whole extraction. In practice that means name and price are required, while almost everything else (specs, tags, variants, images, breadcrumbs, currency, availability, identifiers, brand) is optional and defaults to an empty value.

Required fields appear on every valid product; optional fields default to empty so a missing value does not fail the extraction.

Specs, Variants, Tags, Images, and Breadcrumbs

Real product pages carry more than a name and a price, and each of these shapes maps cleanly to a schema construct. Specifications can be a nested typed object when you know the keys in advance (RAM, storage, weight) or a dict[str, str] when they vary product to product. Variants are a list of objects (each with its own size, color, SKU, and price). Tags, image URLs, and breadcrumb trails are all lists of strings.

Because there is no prompt, field descriptions do the work that instructions would normally do. A description of "the primary price of the product, not a struck-through list price" or "the exact product name as shown in the title or primary heading" tells the model how to resolve the ambiguity that every real page contains. Here is the reusable schema in both Pydantic and Zod:

from pydantic import BaseModel, Field


class Variant(BaseModel):
    sku: str | None = Field(default=None, description="Variant SKU or identifier.")
    title: str | None = Field(default=None, description="Variant name, e.g. size or color.")
    price: float | None = Field(default=None, description="Variant price if it differs from the primary price.")


class Product(BaseModel):
    # Required: present on every valid product.
    name: str = Field(
        ...,
        description="Exact product name as shown in the title or primary heading.",
    )
    price: float = Field(
        ...,
        description="Primary price of the product, not a struck-through list price.",
    )

    # Optional: frequently missing, so default to empty values instead of failing.
    currency: str | None = Field(default=None, description="ISO currency code, e.g. USD.")
    availability: str | None = Field(default=None, description="Stock status, e.g. in stock or out of stock.")
    sku: str | None = Field(default=None, description="Primary SKU or GTIN if present.")
    brand: str | None = Field(default=None, description="Brand or manufacturer name.")
    specs: dict[str, str] = Field(
        default_factory=dict,
        description="Key product specifications as name/value pairs.",
    )
    variants: list[Variant] = Field(
        default_factory=list,
        description="Purchasable variants such as size or color options.",
    )
    tags: list[str] = Field(
        default_factory=list,
        description="Tags or labels assigned to the product.",
    )
    images: list[str] = Field(
        default_factory=list,
        description="Image URLs for the product.",
    )
    breadcrumbs: list[str] = Field(
        default_factory=list,
        description="Category breadcrumb trail from the page.",
    )

import { z } from "zod";

const Variant = z.object({
  sku: z.string().optional().describe("Variant SKU or identifier."),
  title: z.string().optional().describe("Variant name, e.g. size or color."),
  price: z
    .number()
    .optional()
    .describe("Variant price if it differs from the primary price."),
});

export const Product = z.object({
  // Required: present on every valid product.
  name: z
    .string()
    .describe("Exact product name as shown in the title or primary heading."),
  price: z
    .number()
    .describe("Primary price of the product, not a struck-through list price."),

  // Optional: frequently missing, so default to empty values instead of failing.
  currency: z.string().optional().describe("ISO currency code, e.g. USD."),
  availability: z
    .string()
    .optional()
    .describe("Stock status, e.g. in stock or out of stock."),
  sku: z.string().optional().describe("Primary SKU or GTIN if present."),
  brand: z.string().optional().describe("Brand or manufacturer name."),
  specs: z
    .record(z.string())
    .default({})
    .describe("Key product specifications as name/value pairs."),
  variants: z
    .array(Variant)
    .default([])
    .describe("Purchasable variants such as size or color options."),
  tags: z
    .array(z.string())
    .default([])
    .describe("Tags or labels assigned to the product."),
  images: z
    .array(z.string())
    .default([])
    .describe("Image URLs for the product."),
  breadcrumbs: z
    .array(z.string())
    .default([])
    .describe("Category breadcrumb trail from the page."),
});

From Messy HTML to a Typed Record: A Worked Example

With a schema in hand, the rest of the pipeline is short. Let's take a deliberately messy product fragment and run it all the way to a validated record. This is the same shape of work whether the page comes from a large marketplace or a small store, because the schema (not a per-site selector) drives the extraction.

Clean the HTML First

Schematron was trained on HTML that had been pre-cleaned with lxml to strip scripts, styles, and inline JavaScript, so aligning your preprocessing with that improves accuracy and cuts token count. Other tools (Readability, Trafilatura, BeautifulSoup, or even targeted regex) are acceptable; lxml is simply the one that matches training. Err on the side of removing less rather than more, since the model is robust to leftover noise.

import lxml.html as LH
from lxml.html.clean import Cleaner

HTML_CLEANER = Cleaner(
    scripts=True,
    javascript=True,
    style=True,
    inline_style=True,
    safe_attrs_only=False,
)


def strip_noise(html: str) -> str:
    """Remove scripts, styles, and JavaScript from HTML using lxml."""
    if not html or not html.strip():
        return ""
    try:
        doc = LH.fromstring(html)
        cleaned = HTML_CLEANER.clean_html(doc)
        return LH.tostring(cleaned, encoding="unicode")
    except Exception:
        return ""

Here is the kind of input you are dealing with after fetching and a light clean: inconsistent tags, inline styling, and product facts scattered across the markup.

<nav class="breadcrumbs">
  <a href="/">Home</a> &raquo; <a href="/laptops">Laptops</a> &raquo; <span>MacBook Pro M3</span>
</nav>
<div id="item" style="padding:0">
  <h2 class="title">MacBook Pro M3</h2>
  <p>Price: <b>$2,499.99</b> USD</p>
  <span class="stock">In stock</span>
  <ul info>
    <li>RAM: 16GB</li>
    <li>Storage: 512GB SSD</li>
  </ul>
  <span class="tag">laptop</span>
  <span class="tag">professional</span>
  <span class="tag">macbook</span>
  <span class="tag">apple</span>
</div>

Call Schematron with the Schema

The call uses the OpenAI-compatible API, so if you already use the OpenAI SDK, you only change the base URL and the model id. Pass the cleaned HTML as the user message and the schema through response_format, keep temperature at zero (the model is trained to perform best there), and you are done. The full reference for this lives in the Schematron docs.

import os

from openai import OpenAI

from product_schema import Product
from clean_html import strip_noise

# Client setup (OpenAI-compatible)
client = OpenAI(
    base_url="https://api.inference.net/v1",
    api_key=os.environ.get("INFERENCE_API_KEY"),
)

with open("messy-product.html") as f:
    html = strip_noise(f.read())

resp = client.beta.chat.completions.parse(
    model="inference-net/schematron-v2-small",
    messages=[
        {"role": "user", "content": html},
    ],
    response_format=Product,
    temperature=0,
)

print(resp.choices[0].message.parsed.model_dump_json(indent=2))

The TypeScript path is identical in spirit, using Zod for the schema:

import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { Product } from "./product-schema";

const openai = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const resp = await openai.chat.completions.parse({
  model: "inference-net/schematron-v2-small",
  messages: [{ role: "user", content: html }],
  response_format: zodResponseFormat(Product, "product"),
  temperature: 0,
});

console.log(resp.choices[0].message.content);

The JSON Output

Because the model runs in strict JSON mode, the response conforms to your schema every time. For the fragment above, you get a clean, typed record:

{
  "name": "MacBook Pro M3",
  "price": 2499.99,
  "currency": "USD",
  "availability": "In stock",
  "specs": {
    "RAM": "16GB",
    "Storage": "512GB SSD"
  },
  "tags": [
    "laptop",
    "professional",
    "macbook",
    "apple"
  ],
  "breadcrumbs": [
    "Home",
    "Laptops",
    "MacBook Pro M3"
  ]
}

That is the core of product data extraction in JSON format: messy HTML in, a record that matches your schema out, with no selector to maintain when the page changes next month.

Validate Before You Trust the Record

Strict JSON mode guarantees the output matches your schema's shape, but it does not guarantee the values make business sense. A price can parse as a number and still be wrong; a currency can be missing on a page that should have one. Validate on ingest anyway: parse the response with Pydantic (or your validator of choice), handle validation errors explicitly, and add the business-rule checks your feed depends on.

from pydantic import ValidationError

from product_schema import Product

ALLOWED_CURRENCIES = {"USD", "EUR", "GBP"}


def validate_record(raw: dict) -> Product | None:
    """Return a validated Product, or None to route the record to a review queue."""
    try:
        product = Product.model_validate(raw)
    except ValidationError as exc:
        # Shape did not match the schema. Send to a review queue, do not load.
        print(f"schema validation failed: {exc}")
        return None

    # Business rules: shape is valid, but the values still have to make sense.
    if product.price <= 0:
        print("rejected: price must be greater than zero")
        return None
    if product.currency is not None and product.currency not in ALLOWED_CURRENCIES:
        print(f"rejected: unexpected currency {product.currency}")
        return None

    return product

The pattern is simple, but it is the gate that protects everything downstream. Records that parse cleanly and pass your rules flow into the feed. Records that fail go to a review queue instead of quietly corrupting your catalog, which is exactly the kind of bug you do not want to find three weeks later in production. It is belt-and-suspenders by design, and it costs you almost nothing.

Format the Downstream Product Feed

Once a record is validated, it is your canonical product object, and you can shape it into whatever the consumer needs. Keep the extracted record canonical and do the feed-specific mapping in a separate step, so a single extraction can serve a catalog import, a marketplace listing format, and a search index document without re-running the model.

from product_schema import Product


def to_feed_row(product: Product) -> dict:
    """Map a canonical Product into one downstream feed format.

    Keep extraction and mapping separate so one extraction can serve many feeds.
    """
    return {
        "title": product.name,
        "price": f"{product.price:.2f}",
        "currency": product.currency or "USD",
        "availability": product.availability or "unknown",
        "brand": product.brand or "",
        "attributes": product.specs,
        "labels": product.tags,
        "image_links": product.images,
    }

Separating extraction from mapping keeps both simple. The extraction layer produces one well-typed record per page, and each downstream feed gets its own thin mapping function. When a feed format changes, you touch the mapper, not the extraction.

Scale Extraction Across Many Sites

The reason a schema-first approach scales is that it generalizes. The same Product schema runs against HTML from any source, because extraction is driven by the schema rather than by selectors tuned to one site's DOM. Adding a new site to your pipeline usually means adding a fetcher, not a new parser.

For volume, move to the asynchronous APIs instead of looping synchronous calls. The Batch API accepts up to 50,000 extraction jobs in a single submission, lets you poll status asynchronously, and can send optional webhook updates within a 24-hour window. For smaller bundles, the Group API takes up to 50 requests in one payload and gives you a single callback when the whole group finishes. Both are documented under the Batch API and Group API pages.

One thing to watch as you scale is page size. Each request has a long but finite context window, so very large pages should be chunked or trimmed to the relevant region before extraction. If you are weighing whether to build this orchestration yourself or lean on a managed layer, our piece on build vs buy for an extraction pipeline walks through the tradeoffs.

What It Costs: Schematron V2 Small vs Turbo

Cost on a catalog run is dominated by input tokens, because product pages are large and the JSON you get back is small. That makes the per-token rate the number to optimize, and it is where the two V2 models differ.

Pricing from the current model pages. Turbo is roughly 40% cheaper per token on both input and output.

The table above shows the two list prices: V2 Turbo undercuts V2 Small by roughly 40% on both input and output tokens. The rule of thumb is straightforward: reach for Turbo on high-throughput, cost-sensitive catalog runs at scale, and use Small for the hardest pages, the most complex schemas, or the longest documents where extraction quality matters most.

For a sense of scale, a cleaned product page is usually a few thousand input tokens with a small JSON output, so a run of 1,000 pages lands in the low tens of cents on Turbo. At that price the bottleneck is almost never the extraction model. If high-throughput extraction is your default, Schematron V2 Turbo is the model to start with.

How Good Is the Extraction? Schematron V2 Quality Metrics

Skeptical data engineers should not take "it returns clean JSON" on faith, so here are the numbers from the Schematron V2 launch. Because the model runs in strict JSON mode with 100% schema adherence, valid JSON is not the interesting question; field quality is.

On an LLM-as-judge evaluation (GPT 5.4 as judge, scored 1 to 5), Schematron V2 Small scores 4.060 and V2 Turbo scores 4.039, putting both within a hair of each other and of the previous-generation 8B model at 4.070. On SimpleQA, which measures factual accuracy in web-search-augmented extraction, V2 Small reaches 83.10 and V2 Turbo reaches 79.42, against a GPT-5 Nano no-search baseline of 8.54 that shows how far a general model drifts without grounding. On throughput, the gap reverses in Turbo's favor: on a single H100 with a 10k-input, 500-output workload, Turbo runs 4.14 requests per second to Small's 2.47.

The takeaway lines up with the cost guidance. Small edges Turbo on quality, Turbo wins decisively on speed, and both are close enough to frontier quality that the choice is about throughput and budget, not about whether the extraction will be good enough. For a deeper look at how these models are evaluated, see our breakdown of how extraction models are benchmarked.

When Schematron Is Not the Right Layer

A schema-guided extraction model is not the answer to every web data problem, and knowing where it does not fit will save you time.

• Your problem is fetching, not structuring. If you are fighting rate limits, rendering, or anti-bot defenses, that is a crawler, proxy, or browser-automation problem. Extraction starts after you have the HTML.
• The data is already clean and machine-readable. If a page exposes well-formed JSON-LD or the site offers a product API, parse it directly. You do not need a model to read structured data that is already structured.
• You have one fixed template that never changes. For a single, stable page layout, a maintained CSS or XPath selector can be cheaper than any model call.
• Your inputs are scanned PDFs or images. That is OCR and document-intelligence territory, not HTML extraction.

Being honest about these boundaries is also what makes the rest of the pipeline trustworthy: use the model where messy, varied HTML is the actual problem, and use simpler tools everywhere else.

Frequently Asked Questions

What fields should a product schema include? Make name and price required, and treat the rest (currency, availability, SKU or GTIN, brand, specs, variants, tags, images, breadcrumbs) as optional fields with defaults so a missing value does not fail the extraction.

How do you handle product variants and specifications? Model variants as a list of objects, each with its own attributes like size, color, and price, and model specs as a nested object when the keys are known or a string-to-string map when they vary across products. Field descriptions tell the model how to interpret each one.

Do I send the whole page or just part of it? You can send long pages thanks to the model's long-context window, but each request still has a finite limit, so chunk or trim very large pages to the relevant region before extraction.

How much does product data extraction cost with an API? At list pricing, Schematron V2 Small is $0.05 per million input tokens and $0.25 per million output tokens, and V2 Turbo is $0.03 and $0.15 respectively, so a thousand typical product pages costs in the low tens of cents on Turbo.

Can I use the same schema across different sites? Yes. Because extraction is schema-driven rather than selector-driven, one Product schema runs against HTML from any source, which is the whole reason the approach scales across many sites.

Conclusion

Product data extraction stops being a maintenance treadmill once you treat it as a pipeline of independent layers: design a typed schema once, clean the HTML, send it to a schema-guided model, validate the result, map it into your feed, and scale the whole thing with batch APIs. The schema is the contract that makes every step downstream predictable, and it is what lets one extraction layer serve every site you add.

If you want to turn a real product page into a typed record, the fastest way to see it work is to run your own schema and a page of messy HTML through the model.

Related Reading

• HTML-to-JSON Extraction API: Extract Typed Data from Messy Web Pages. The typed-extraction-versus-DOM-converter foundation this guide builds on.
• AI Web Scraping in Python: HTML to Validated JSON. The full fetch, clean, and validate code path in Python.
• Best LLM for Data Extraction: Schematron Quality Metrics. How to benchmark extraction quality on your own pages.
• Price Scraping with AI: Extract Product Prices into JSON. The price-focused slice of this workflow when you only need the number.
• Best Web Scraping API for E-Commerce Product Data. How to grade fetch and extraction APIs for product records by cost and quality.