Extracting Product Attributes from PDFs using PAE Framework

Apurva Sinha28 Jun 2024

Problem

Retail assortment planners get fashion trend forecasts as PDF reports: unstructured text next to pictures. The reports describe products in detail (knitted sweater vests, roll neck, slouchy fit), but nothing is structured. Until attributes are extracted, the content cannot be linked to the catalog or used to decide which products to carry. The goal is catalog-ready attribute values read straight from such PDFs.

Example from the article: a trend-report page showing knitted sweater vests with a roll neck and button detailing, from which Color, Sleeve Style, Product Type, Material, Features, Categories, Age and Neck are extracted.

Solution in brief

The Product Attribute Extraction (PAE) engine splits each PDF into text and images and runs them in parallel through two LLM prompts, one over the OCR text and one over the images. Attributes are merged per page. A pre-trained BERT model then embeds the extracted and catalog values, and cosine similarity matches each extracted value to a catalog value.

Questions this case answers
  • How to turn unstructured multi-page documents with text and images into a fixed set of structured fields without labeled training data
  • How to normalize free-form extracted values to the controlled vocabulary of an existing catalog
  • How to pull structured product attributes out of PDF trend reports that mix text and pictures?
  • How to map attribute values extracted by an LLM onto the values already in the product catalog?

Tools and methods

  • OCR with spell correction — turns page images into text, then fixes OCR errors with LanguageTool
  • Text LLM prompt — pulls eight attributes from the page text
  • Vision LLM prompt — pulls the same attributes from base64-encoded images
  • BERT (uncased) embeddings with cosine similarity — maps extracted values to catalog values
Continue reading — it's free

Google or a one-time email code. No password.