Using Machine Learning to Understand How Branding in Photos Affects the Car Shopping Experience

Samad Patel24 Mar 2022

Problem

TrueCar is a car marketplace where shoppers browse dealer inventory through photos. Some dealerships add their name and phone number to the photos. The analytics team wanted to know whether shoppers behave differently with dealers who do this, measured as a difference in sales. Two things got in the way: nobody had labeled which of the terabytes of photos carry branding, and raw sales say little about what was expected of each dealer.

Example from the article: 50 million dealer photos had to be turned into a count of branded photos per dealership before any sales comparison was possible.

Solution in brief

Run every photo through managed OCR to find text, classify that text to see whether it holds dealer details, and aggregate to dealer-level branding counts. Compare dealers by the error of an existing sales prediction model instead of raw sales. Test it with a linear regression and with a t-test between "High" and "Low" branders.

Questions this case answers
  • How to estimate whether a seller-chosen content attribute is associated with outcomes when sellers cannot be randomized and the attribute is only visible inside unlabeled images
  • How to extract a structured per-entity feature from millions of images with off-the-shelf models and no labeled training set
  • How to find out whether sellers who add their own branding to listing photos get different buyer behavior or more sales?
  • How to turn millions of listing photos into a per-seller feature when no one has labeled which photos carry branding?

Tools and methods

  • AWS Rekognition — reads any text present in a photo
  • AWS Comprehend — classifies that text as dealer name, phone number and similar details
  • DynamoDB — stores both services' responses for every photo
  • Prediction error of the sales model — actual minus predicted sales per dealership, the outcome metric
  • Linear regression — tests whether branding counts explain the prediction error
  • T-test — compares mean prediction error of High and Low Branders
Continue reading — it's free

Google or a one-time email code. No password.