Company
Thumbtack
10 cases from Thumbtack (Marketplace, Adtech), each digested from a public talk or engineering post and linked to the original. 7 were checked in an online A/B test; 2 report results in production.
Tasks: Evaluate models, Rank search results, Measure impact without an A/B test, Get experiment results faster, Recommend from what the user just did and 3 more. Methods: LLM-as-a-judge, Human in the loop, Gradient boosting, Position bias correction, Deep & Cross Network, SHAP and 24 more. Tools: MLflow, SHAP, LangGraph, OpenAI API, PyTorch, Amazon S3 and 10 more.
Tech blog: thumbtack.com
| Workbench: Building an AI Content Generation and Quality Assurance System at Scale | marketplace | 2026-10 | |
| Building a Transformer-Based Category Recommender at Thumbtack | marketplace | 2026-09 | |
| Treating Pricing Changes Like Code Deploys | marketplace | 2026-07 | |
| Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI | marketplace | 2026-04 | |
| Automating Ad Generations with LLMs | marketplace | 2025-07 | |
| How We Personalized Recommendations for Professionals on Thumbtack | marketplace | 2024-06 | |
| Accelerating Ranking Experimentation at Thumbtack with Interleaving | marketplace | 2023-07 | |
| Leveraging Switchbacks: Enhancing Market-Based Testing at Thumbtack | marketplace | 2023-07 | |
| Leaves to Neurons: Using Deep-Cross Networks for Ranking Search Results | marketplace | 2023-05 | |
| Evolution of Search Ranking at Thumbtack | marketplace | 2022-12 |