What are you trying to do?

Pick a task, narrow by your constraint — see how other companies solved it.

Experiments & causal

Measure what really changed — when you can't randomize, when traffic is thin, and when users affect each other.

Measure impact without an A/B test

Launches, campaigns and policies you could not randomize: diff-in-diff, synthetic control, matching, double ML.

61 cases →
Actual vs synthetic control
launchpre-launch fitactualsyntheticeffectbeforeafter

Trustworthy A/B tests

Faster answers on thin traffic, and no false wins from many metrics, variants and peeks.

raw metricafter CUPEDCICI
spillovertreatedcontrol

Experiment when users affect each other

Marketplaces, networks and logistics where treated users leak into control: switchbacks, clusters, geo.

30 cases →

Decide who gets an offer

Give the promo only to people who change their mind because of it: uplift, CATE, next best action.

47 cases →
Sleeping dogsSure thingsLost causesPersuadablestreat only those who buy because of the offer

Predict long-term effect from short tests

Judge retention, LTV and revenue a year out from a two-week experiment: surrogates and holdouts.

23 cases →
2-week test1-year effectsurrogate model

Metrics & behavior

Know why a number moved, what users really do, and what is coming next.

Explain a metric change

Find which segment, platform or factor moved the metric: decomposition, drill-down, automated root cause.

25 cases →
Why did orders drop?
last wkiOSAndroidWebnew usersthis wkWeb explainsmost of the drop
incidentexpected range

Detect anomalies

Catch incidents, silent failures and broken data before users notice.

63 cases →
orders, dailynext 10 daysweekend peaks

Forecast demand and load

Plan supply, couriers, stock and capacity from demand forecasts with honest uncertainty.

48 cases →
opensearchfeedcartbuyleave

Analyze user behavior

Paths, funnels, cohorts and habits: what users actually do in the product.

31 cases →

Make metric definitions consistent

Metric trees and semantic layers so every team computes revenue, orders and retention the same way.

38 cases →
Metric tree
GMVordersavg checkbuyersfrequencypriceitems

Search & recommendations

Show each user the right item out of millions — fast, fresh and not the same thing twice.

running shoesTrail runner 30.92Road racer0.78Running socks0.41Yoga mat0.12

Rank search results

Query understanding, retrieval and learning to rank for product and content search.

136 cases →
10Mall items1,000candidate generation100ranking10shown to the user

Retrieve and rank from millions of items

Candidate generation, two-tower retrieval and a ranker that fit into a latency budget.

128 cases →
runnershome cooksgamersnew user → nearest

Learn user and item embeddings

Train user, item and text embeddings once, reuse them across search, recs and models.

70 cases →

Fresh users, fresh items

New users and new items with no history, and intent read from the last few actions.

no clicks yetbrandcategorytextnew itemsimilar items
relevance onlywith diversity

Balance relevance and diversity

Keep feeds relevant without showing the same thing again: diversity, exploration, multiple goals.

41 cases →

Prediction & decisions

Turn predictions into decisions: prices, budgets, risk and trust.

Optimize decisions

Prices, budgets, allocations, routes and schedules chosen by models, not by gut.

91 cases →
Price vs profit
best pricedemandprofitprice →
1 card6 accounts, one payment card

Catch fraud and abuse

Fraud rings, fake accounts and abuse under adversarial pressure and rare positives.

71 cases →
model scorelisting #1removecomment #2reviewphoto #3okreview #4remove

Moderate content at scale

Spam, safety and policy checks on listings, reviews and messages, with people for the hard cases.

56 cases →
iPhone 15 128GBshop A · blackApple iPhone15shop B · 128 gb=iPhone 152 offersmatch score 0.97

Match entities

One product, one seller, one customer: entity resolution across messy records.

37 cases →

Predict churn and lifetime value

Who is about to leave and who is worth the effort: churn and LTV models that drive action.

25 cases →
low riskmediumhigh riskact heredays since signup →

LLM & data products

LLM features that answer correctly, cite their sources and get measured before they ship.

Evaluate models

Offline and online evaluation of ML models, LLMs and agents: metrics, judges, benchmarks, monitoring.

163 cases →
Eval run
v1v2v3accuracyfaithfulnesslatency ↓LLM judge + humans
Which cities grew orders?SELECT city, orders_momFROM metrics.ordersORDER BY 2 DESC LIMIT 3answer + chart

Answer questions over data

Text-to-SQL, BI assistants and analytics agents that answer questions about your data.

52 cases →
Refund rules for B2B?[1][2][3]answer[1][3]retrieve chunks → answer with citations

Build an assistant over internal docs

RAG and assistants over internal documents, support articles and knowledge bases.

59 cases →

Label data with LLMs

Replace slow manual labeling with LLM juries, then distill the labels into a small model.

33 cases →
LLM 1✓LLM 2✓LLM 3✗label: foodsmall modelfine-tunedmajority vote of LLM judges, spot-checked by people

Platform & cost

The platform underneath: data you can trust and models you can afford.

app eventspaymentslogslakehousefeaturesBImodelsgoverned, one source of truth

Build a data platform

Pipelines, lakehouse, feature store, governance and MLOps that teams actually use.

90 cases →
large modeldistillsmallcostlatencyqualitysame quality, a fraction of the cost

Reduce ML cost

Distillation, quantization and serving tricks that cut training and inference cost.

60 cases →

Your task doesn’t fit a box?

Describe it to your own AI agent. Through MCP it searches this archive, compares cases and tells you what’s missing.

Connect via MCP