# effectsize.io > Applied product analytics and ML cases from real companies. Each case digests one public talk or engineering post: the problem, the method that solved it, the data needed, the evidence and the pitfalls, with a link to the original. ## Topics - [Clickstream and event analysis](https://effectsize.io/cases/clickstream/): Sessionization, paths and funnels, process mining, sequence mining, transitions between events, journey maps, tracking plan and event design. - [User behavior analysis](https://effectsize.io/cases/behavior/): Retention, cohorts, activation, aha-moment, engagement, behavioral segmentation and clustering, behavior-based churn analysis. - [Finding insights in data](https://effectsize.io/cases/insights/): Metric change analysis (root cause), metric decomposition, anomaly detection, auto-insights, exploratory analysis frameworks. - [Product metrics](https://effectsize.io/cases/metrics/): North Star, metric trees, proxy and surrogate metrics, guardrails, LTV/unit economics in a product context. - [A/B tests](https://effectsize.io/cases/ab-testing/): Design, power and MDE, variance reduction (CUPED, stratification), sequential testing, multiple comparisons, SRM, peeking, ratio metrics, bootstrap, experimentation platforms. - [Advanced experiments](https://effectsize.io/cases/experimentation-advanced/): Network effects and interference, two-sided marketplaces, switchback, cluster randomization, long-term effects, bandits, interleaving. - [Quasi-experiments and causal inference](https://effectsize.io/cases/causal/): DiD, synthetic control, RDD, IV, matching/PSM, CausalImpact, effect estimation without an A/B test, double ML. - [Uplift modeling and HTE](https://effectsize.io/cases/uplift/): CATE, meta-learners (S/T/X/R/DR), uplift trees, communication and promo targeting, policy learning, uplift evaluation (Qini, AUUC). - [Recommender systems](https://effectsize.io/cases/recsys/): Candidate generation, ranking, embeddings, sequential/session-based recommendations, cold start, diversity, feedback loops, offline/online evaluation. - [Personalization](https://effectsize.io/cases/personalization/): Personalized UX, content, notifications, pricing and offers, contextual bandits, next-best-action. - [Search](https://effectsize.io/cases/search/): Ranking, learning-to-rank, query understanding, autocomplete, relevance and its labeling, search metrics, semantic/hybrid search in a product. - [Predictive product analytics](https://effectsize.io/cases/predictive/): Churn prediction, propensity, LTV prediction, lead scoring, metric forecasting. - [Growth and monetization through data](https://effectsize.io/cases/growth/): Onboarding analysis, paywall/pricing experiments, reactivation, virality — when there is an analytical or experimental part. - [Anti-fraud and trust & safety](https://effectsize.io/cases/fraud/): Fraud and abuse detection in a product, transaction and authentication scoring, spam and fake objects/accounts, fraud economics and choosing the anti-fraud target, graph features, golden set for moderation. - [Analytics organization](https://effectsize.io/cases/analytics-practice/): Data-informed processes, experimentation culture, how an analyst works with the product, hypothesis prioritization. - [LLM in analytics and product](https://effectsize.io/cases/llm-in-analytics/): LLMs for data analysis, evaluating LLM features with experiments, LLM ranking and recommendations. - [Infrastructure for analytics](https://effectsize.io/cases/data-infra/): Event pipelines, event stores, feature store — only if directly tied to the topics above. ## Tasks - [How to measure impact without an A/B test](https://effectsize.io/tasks/measure-without-ab-test/): estimate the effect of a launch, campaign or policy when you could not randomize: diff-in-diff, synthetic control, matching, instrumental variables, double ML - [How to get experiment results faster](https://effectsize.io/tasks/speed-up-experiments/): reach a confident A/B readout with less traffic or time: variance reduction (CUPED), sequential testing, interleaving, power and MDE planning - [How to experiment when users affect each other](https://effectsize.io/tasks/experiment-with-interference/): test changes in marketplaces, networks and logistics where treated users affect control: switchbacks, cluster and geo randomization - [How to avoid false wins in A/B tests](https://effectsize.io/tasks/avoid-false-wins/): keep A/B results honest with many metrics, variants and peeks: multiple-testing control, SRM checks, Bayesian readouts, novelty effects - [How to predict long-term effect from short tests](https://effectsize.io/tasks/predict-long-term-effect/): judge the long-term impact (retention, LTV, revenue) of a change from a short experiment: surrogate and proxy metrics, holdouts - [How to decide who gets an offer](https://effectsize.io/tasks/target-offers/): choose who should get a promo, discount, message or intervention and which one: uplift modeling, CATE, next best action, contextual bandits - [How to explain a metric change](https://effectsize.io/tasks/explain-metric-change/): find why a metric moved: root cause analysis, decomposition, drill-down, attributing a change to segments or factors - [How to analyze user behavior](https://effectsize.io/tasks/analyze-user-behavior/): understand how users actually behave: paths, funnels, retention, cohorts, habits and engagement patterns - [How to make metric definitions consistent](https://effectsize.io/tasks/align-metric-definitions/): make every team compute a metric the same way: metric trees, semantic layers, governed definitions across fragmented data - [How to detect anomalies](https://effectsize.io/tasks/detect-anomalies/): catch incidents, silent failures, data-quality breaks and unusual behaviour in metrics, pipelines and models before users notice - [How to forecast demand and load](https://effectsize.io/tasks/forecast-demand/): predict demand, orders, load or business metrics ahead of time to plan supply, staff, inventory and capacity - [How to rank search results](https://effectsize.io/tasks/rank-search-results/): retrieve and rank results for a query: product search, relevance, query understanding, semantic and hybrid retrieval - [How to retrieve and rank from millions of items](https://effectsize.io/tasks/build-two-stage-recommender/): serve recommendations from millions of items within latency budgets: candidate generation, two-tower retrieval, then ranking - [How to recommend to new users and new items](https://effectsize.io/tasks/recommend-cold-start/): recommend to brand-new users and surface brand-new items with little or no interaction history - [How to recommend from what the user just did](https://effectsize.io/tasks/recommend-from-sessions/): use the sequence of a user's recent actions to predict what they want next: sequential and session-based recommenders, transformers over events - [How to balance relevance and diversity](https://effectsize.io/tasks/balance-feed-goals/): keep feeds and recommendations relevant without collapsing into the same items: diversity, exploration, multiple objectives - [How to learn user and item embeddings](https://effectsize.io/tasks/learn-embeddings/): train user, item, text or transaction embeddings once and reuse them across search, recommendations and prediction models - [How to predict churn and lifetime value](https://effectsize.io/tasks/predict-churn-ltv/): predict who will churn or how much a customer is worth, and use the score to retain or prioritize them - [How to optimize decisions](https://effectsize.io/tasks/optimize-decisions/): choose prices, budgets, allocations, routes or schedules: pricing, elasticity, marketing mix, operations optimization - [How to catch fraud and abuse](https://effectsize.io/tasks/catch-fraud/): find fraud, abuse, fake accounts and policy violations under adversarial pressure and rare positives - [How to moderate content at scale](https://effectsize.io/tasks/moderate-content/): review user content, listings and messages at scale for spam, safety and policy violations with ML and LLMs - [How to match entities](https://effectsize.io/tasks/match-entities/): link or deduplicate records and objects: product matching, entity resolution, account merging - [How to answer questions over data](https://effectsize.io/tasks/answer-questions-over-data/): let people ask questions of structured data in natural language: text-to-SQL, BI assistants, analytics agents - [How to build an assistant over internal docs](https://effectsize.io/tasks/build-rag-assistant/): answer questions over internal documents and knowledge bases: RAG, support assistants, chatbots over company texts - [How to label data with LLMs](https://effectsize.io/tasks/label-data-with-llms/): replace or speed up manual labeling with LLMs: LLM labeling and juries, distillation into small models, label quality checks - [How to evaluate models](https://effectsize.io/tasks/evaluate-models/): measure the quality of ML and LLM models or agents offline and in production: metrics, LLM judges, benchmarks, monitoring - [How to build a data platform](https://effectsize.io/tasks/build-data-platform/): build data and ML infrastructure: pipelines, storage, feature stores, governance, access control, MLOps, LLM platforms - [How to reduce ML cost](https://effectsize.io/tasks/reduce-ml-cost/): make training or inference cheaper and faster: distillation, quantization, serving efficiency, model consolidation ## Cases - [From Data Silos to Feature Serving: How We Unified Fragmented Data for Training, Inference, and Reuse](https://effectsize.io/cases/from-data-silos-to-feature-serving-how-we-unified-fragmented/): HubSpot - [PredictPay: From Hackathon Idea to Smarter Payments at Scale](https://effectsize.io/cases/predictpay-from-hackathon-idea-to-smarter-payments-at-scale/): Affirm - [From Activity to Intent: Generating User Journeys with LLMs](https://effectsize.io/cases/from-activity-to-intent-generating-user-journeys-with-llms/): Pinterest - [Metrics Board: Building an Agent-ready Metrics Layer](https://effectsize.io/cases/metrics-board-building-an-agent-ready-metrics-layer/): Pinterest - [Personalization without user identity](https://effectsize.io/cases/personalization-without-user-identity/): Airbnb - [The guest journey, updated in real time: extending Airbnb’s sequence recommender with Chronon](https://effectsize.io/cases/the-guest-journey-updated-in-real-time-extending-airbnb-s/): Airbnb - [Learning Richer Credit Representations with Transformers at Affirm](https://effectsize.io/cases/learning-richer-credit-representations-with-transformers-at/): Affirm - [Refreshing the Travel-Time Map Behind Lyft’s Marketplace: Rebuilding Neighborhood Reachability Signals](https://effectsize.io/cases/refreshing-the-travel-time-map-behind-lyft-s-marketplace/): Lyft - [The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scale](https://effectsize.io/cases/the-lifecycle-of-llm-as-a-judge-building-aligning-and/): Netflix - [MAPS: Netflix’s Multimodal Asset Personalization at Scale](https://effectsize.io/cases/maps-netflix-s-multimodal-asset-personalization-at-scale/): Netflix - [Developing a model to detect harmful content from OpenChat titles and descriptions](https://effectsize.io/cases/developing-a-model-to-detect-harmful-content-from-openchat/): LY Corporation - [Correlation Lied to Us: Rethinking Product Impact with Causal Inference](https://effectsize.io/cases/correlation-lied-to-us-rethinking-product-impact-with-causal/): OLX - [Personalizing Airbnb search by learning from the guest journey](https://effectsize.io/cases/personalizing-airbnb-search-by-learning-from-the-guest/): Airbnb - [Online Learning for Cost-Efficient LLM Routing](https://effectsize.io/cases/online-learning-for-cost-efficient-llm-routing/): Ramp - [Variance Reduction Below the Randomization Grain](https://effectsize.io/cases/variance-reduction-below-the-randomization-grain/): Instacart - [Leveraging PyFixest for High-Cardinality Marketplace Modeling at Instacart](https://effectsize.io/cases/leveraging-pyfixest-for-high-cardinality-marketplace/): Instacart - [A Human-Augmenting Agentic Workflow for Causal Inference](https://effectsize.io/cases/a-human-augmenting-agentic-workflow-for-causal-inference/): Netflix - [Advanced SQL Patterns for Production Analytics](https://effectsize.io/cases/advanced-sql-patterns-for-production-analytics/): Databricks - [Beyond Dashboards: How Grupo Bimbo Delivers Insights with Databricks AgentBricks](https://effectsize.io/cases/beyond-dashboards-how-grupo-bimbo-delivers-insights-with/): Grupo Bimbo - [Databricks on Databricks: Automating Service Reliability at Scale](https://effectsize.io/cases/databricks-on-databricks-automating-service-reliability-at/): Databricks - [Evolution of the Databricks Commercial Motion: Building Scalable Intelligence on Databricks](https://effectsize.io/cases/evolution-of-the-databricks-commercial-motion-building/): Databricks - [Ultra Low Latency Streaming for Real-Time Feature Engineering](https://effectsize.io/cases/ultra-low-latency-streaming-for-real-time-feature/): Databricks - [Prada Group Serves High-Quality Business Metrics with Lakebase](https://effectsize.io/cases/prada-group-serves-high-quality-business-metrics-with/): Prada Group - [Semantic Search for AI Agents at Scale: Retrieval and Ranking for LinkedIn’s Hiring Assistant](https://effectsize.io/cases/semantic-search-for-ai-agents-at-scale-retrieval-and-ranking/): LinkedIn - [TargetRun: AI-Powered Trip Prediction for Delivering Timely Offers](https://effectsize.io/cases/targetrun-ai-powered-trip-prediction-for-delivering-timely/): Target - [AI Blender in Search: How We Stopped Worshipping Clicks and Moved Block Placement to LLMs](https://effectsize.io/cases/ai-blender-in-search-how-we-stopped-worshipping-clicks-and/): Yandex - [Uplift Models in Promotional Offers](https://effectsize.io/cases/uplift-models-in-promotional-offers/): VTB - [A Reranker in Retail: Turning Advertising into Recommendations](https://effectsize.io/cases/a-reranker-in-retail-turning-advertising-into/): X5 Tech - [How Ad Ranking Works at Avito](https://effectsize.io/cases/how-ad-ranking-works-at-avito/): Avito - [A Cluster-Based Approach to CTR Forecasting for Millions of Advertised Products](https://effectsize.io/cases/a-cluster-based-approach-to-ctr-forecasting-for-millions-of/): Wildberries & Russ - [A Hybrid Generative-Ranking Model Based on Semantic IDs in Yandex Music Recommendations](https://effectsize.io/cases/a-hybrid-generative-ranking-model-based-on-semantic-ids-in/): Yandex - [Neural Network Recommendations for Smart TV: Moving from Classical ML](https://effectsize.io/cases/neural-network-recommendations-for-smart-tv-moving-from/): Yandex - [How Transformers Increased Candidate Diversity on the Avito Home Page](https://effectsize.io/cases/how-transformers-increased-candidate-diversity-on-the-avito/): Avito - [Neural Ranking for Recommendations](https://effectsize.io/cases/neural-ranking-for-recommendations/): Avito - [How to Integrate AI into Products When They Are All Different](https://effectsize.io/cases/how-to-integrate-ai-into-products-when-they-are-all/): Kontur - [About Anti-Fraud and Cats: How to Fight Fraud, Not Just Detect It Better](https://effectsize.io/cases/about-anti-fraud-and-cats-how-to-fight-fraud-not-just-detect/): Yandex - [LLMs for Arbitrage in New-Build Real Estate](https://effectsize.io/cases/llms-for-arbitrage-in-new-build-real-estate/): Avito - [A Speech Platform in Production](https://effectsize.io/cases/a-speech-platform-in-production/): X5 Tech - [An Authorization Model in a Lakehouse Without Downtime](https://effectsize.io/cases/an-authorization-model-in-a-lakehouse-without-downtime/): X5 Tech - [ABsalute: A/B Testing Automation](https://effectsize.io/cases/absalute-a-b-testing-automation/): X5 Tech - [Real-Time Analytics on ClickHouse](https://effectsize.io/cases/real-time-analytics-on-clickhouse/): X5 Tech - [How we build ML pipelines for pathomorphology: data, annotation, agents](https://effectsize.io/cases/how-we-build-ml-pipelines-for-pathomorphology-data/): OneCell - [ML Ad Forecaster: Estimating Results Before Launch](https://effectsize.io/cases/ml-ad-forecaster-estimating-results-before-launch/): Avito - [How Developer Productivity Changed with AI4SDLC Tools](https://effectsize.io/cases/how-developer-productivity-changed-with-ai4sdlc-tools/): T-Bank - [Managing Oil and Gas Field Development with a Multi-Agent System and Reinforcement Learning](https://effectsize.io/cases/managing-oil-and-gas-field-development-with-a-multi-agent/): Gazprom Neft - [What If Your Page Layout Could Learn? Optimizing with Bandits](https://effectsize.io/cases/what-if-your-page-layout-could-learn-optimizing-with-bandits/): Target - [Balancing Speed and Accuracy in Digital Order Fulfillment: The Next-Gen Solution](https://effectsize.io/cases/balancing-speed-and-accuracy-in-digital-order-fulfillment/): Target - [How Khan Academy Is Building a Better AI Tutor: Our Most Recent Learnings](https://effectsize.io/cases/how-khan-academy-is-building-a-better-ai-tutor-our-most/): Khan Academy - [Better Experiments with LLM Evals — A funnel, not a fork](https://effectsize.io/cases/better-experiments-with-llm-evals-a-funnel-not-a-fork/): Spotify - [We Tested Marketing Incentives to AI Agents. Here's What Happened.](https://effectsize.io/cases/we-tested-marketing-incentives-to-ai-agents-here-s-what/): Ramp - [Measure Less to Learn More: Using Fewer, Higher-quality Metrics to Capture What Matters](https://effectsize.io/cases/measure-less-to-learn-more-using-fewer-higher-quality/): Discord - [Using LLMs to build content embeddings for search and recommendations](https://effectsize.io/cases/using-llms-to-build-content-embeddings-for-search-and/): DoorDash - [Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces](https://effectsize.io/cases/beyond-a-b-testing-using-surrogacy-and-region-splits-to/): Lyft - [Trusting the Untestable: Validation and Diagnostics for the Doubly Robust Models](https://effectsize.io/cases/trusting-the-untestable-validation-and-diagnostics-for-the/): Lyft - [Interleaving for Accelerated Testing](https://effectsize.io/cases/interleaving-for-accelerated-testing/): Expedia Group - [DashCLIP: Leveraging multimodal models for generating semantic embeddings](https://effectsize.io/cases/dashclip-leveraging-multimodal-models-for-generating/): DoorDash - [From print to digital: Making weekly flyers shoppable at Instacart through computer vision and LLMs](https://effectsize.io/cases/from-print-to-digital-making-weekly-flyers-shoppable-at/): Instacart - [DrP: Meta’s Root Cause Analysis Platform at Scale](https://effectsize.io/cases/drp-meta-s-root-cause-analysis-platform-at-scale/): Meta - [Smarter promotions with causal machine learning](https://effectsize.io/cases/smarter-promotions-with-causal-machine-learning/): DoorDash - [LLM-Powered Relevance Assessment for Pinterest Search](https://effectsize.io/cases/llm-powered-relevance-assessment-for-pinterest-search/): Pinterest - [Is Traditional Machine Learning Dead? Comparing LLMs and Gradient Boosting](https://effectsize.io/cases/is-traditional-machine-learning-dead-comparing-llms-and/): Gusto - [How Pinterest Built a Real‑Time Radar for Violative Content using AI](https://effectsize.io/cases/how-pinterest-built-a-real-time-radar-for-violative-content/): Pinterest - [When GenAI Meets Personalization: Powering DoorDash's next-generation homepage experience](https://effectsize.io/cases/when-genai-meets-personalization-powering-doordash-s-next/): DoorDash - [Reducing experiment duration with predicted control variates](https://effectsize.io/cases/reducing-experiment-duration-with-predicted-control-variates/): Etsy - [Building The Intent Engine: How Instacart is Revamping Query Understanding with LLMs](https://effectsize.io/cases/building-the-intent-engine-how-instacart-is-revamping-query/): Instacart - [Reducing Irrelevant Impressions of the Thematic Block in Geo Search for Navigation Scenarios](https://effectsize.io/cases/reducing-irrelevant-impressions-of-the-thematic-block-in-geo/): Yandex - [Pricing in Yandex Lavka](https://effectsize.io/cases/pricing-in-yandex-lavka/): Yandex Lavka - [SafeChat: DoorDash's AI-powered safety feature](https://effectsize.io/cases/safechat-doordash-s-ai-powered-safety-feature/): DoorDash - [Identify User Journeys at Pinterest](https://effectsize.io/cases/identify-user-journeys-at-pinterest/): Pinterest - [Unlocking Faster Insights with Experimenter-Defined Segmentations](https://effectsize.io/cases/unlocking-faster-insights-with-experimenter-defined/): Etsy - [My Starter Project on the Lyft Rider Data Science Team](https://effectsize.io/cases/my-starter-project-on-the-lyft-rider-data-science-team/): Lyft - [How we built it: Real-time analytics for Stripe Billing](https://effectsize.io/cases/how-we-built-it-real-time-analytics-for-stripe-billing/): Stripe - [Building an anomaly detection platform at DoorDash to catch fraud trends early](https://effectsize.io/cases/building-an-anomaly-detection-platform-at-doordash-to-catch/): DoorDash - [Extracting trending keywords from OpenChat messages](https://effectsize.io/cases/extracting-trending-keywords-from-openchat-messages/): LY Corporation - [Beyond Winning: Spotify's Experiments with Learning Framework](https://effectsize.io/cases/beyond-winning-spotify-s-experiments-with-learning-framework/): Spotify - [A scalable LLM approach to enhancing chatbot knowledge with user-generated content](https://effectsize.io/cases/a-scalable-llm-approach-to-enhancing-chatbot-knowledge-with/): DoorDash - [Khan Academy Improves State Test Scores: Results from New 3-Year Efficacy Study](https://effectsize.io/cases/khan-academy-improves-state-test-scores-results-from-new-3/): Khan Academy - [Automating Ad Generations with LLMs](https://effectsize.io/cases/automating-ad-generations-with-llms/): Thumbtack - [How Instacart Built a Modern Search Infrastructure on Postgres](https://effectsize.io/cases/how-instacart-built-a-modern-search-infrastructure-on/): Instacart - [A Neural Network User Interest Model: How We Improved Diversity and Relevance of Recommendations on the Avito Home Page](https://effectsize.io/cases/a-neural-network-user-interest-model-how-we-improved/): Avito - [How the Effect Forecast for Paid Listing Promotion at Avito Works: Algorithms and Metrics Under the Hood](https://effectsize.io/cases/how-the-effect-forecast-for-paid-listing-promotion-at-avito/): Avito - [How LLM Automates Work at Avito](https://effectsize.io/cases/how-llm-automates-work-at-avito/): Avito - [Description Generation One Year Later. What We Managed to Understand Over the Year](https://effectsize.io/cases/description-generation-one-year-later-what-we-managed-to/): Avito - [Ranking VK Stories](https://effectsize.io/cases/ranking-vk-stories/): VK - [Graph Methods in Recommendations: How We Handle Large Interaction Graphs](https://effectsize.io/cases/graph-methods-in-recommendations-how-we-handle-large/): VK - [Scaling Recommender Systems](https://effectsize.io/cases/scaling-recommender-systems/): Yandex - [Experience with Transformer-Based Personalization at Avito](https://effectsize.io/cases/experience-with-transformer-based-personalization-at-avito/): Avito - [Product Applications of Yandex VLM](https://effectsize.io/cases/product-applications-of-yandex-vlm/): Yandex - [How We Grew Sales Team Efficiency with Customer Ranking](https://effectsize.io/cases/how-we-grew-sales-team-efficiency-with-customer-ranking/): Avito - [The Social Graph as an Ad Targeting Tool](https://effectsize.io/cases/the-social-graph-as-an-ad-targeting-tool/): Tele2 - [ML in AdTech. The Battle for the Click: How We Beat Historical CTR in Advertising](https://effectsize.io/cases/ml-in-adtech-the-battle-for-the-click-how-we-beat-historical/): Avito - [UpLift RecSys: A Universal Framework for Developing RecSys Aimed at Business Revenue Growth](https://effectsize.io/cases/uplift-recsys-a-universal-framework-for-developing-recsys/) - [Cooking the Treats Right: How ML Helps Run Promotions Efficiently](https://effectsize.io/cases/cooking-the-treats-right-how-ml-helps-run-promotions/): Raiffeisenbank - [Renting Out Smartly: Launching a DS Product in Short-Term Rentals](https://effectsize.io/cases/renting-out-smartly-launching-a-ds-product-in-short-term/): Avito - [How Large Language Models Help Users Formulate Queries in Yandex Maps](https://effectsize.io/cases/how-large-language-models-help-users-formulate-queries-in/): Yandex - [How to Find Students' Opinions and Not Drown in Spam?](https://effectsize.io/cases/how-to-find-students-opinions-and-not-drown-in-spam/): Tomsk State University - [Analytics, Data, Metrics and Motivation: How to Build a Data Labeling Process in a Large Bank](https://effectsize.io/cases/analytics-data-metrics-and-motivation-how-to-build-a-data/): Alfa-Bank - [Predicting Time to Sell Real Estate: Why You Can't Just Fit-Predict](https://effectsize.io/cases/predicting-time-to-sell-real-estate-why-you-can-t-just-fit/): Avito - [How One Product Helps Build Scoring for All of a Bank's Products (A DS Perspective)](https://effectsize.io/cases/how-one-product-helps-build-scoring-for-all-of-a-bank-s/): Ozon Bank - [High-Speed Multimodal Moderation: LLMs and VLMs in Real Production](https://effectsize.io/cases/high-speed-multimodal-moderation-llms-and-vlms-in-real/): Wildberries - [Moderation Platformization](https://effectsize.io/cases/moderation-platformization/): Avito - [Validating Propensity Score Matching: Metrics and Tests](https://effectsize.io/cases/validating-propensity-score-matching-metrics-and-tests/): Yandex Go - [How We Teach Alice to Activate on Intonation](https://effectsize.io/cases/how-we-teach-alice-to-activate-on-intonation/): Yandex - [Optimizing Regular Prices: Combining Experimental Approaches, ML and Economic Theory](https://effectsize.io/cases/optimizing-regular-prices-combining-experimental-approaches/): Yandex Lavka - [Price Recognition in Retail with PaddleOCR](https://effectsize.io/cases/price-recognition-in-retail-with-paddleocr/): Napoleon IT - [How Duolingo sped up data analysis by 68X with sampling](https://effectsize.io/cases/how-duolingo-sped-up-data-analysis-by-68x-with-sampling/): Duolingo - [How Pinterest Accelerates ML Feature Iterations via Effective Backfill](https://effectsize.io/cases/how-pinterest-accelerates-ml-feature-iterations-via/): Pinterest - [Streamlining RiskOps with the SOP agent framework](https://effectsize.io/cases/streamlining-riskops-with-the-sop-agent-framework/): Grab - [How we tested the conversion impact of global payment methods](https://effectsize.io/cases/how-we-tested-the-conversion-impact-of-global-payment/): Stripe - [Improving Pinterest Search Relevance Using Large Language Models](https://effectsize.io/cases/improving-pinterest-search-relevance-using-large-language/): Pinterest - [Leveraging Knowledge Graphs in Real Estate Search](https://effectsize.io/cases/leveraging-knowledge-graphs-in-real-estate-search/): Zillow - [Embedding-based retrieval for Airbnb search](https://effectsize.io/cases/embedding-based-retrieval-for-airbnb-search/): Airbnb - [How machine learning supercharged our revenue by millions of dollars](https://effectsize.io/cases/how-machine-learning-supercharged-our-revenue-by-millions-of/): Duolingo - [Single AI View of Customer: A retailers guide to Know Your Customer Better using Customer Embeddings and Customer Persona](https://effectsize.io/cases/single-ai-view-of-customer-a-retailers-guide-to-know-your/): Walmart - [Estimating Incremental Lift in Customer Value (Delta CV) using Synthetic Control](https://effectsize.io/cases/estimating-incremental-lift-in-customer-value-delta-cv-using/): PayPal - [How we aggregate 70 billion prices to show price context to our users](https://effectsize.io/cases/how-we-aggregate-70-billion-prices-to-show-price-context-to/): trivago - [The Quest to Understand Metric Movements](https://effectsize.io/cases/the-quest-to-understand-metric-movements/): Pinterest - [Search Query Understanding with LLMs: From Ideation to Production](https://effectsize.io/cases/search-query-understanding-with-llms-from-ideation-to/): Yelp - [Using Marketplace Marginal Values to Address Interference Bias](https://effectsize.io/cases/using-marketplace-marginal-values-to-address-interference/): Lyft - [Using ML to detect and respond to performance degradations in slices of Stripe payments](https://effectsize.io/cases/using-ml-to-detect-and-respond-to-performance-degradations/): Stripe - [Buyer Journey Insights with Data-Driven Attribution](https://effectsize.io/cases/buyer-journey-insights-with-data-driven-attribution/): LinkedIn - [Improving search ranking for Maps](https://effectsize.io/cases/improving-search-ranking-for-maps/): Airbnb - [The ML flywheel: How we continually improve our models to reduce card testing](https://effectsize.io/cases/the-ml-flywheel-how-we-continually-improve-our-models-to/): Stripe - [Next Best Action platform: democratizing personalization with contextual bandits](https://effectsize.io/cases/next-best-action-platform-democratizing-personalization-with/): Zillow - [Measuring Product Impact Without A/B Testing: How Discord Used the Synthetic Control Method for Voice Messages](https://effectsize.io/cases/measuring-product-impact-without-a-b-testing-how-discord/): Discord - [From Data to Insights: Segmenting Airbnb's Supply](https://effectsize.io/cases/from-data-to-insights-segmenting-airbnb-s-supply/): Airbnb - [Graphs in Anti-Fraud and How We Apply Them](https://effectsize.io/cases/graphs-in-anti-fraud-and-how-we-apply-them/): Yandex - [Part 1: DoorDash 2024 summer intern projects](https://effectsize.io/cases/part-1-doordash-2024-summer-intern-projects/): DoorDash - [5 product lessons we learned from building Friend Streak](https://effectsize.io/cases/5-product-lessons-we-learned-from-building-friend-streak/): Duolingo - [How DoorDash is pushing experimentation boundaries with interleaving designs](https://effectsize.io/cases/how-doordash-is-pushing-experimentation-boundaries-with/): DoorDash - [Improve Your Next Experiment by Learning Better Proxy Metrics From Past Experiments](https://effectsize.io/cases/improve-your-next-experiment-by-learning-better-proxy/): Netflix - [NEP: Notification System and Relevance](https://effectsize.io/cases/nep-notification-system-and-relevance/): Pinterest - [Contextual Offer Recommendations Engine at Target](https://effectsize.io/cases/contextual-offer-recommendations-engine-at-target/): Target - [Machine Learning in Content Moderation at Etsy](https://effectsize.io/cases/machine-learning-in-content-moderation-at-etsy/): Etsy - [Surprising findings from our analysis of 3DS transactions in the US](https://effectsize.io/cases/surprising-findings-from-our-analysis-of-3ds-transactions-in/): Stripe - [Criteo's Privacy Sandbox Market Testing](https://effectsize.io/cases/criteo-s-privacy-sandbox-market-testing/): Criteo - [Leverage graph technology for real-time Fraud Detection and Prevention](https://effectsize.io/cases/leverage-graph-technology-for-real-time-fraud-detection-and/): Booking.com - [How Canva collects 25 billion events per day](https://effectsize.io/cases/how-canva-collects-25-billion-events-per-day/): Canva - [Bandits for Marketing Optimization](https://effectsize.io/cases/bandits-for-marketing-optimization/): Instacart - [Unlocking 66% Faster Experiments with Covariate Adjustment: Power Analysis for Clustered Data, Ratio Metrics, and OLS Regression](https://effectsize.io/cases/unlocking-66-faster-experiments-with-covariate-adjustment/): Instacart - [Behind the metric: how we developed “Time Spent Learning Well”](https://effectsize.io/cases/behind-the-metric-how-we-developed-time-spent-learning-well/): Duolingo - [Socratic Dialogues with LLMs in Problem Solving](https://effectsize.io/cases/socratic-dialogues-with-llms-in-problem-solving/): Yandex Uchebnik - [Auto-Generating Product Descriptions in Fashion](https://effectsize.io/cases/auto-generating-product-descriptions-in-fashion/): Avito - [Advanced Use of GigaChat at Rabota.ru: Synthetic Data and Search](https://effectsize.io/cases/advanced-use-of-gigachat-at-rabota-ru-synthetic-data-and/): Rabota.ru - [Traffic Filtering in Programmatic Ads](https://effectsize.io/cases/traffic-filtering-in-programmatic-ads/): Teads - [Transforming Recommendations](https://effectsize.io/cases/transforming-recommendations/): VK - [Reading Between the Lines: How to Use ML to Curate Personalized Collections Better Than Editors](https://effectsize.io/cases/reading-between-the-lines-how-to-use-ml-to-curate/): T-Bank - [Deep Learning Clinical Decision Support Systems Reduce Time and Improve Radiologists' Accuracy in Routine CT Interpretation](https://effectsize.io/cases/deep-learning-clinical-decision-support-systems-reduce-time/): AUMI.AI - [LLMs at Avito](https://effectsize.io/cases/llms-at-avito/): Avito - [Ranking or Calibration? How to Measure the Quality of Advertising CTR Models](https://effectsize.io/cases/ranking-or-calibration-how-to-measure-the-quality-of/): Avito - [How We Cut Forecasting Time from Two Weeks to Five Minutes](https://effectsize.io/cases/how-we-cut-forecasting-time-from-two-weeks-to-five-minutes/): Avito - [Application of NLP for Customer Satisfaction Assessment and Optimization of SMS Traffic](https://effectsize.io/cases/application-of-nlp-for-customer-satisfaction-assessment-and/): VTB - [Monitoring Machine Learning Models](https://effectsize.io/cases/monitoring-machine-learning-models/): oneFactor - [Flotation Monitoring: Rapid Optimization Support](https://effectsize.io/cases/flotation-monitoring-rapid-optimization-support/): Nornickel - [How ML Product Development for Manufacturing Works at EuroChem](https://effectsize.io/cases/how-ml-product-development-for-manufacturing-works-at/): EuroChem - [Data Strategy in a Large Group of Companies](https://effectsize.io/cases/data-strategy-in-a-large-group-of-companies/): MTS - [How We Launched a New Promotion Model: Matching and Regional Tests](https://effectsize.io/cases/how-we-launched-a-new-promotion-model-matching-and-regional/): Avito - [Generative and Contrastive Learning for Event Sequences: Which Is Better and Can They Be Combined?](https://effectsize.io/cases/generative-and-contrastive-learning-for-event-sequences/): Skoltech - [Bayesian Truth Serum in Dataset Labeling: Finding the Truth When There Is None](https://effectsize.io/cases/bayesian-truth-serum-in-dataset-labeling-finding-the-truth/): ISS Art - [Calculation of spread for deposits with optionality. A synergy of statistical and financial-mathematical approaches](https://effectsize.io/cases/calculation-of-spread-for-deposits-with-optionality-a/): VTB - [Economics of the Data Office](https://effectsize.io/cases/economics-of-the-data-office/): Gazprom ID - [Matching LinkedIn members with the right Premium products](https://effectsize.io/cases/matching-linkedin-members-with-the-right-premium-products/): LinkedIn - [Sharpening the Blur: Removing dilution to maximize experiment power](https://effectsize.io/cases/sharpening-the-blur-removing-dilution-to-maximize-experiment/): DoorDash - [Measuring Marketing Effectiveness in a Cookie-Less World](https://effectsize.io/cases/measuring-marketing-effectiveness-in-a-cookie-less-world/): Grammarly - [Airbnb Brandometer: Powering brand perception measurement on social media data with AI](https://effectsize.io/cases/airbnb-brandometer-powering-brand-perception-measurement-on/): Airbnb - [How to Determine Causal Effects when A/B Tests are Infeasible through Adopter Analysis](https://effectsize.io/cases/how-to-determine-causal-effects-when-a-b-tests-are/): Walmart - [Using AI to Understand the Complexities and Pitfalls of Real Estate Data](https://effectsize.io/cases/using-ai-to-understand-the-complexities-and-pitfalls-of-real/): Zillow - [Optimizing at the Edge: Using Regression Discontinuity Designs to Power Decision-Making](https://effectsize.io/cases/optimizing-at-the-edge-using-regression-discontinuity/): Instacart - [Shepherd: How Stripe adapted Chronon to scale ML feature development](https://effectsize.io/cases/shepherd-how-stripe-adapted-chronon-to-scale-ml-feature/): Stripe - [How we built Text-to-SQL at Pinterest](https://effectsize.io/cases/how-we-built-text-to-sql-at-pinterest/): Pinterest - [Sequential A/B Testing Keeps the World Streaming Netflix. Part 1: Continuous Data](https://effectsize.io/cases/sequential-a-b-testing-keeps-the-world-streaming-netflix-2/): Netflix - [Sequential A/B Testing Keeps the World Streaming Netflix Part 2: Counting Processes](https://effectsize.io/cases/sequential-a-b-testing-keeps-the-world-streaming-netflix/): Netflix - [Real-time Fraud Detection with Yoda and ClickHouse](https://effectsize.io/cases/real-time-fraud-detection-with-yoda-and-clickhouse/): Instacart - [Yelp’s AI pipeline for inappropriate language detection in reviews](https://effectsize.io/cases/yelp-s-ai-pipeline-for-inappropriate-language-detection-in/): Yelp - [How the New York Times Games Data Team Revamped Its Reporting](https://effectsize.io/cases/how-the-new-york-times-games-data-team-revamped-its/): The New York Times - [Experiment Faster and with Less Effort](https://effectsize.io/cases/experiment-faster-and-with-less-effort/): DoorDash - [Privacy preserving single post analytics](https://effectsize.io/cases/privacy-preserving-single-post-analytics/): LinkedIn - [Leveraging Flink to Detect User Sessions and Engage DoorDash Consumers with Real-Time Notifications](https://effectsize.io/cases/leveraging-flink-to-detect-user-sessions-and-engage-doordash/): DoorDash - [Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Sampling](https://effectsize.io/cases/let-ai-entertain-you-increasing-user-engagement-with/): Nextdoor - [Exclude from Your Taste Profile](https://effectsize.io/cases/exclude-from-your-taste-profile/): Spotify - [End-to-end Keyword Bidding for Apple Search Ads](https://effectsize.io/cases/end-to-end-keyword-bidding-for-apple-search-ads/): Canva - [Why Khan Academy will be using \"skills to proficient\" to measure learning outcomes (and you should too!)](https://effectsize.io/cases/why-khan-academy-will-be-using-skills-to-proficient-to/): Khan Academy - [Stepping up marketing for advertisers: Scalable lookalike audience](https://effectsize.io/cases/stepping-up-marketing-for-advertisers-scalable-lookalike/): Grab - [Accelerating Ranking Experimentation at Thumbtack with Interleaving](https://effectsize.io/cases/accelerating-ranking-experimentation-at-thumbtack-with/): Thumbtack - [Leveraging Switchbacks: Enhancing Market-Based Testing at Thumbtack](https://effectsize.io/cases/leveraging-switchbacks-enhancing-market-based-testing-at/): Thumbtack - [How Instacart Modernized the Prediction of Real Time Availability for Hundreds of Millions of Items While Saving Costs](https://effectsize.io/cases/how-instacart-modernized-the-prediction-of-real-time/): Instacart - [Leveraging Real-Time User Actions to Personalize Etsy Ads](https://effectsize.io/cases/leveraging-real-time-user-actions-to-personalize-etsy-ads/): Etsy - [Using Contextual Bandit models in large action spaces at Instacart](https://effectsize.io/cases/using-contextual-bandit-models-in-large-action-spaces-at/): Instacart - [Improving Item Recommendation Accuracy Using Collaborative Filtering and Vector Search Engine](https://effectsize.io/cases/improving-item-recommendation-accuracy-using-collaborative/): Mercari - [Experimenting with Machine Learning to Target In-App Messaging](https://effectsize.io/cases/experimenting-with-machine-learning-to-target-in-app/): Spotify - [NLP at Alfa-Bank, Part 4: The Development History of a Chatbot](https://effectsize.io/cases/nlp-at-alfa-bank-part-4-the-development-history-of-a-chatbot/): Alfa-Bank - [A Job Posting Summary Generation Service Based on an LM Fine-Tuned with RLHF](https://effectsize.io/cases/a-job-posting-summary-generation-service-based-on-an-lm-fine/): Rabota.ru - [Phone Number Scanner](https://effectsize.io/cases/phone-number-scanner/) - [Clean Labels for ML](https://effectsize.io/cases/clean-labels-for-ml/): Yandex - [NLP at Alfa-Bank, Part 7: Analyzing Customer Feedback](https://effectsize.io/cases/nlp-at-alfa-bank-part-7-analyzing-customer-feedback/): Alfa-Bank - [Field Crowdsourcing: What It Is and How to Use It](https://effectsize.io/cases/field-crowdsourcing-what-it-is-and-how-to-use-it/): Yandex - [Learning annotator's style in medical imaging](https://effectsize.io/cases/learning-annotator-s-style-in-medical-imaging/) - [Metrics and Loss Functions in Machine Learning System Design](https://effectsize.io/cases/metrics-and-loss-functions-in-machine-learning-system-design/) - [Warden: Real Time Anomaly Detection at Pinterest](https://effectsize.io/cases/warden-real-time-anomaly-detection-at-pinterest/): Pinterest - [An ML based approach to proactive advertiser churn prevention](https://effectsize.io/cases/an-ml-based-approach-to-proactive-advertiser-churn/): Pinterest - [Building a large scale unsupervised model anomaly detection system — Part 2](https://effectsize.io/cases/building-a-large-scale-unsupervised-model-anomaly-detection/): Lyft - [Understanding a diverse user base with frequency segmentation at scale](https://effectsize.io/cases/understanding-a-diverse-user-base-with-frequency/): Canva - [Market Segmentation for Geo-Testing at Scale](https://effectsize.io/cases/market-segmentation-for-geo-testing-at-scale/): Expedia Group - [Quantifying Efficiency in Ridesharing Marketplaces](https://effectsize.io/cases/quantifying-efficiency-in-ridesharing-marketplaces/): Lyft - [D3: An Automated System to Detect Data Drifts](https://effectsize.io/cases/d3-an-automated-system-to-detect-data-drifts/): Uber - [Meaningful metrics: How data sharpened the focus of product teams](https://effectsize.io/cases/meaningful-metrics-how-data-sharpened-the-focus-of-product/): Duolingo - [How Zillow Data Science Measures Business Outcomes with Bayesian Statistics](https://effectsize.io/cases/how-zillow-data-science-measures-business-outcomes-with/): Zillow - [Improving the customer’s experience via ML-driven payment routing](https://effectsize.io/cases/improving-the-customer-s-experience-via-ml-driven-payment/): LinkedIn - [Personalized snippet suggestions without storing user text (Grammarly)](https://effectsize.io/cases/personalized-snippet-suggestions-without-storing-user-text/): Grammarly - [Causal Machine Learning for Creative Insights](https://effectsize.io/cases/causal-machine-learning-for-creative-insights/): Netflix - [Catching Anomalies Early in Mobile App Releases](https://effectsize.io/cases/catching-anomalies-early-in-mobile-app-releases/): Nextdoor - [How DoorDash Upgraded a Heuristic with ML to Save Thousands of Canceled Orders](https://effectsize.io/cases/how-doordash-upgraded-a-heuristic-with-ml-to-save-thousands/): DoorDash - [Selecting the Best Image for Each Merchant Using Exploration and Machine Learning](https://effectsize.io/cases/selecting-the-best-image-for-each-merchant-using-exploration/): DoorDash - [Ocelot: Scaling observational causal inference at LinkedIn](https://effectsize.io/cases/ocelot-scaling-observational-causal-inference-at-linkedin/): LinkedIn - [Understanding the collective impact of experiments](https://effectsize.io/cases/understanding-the-collective-impact-of-experiments/): Etsy - [Marketing Attribution: Evaluating The Path to Purchase in the Product Ecosystem](https://effectsize.io/cases/marketing-attribution-evaluating-the-path-to-purchase-in-the/): trivago - [Improving Instagram notification management with machine learning and causal inference](https://effectsize.io/cases/improving-instagram-notification-management-with-machine/): Meta - [Beyond A/B test : Speeding up Airbnb Search Ranking Experimentation through Interleaving](https://effectsize.io/cases/beyond-a-b-test-speeding-up-airbnb-search-ranking/): Airbnb - [Challenges and practical lessons from building a deep-learning-based ads CTR prediction model](https://effectsize.io/cases/challenges-and-practical-lessons-from-building-a-deep/): LinkedIn - [How The New York Times Uses Machine Learning To Make Its Paywall Smarter](https://effectsize.io/cases/how-the-new-york-times-uses-machine-learning-to-make-its/): The New York Times - [Faster ML Experimentation at Etsy with Interleaving](https://effectsize.io/cases/faster-ml-experimentation-at-etsy-with-interleaving/): Etsy - [Feature Investigation: Automatically Detect Drift in your Data Over Time](https://effectsize.io/cases/feature-investigation-automatically-detect-drift-in-your/): Feedzai - [Measuring downstream impact on social networks by using an attribution framework](https://effectsize.io/cases/measuring-downstream-impact-on-social-networks-by-using-an/): LinkedIn - [Measuring marketing incremental impacts beyond last click attribution](https://effectsize.io/cases/measuring-marketing-incremental-impacts-beyond-last-click/): LinkedIn - [T-LEAF: Taxonomy Learning and EvaluAtion Framework](https://effectsize.io/cases/t-leaf-taxonomy-learning-and-evaluation-framework/): Airbnb - [How Bandit Testing Improves A/B Tests at Wealthfront](https://effectsize.io/cases/how-bandit-testing-improves-a-b-tests-at-wealthfront/): Wealthfront - [From look-alike to uplift in customer lifecycle models](https://effectsize.io/cases/from-look-alike-to-uplift-in-customer-lifecycle-models/): VTB - [Selecting and monitoring physician annotators](https://effectsize.io/cases/selecting-and-monitoring-physician-annotators/): Celsus - [Transformers for Generalizing User Behavior in Yandex.Taxi](https://effectsize.io/cases/transformers-for-generalizing-user-behavior-in-yandex-taxi/): Yandex Go - [DL in finance: key results, production and new challenges](https://effectsize.io/cases/dl-in-finance-key-results-production-and-new-challenges/): Alfa-Bank - [Neural Network Models on Card Transactions](https://effectsize.io/cases/neural-network-models-on-card-transactions/): Alfa-Bank - [Image Quality Assessment for Magnetic Resonance Imaging](https://effectsize.io/cases/image-quality-assessment-for-magnetic-resonance-imaging/): Philips Research - [Matching bank and telecom customers: 3rd place at Data Fusion Contest 2022](https://effectsize.io/cases/matching-bank-and-telecom-customers-3rd-place-at-data-fusion/): VTB - [Review of solutions from the VTB DS team: matching transaction and clickstream sequences](https://effectsize.io/cases/review-of-solutions-from-the-vtb-ds-team-matching/): VTB - [Clustering subscriber messages](https://effectsize.io/cases/clustering-subscriber-messages/): MegaFon - [Study of the effectiveness of various clustering methods for analyzing the structure of gamma-hadron families](https://effectsize.io/cases/study-of-the-effectiveness-of-various-clustering-methods-for/): Lebedev Physical Institute - [Causal Inference. Advanced modeling methods](https://effectsize.io/cases/causal-inference-advanced-modeling-methods/): Glowbyte - [Full-Spectrum ML Model Monitoring at Lyft](https://effectsize.io/cases/full-spectrum-ml-model-monitoring-at-lyft/): Lyft - [How a small design change uplifted the goshop experience](https://effectsize.io/cases/how-a-small-design-change-uplifted-the-goshop-experience/): Gojek - [Suspecting the Unsuspected. Extracting and Analyzing Log Anomalies](https://effectsize.io/cases/suspecting-the-unsuspected-extracting-and-analyzing-log/): Mercari - [Using predictive technology to foster constructive conversations](https://effectsize.io/cases/using-predictive-technology-to-foster-constructive/): Nextdoor - [SHAP Plots: The Crystal Ball for UI Test Ideas](https://effectsize.io/cases/shap-plots-the-crystal-ball-for-ui-test-ideas/): Indeed - [The journey to build an explainable AI-driven recommendation system to help scale sales efficiency across LinkedIn](https://effectsize.io/cases/the-journey-to-build-an-explainable-ai-driven-recommendation/): LinkedIn - [Representative Sampling: A Statistical Method to Derive Data Insights from Noisy Data Distributions](https://effectsize.io/cases/representative-sampling-a-statistical-method-to-derive-data/): Walmart - [Using Machine Learning to Understand How Branding in Photos Affects the Car Shopping Experience](https://effectsize.io/cases/using-machine-learning-to-understand-how-branding-in-photos/): TrueCar - [Artificial Counterfactual Estimation (ACE): Machine Learning-Based Causal Inference at Airbnb](https://effectsize.io/cases/artificial-counterfactual-estimation-ace-machine-learning/): Airbnb - [The habit-building research behind your Duolingo streak](https://effectsize.io/cases/the-habit-building-research-behind-your-duolingo-streak/): Duolingo - [Experiment without the wait: Speeding up the iteration cycle with Offline Replay Experimentation](https://effectsize.io/cases/experiment-without-the-wait-speeding-up-the-iteration-cycle/): Pinterest - [How we adjust in-app messaging for learners in different languages](https://effectsize.io/cases/how-we-adjust-in-app-messaging-for-learners-in-different/): Duolingo - [Building for Balance](https://effectsize.io/cases/building-for-balance/): Instacart ## Optional - [All cases](https://effectsize.io/cases/) - [Tech blogs directory](https://effectsize.io/blogs/) - [AI and crawler policy](https://effectsize.io/ai/) - [Methodology](https://effectsize.io/about/methodology/)