Advancing Enterprise AI: How Wix is Democratizing RAG Evaluation

Dvir Cohen30 Jun 2025

Problem

Teams running a RAG assistant over a help center see wrong answers but cannot tell whether retrieval or generation is at fault. Public benchmarks are built from Wikipedia and have one source document per question, so they do not match support questions that need several articles. Evaluation tools return scores such as "Context Adherence: 0.4" with no cause and no fix. So every prompt or retrieval change is a guess.

Example from the article: WixQA-ExpertWritten holds real Wix customer queries such as configuring domain settings, troubleshooting SSL certificates or setting up e-commerce functionality, and many WixQA answers need information from several knowledge base articles.

Solution in brief

Test on real support questions, including ones that need several articles. Score every record with six LLM-judge metrics, each tied to one link of the pipeline, then let an LLM aggregate the scores into per-metric explanations and a short prioritized list of changes. Wix open-sourced the dataset (WixQA) and the framework (RAGXplain).

Questions this case answers
  • How to diagnose which stage of a multi-step retrieve-and-generate pipeline is failing when evaluation gives only aggregate scores
  • How to build a realistic evaluation set for a question-answering system whose answers need several source documents
  • How to find which stage of a RAG support assistant (retrieval or generation) causes wrong answers?
  • How to turn RAG metric scores into concrete changes to the prompt and retrieval settings?

Tools and methods

  • WixQA — three QA sets and a knowledge base snapshot of 6,221 help articles
  • RAGXplain — six-metric LLM-judge evaluation with generated explanations and recommendations
  • LLM-as-a-judge — scores each record per metric and explains the score
Continue reading — it's free

Google or a one-time email code. No password.