Problem
A support team wants an LLM to answer customer questions from internal documentation, for example "How do I add a contractor?". Sometimes the model answers well. Other times it makes things up or gives a vague reply. Classic ML models return a class and a confidence, so a threshold can trade coverage against precision. LLM output is free text, so that control seems unavailable and bad answers reach customers.
Example from the article: Gusto ran 1000 support questions through its question-answering LLM service, and support experts labeled each answer as good or bad.
Solution in brief
Use the average log-probability of the generated tokens (Seq-Logprob) as the answer's confidence. It comes free with generation. Label a sample of answers, check that confidence tracks quality, and set a threshold. Answers below it are rejected, sent to an expert, or retried with more information.
Questions this case answers
- How to cheaply estimate whether a free-form generated answer is reliable, when there is no class label or native confidence score to threshold
- How to trade answer coverage against answer quality for a generative system using a threshold on a by-product signal
- How to stop low-quality LLM answers from reaching customers without a second model call?
- How to get a precision-recall trade-off for a free-text LLM feature?
Tools and methods
- Seq-Logprob (sequence log-probability) — mean log-probability of the generated tokens, used as the confidence score
- Precision-recall curve — shows the coverage versus quality trade-off for each confidence threshold
- Expert labeling — support experts mark answers good or bad to calibrate the threshold
- OpenAI API logprobs — source of the token log-probabilities
Google or a one-time email code. No password.