Problem
A real-estate chat assistant built on a general LLM may answer queries that touch legally protected classes (race, familial status, disability, source of income and others). Answering them can count as discrimination or "steering" under the Fair Housing Act, so the assistant must refuse these requests. Refusing too much also hurts: it blocks legitimate needs, such as a search for accessible homes. For violations recall matters most.
Example from the article: the query "Homes in a quiet location without families with young kids in Seattle" gets a helpful answer from a state-of-the-art LLM, which would likely be familial-status discrimination.
Solution in brief
Zillow compared a prompt instruction, a stop list and a fine-tuned BERT classifier. Each fails differently, so the final guardrail combines all three: stop list and classifier run on the user input and on the LLM output, and the prompt carries a compliance instruction. A flagged text is replaced by a fixed refusal message.
Questions this case answers
- How to block policy-violating inputs and outputs of an LLM assistant when violations are rare, context-dependent and a missed violation is costly
- How to build a training set for a rare-class text classifier when no labeled data exists and legal experts must define the labels
- How to stop an LLM chat assistant from answering queries that break anti-discrimination rules?
- How to choose between a prompt, a stop list and a trained classifier for a compliance guardrail?
Tools and methods
- Prompt instruction — tells the LLM to politely refuse queries that reference protected classes
- Stop list — exact-match phrases for the worst violations
- BERT sequence classifier — fine-tuned binary compliant / non-compliant detector with a tunable threshold
- Weak labeling with few-shot LLM prompts — first-pass labels that experts then review
- Data augmentation — back-translation, paraphrasing, word-embedding swap, neighboring-character swap
Google or a one-time email code. No password.