Problem
Real estate data carries the history of housing discrimination. Even without race in the data, common attributes work as proxies: price is a strong one because of the wealth gap between Black and White families. Agent-written listing descriptions are now passed to LLMs and embedding models to extract attributes or find similar homes. A system that uses such text can reproduce bias, which matters under Fair Housing rules. Checking text for correlation with a protected class is harder than checking a number such as price or zip code.
Example from the article: Zillow compared descriptions of active single-family listings in majority non-Hispanic white and majority Black census blocks and found differences in length, phrasing and themes.
Solution in brief
Label each document with the demographic group of its location, then compare the two text collections in three steps: token counts, the most distinctive trigrams by odds ratio, and LDA topics named by an LLM and compared by relative lift. Consistent differences mean the text can act as a proxy, so systems built on it need a fairness audit.
Questions this case answers
- How to test whether an unstructured text feature correlates with a protected attribute before it feeds downstream models
- How to compare two large text collections and name the themes that differ between them without manual reading
- How to check whether free text in listings or product pages acts as a proxy for a protected class?
- How to find which phrases and themes distinguish two groups of text documents?
Tools and methods
- Census block demographics — assign each listing to a group by location
- Token counts — quick first comparison
- Trigram odds ratio — phrases most typical of each group
- LDA — 15 topics, compared per group
- LLM topic labeling — name a topic from its top 20 words
- pyLDAvis — interactive topic view
Google or a one-time email code. No password.