Problem
Rabota.ru wanted a short summary on each job vacancy, hoping it would raise conversion and attract job seekers. A summarizer trained on news and articles misses what matters in a vacancy. Employers write mostly about the company, so the model summarizes the company's history instead of duties and conditions. The first attempt skipped supervised fine-tuning (SFT) and went straight to a reward model. Annotators almost never found a good output among the generations and wrote their own summary every time. Writing references by hand is slow and expensive for a team of middle and senior engineers.
Example from the article: for "Sber is looking for a janitor", most of the text describes Sber, and the stock model returns a summary about Sber.
Solution in brief
Build the SFT dataset automatically. Run extractive summarization (TextRank, LSA) to pick three sentences per vacancy, and paraphrase the results that exceed the 200-character limit. Fine-tune on the result, then collect RLHF preferences from employers who choose between two generated variants.
Questions this case answers
- How to build a supervised fine-tuning dataset for a text-shortening generator without annotators by extracting sentences from the source under a length limit
- How to collect preference labels for a generative model from the product's own user choices when raters cannot pick the best of several generations
- How to build an SFT dataset for a domain-specific summarization model without annotators
- How to generate short job-posting summaries that fit a hard character limit
Tools and methods
- TextRank / LSA extractive summarizers — pick the top 3 sentences from a vacancy, no model training.
- Paraphrasing model (Hugging Face) — shortens 200–300 character extracts to under 200.
- BART / T5 — base summarization models for SFT.
- Reward model with in-product choice — employer picks the better of two variants.
Google or a one-time email code. No password.