HRBert2.0: Improving Vectorization of Vacancies and Resumes

Alexey Kozlov, Evgeny Taychinov1 Nov 2024

Problem

Rabota.ru is a job search service for mostly blue-collar professions. One vectorization model for vacancies and resumes (HRBert) feeds almost every recommender: resumes for a vacancy, resume-based recommendations on the main page (as candidate generator), item-to-item, push and skills. The old model took the title and coordinates, but simple filters by radius or region worked better than coordinates. It also made obvious errors, and a large history of user feedback was available to fix them. Labels were sparse too: a vacancy such as "Call-center operator" invites 10 candidates, while hundreds of resumes may truly fit.

Example from the article: the old model scored "Basketball coach" and "Dog handler" at 0.999 similarity and showed such pairs in recommendations.

Solution in brief

Retrain on titles only with triplet loss on a dataset mined from employer and job-seeker actions, old-model errors, synonym lists and user feedback. Expand sparse labels through similar vacancies, filtered by region. A second model adds skills, with missing ones filled by TF-IDF.

Questions this case answers
  • How to train an embedding model that matches the two sides of a matching problem from interaction logs, using positives, soft negatives and hard negatives
  • How to expand sparse relevance labels in a two-sided matching task by treating similar items as sharing their labels
  • How to retrain a job-resume matching embedding model from recommendation interaction logs
  • How to mine positives, soft negatives and hard negatives for a title-matching embedding model

Tools and methods

  • DistilBERT, triplet loss — the final title model: own DistilBERT, mean pooling, two Dropout + Linear layers.
  • Hard and soft negative mining — rejected responses and look-alike titles versus random pairs.
  • Label expansion — merge relevant resumes of vacancies that share resumes, then filter by region.
  • TF-IDF skill imputation — fills 1 to 3 skills from the title when skills are empty.
Continue reading — it's free

Google or a one-time email code. No password.