Problem
Ya zdorov is a health app with a medical record. Users upload a file or a photo of a lab form, and the app must turn it into rows of four fields: biomarker name, value, unit and reference range. Forms come from many laboratories: some have no table structure, some are crooked phone photos. A biomarker has 10 to 20 synonyms, sometimes hundreds, and laboratories abbreviate names in their own way. An unmatched name becomes a duplicate biomarker, so history and charts break.
Example from the article: glucose appears as "Analiz krovi na glyukozu", "Sakhar krovi", "Fasting blood glucose", "FBG", "FBS" and more; ALT and AST are different biomarkers, while Antistreptolysin-O and ASO are the same one.
Solution in brief
Build a reference database of about 2,000 biomarkers with synonyms, units, conversion factors and per-patient-group reference ranges. Recognize forms with OCR plus custom code or with a local multimodal model. Then match new names to existing biomarkers with vector search. The team reports above 90% recognition accuracy, but name merging is still unsolved and is the launch blocker.
Questions this case answers
- How to extract structured records from scanned forms and photos with varying layouts and merge variant names of the same entity into one canonical entity
- How to choose between a custom OCR pipeline and a local multimodal model for document extraction when data must stay inside the company and compute is small
- How to extract structured lab results from scanned forms and photos with different layouts
- How to merge synonyms and abbreviations of biomarker names into one entity
Tools and methods
- Tesseract — OCR after preprocessing and block detection
- Qwen 7B — local multimodal model that returns the required structure from a prompt
- Vector search — matches a recognized name to the database of names and synonyms
- Embedding fine-tuning — planned fix for name matching, trained on collected names and synonyms
Google or a one-time email code. No password.