Problem
A polyclinic doctor has 12–15 minutes per patient. AIRI maintains a production model that reads two years of an electronic medical record and suggests the final ICD code. Cleaning the data is painful, retraining is expensive, the model knows only 363 codes and ignores seasonality such as a flu epidemic. Doctors also enter synonymous or neighbouring codes, so the targets are noisy. The team tested whether an LLM could skip the cleaning and cover more codes.
Example from the article: a visit text says "pain in the right elbow joint" with an established M19.8 (other osteoarthritis); the LLM with an advanced prompt invented a fall on the left hand and answered S63.5 (wrist sprain).
Solution in brief
The team compared prompts, LoRA adapters and RAG on GigaChat against the production Longformer on real visit data. None beat it: the best, LoRA, scored 0.478 hit@1 against 0.67. Noisy records confuse an LLM with clean medical knowledge, so the authors conclude that data cleaning has to come first.
Questions this case answers
- How to classify long, noisy records into one of several hundred codes when the labels are noisy and new codes and seasonal shifts appear
- How to choose between prompting, adapter tuning and retrieval for an LLM classifier that must cover more classes than it was trained on, when cleaning the data is costly
- How to predict the final diagnosis code from a noisy electronic medical record
- How to check whether an LLM can replace a text classifier trained on dirty production data
Tools and methods
- Longformer classifier (AIDA) — production baseline, 363 ICD codes, 30 million records
- Prompt engineering — role, ICD rules, Chain of Thought, Tree-of-Thought, XML tags
- LoRA adapters — trained on GigaChat 7b and BioLLaMA3
- RAG — similar visits by cosine distance, added to the prompt
- hit@1 and hit@3 accuracy — top-1 and top-3 match, also with synonymous codes
Google or a one-time email code. No password.
