Problem
LINE operators review every new or edited OpenChat room name and description for violations of the operation policy. Many rooms are created or edited each day, so manual review is costly. Existing models already cut manual review in several countries. In countries with more granular judgment criteria, they could not review rooms automatically. The team set out to raise accuracy there and in countries that already automate.
Illustrative example: a new room name is submitted. The model must return "no issue", or a penalty level together with the reason, because operators cannot apply a penalty without stating it.
Solution in brief
Fine-tune a small safety-tuned decoder model (Granite Guardian 3.1 2B) with LoRA to emit a penalty code token and a reason token. Read the probabilities of those tokens as confidence and send a room to manual review when the top probability is below a threshold tuned for high precision. Before training, resolve conflicting labels with explicit rules.
Questions this case answers
- How to automate moderation of short user-generated text so that automatic approvals meet a strict precision requirement
- How to build a training set from manual review logs when reviewers gave conflicting verdicts on identical items
- How to automate moderation of user-created room names and descriptions without letting violations through?
- How to clean reviewer labels when the same text received different penalties?
Tools and methods
- Granite Guardian 3.1 2B — IBM's decoder-only safety model, used as the base for fine-tuning
- LoRA — parameter-efficient fine-tuning of the base model
- Candidate-token probabilities — classification by fixed answer tokens
- KV caching — reuses attention keys and values so the reason step is cheap
- Label cleansing rules — one label per identical text
- Confidence threshold — sends low-confidence cases to manual review
Google or a one-time email code. No password.
