Lemana-GPT: Building a Secure Corporate Gateway to LLMs and Scaling AI Products in Retail

Dmitry Terentyev23 May 2026

Problem

Employees of a large retailer (40,000+ people) started using external LLM services on their own. Personal and commercial data flowed to third parties, answers to the same question differed from person to person, and teams without access fell behind on product ideas. The company needed one controlled entry point to LLMs on a limited budget.

Example from the article: contact-center operators kept 27 browser tabs open (returns policy, legislation, stock, product specs). An unaided answer on a return took about 400 seconds.

Solution in brief

Lemana Tech built Lemana-GPT, a single gateway for employees and product teams. Every request is authenticated, masked for personal data, routed by a small LLM cascade to a RAG service or an agent, and logged. A 27B Qwen model on the company's own GPU rack serves most cases, with a cloud provider as fallback.

Questions this case answers
  • How to build a single controlled gateway to LLMs that masks personal data before any request leaves the perimeter and routes requests to models of different size and cost
  • How to roll out an LLM assistant for operators and measure its effect in a pilot
  • How to build a corporate LLM gateway that keeps personal and commercial data inside the company perimeter
  • How to mask personal data (PII) before a request reaches an internal or external LLM

Tools and methods

  • PII masking and guardrails — pattern detection plus NER, two in-house BERT-like models (toxicity, personal data), pseudonymization before any LLM call
  • LLM routing — two fast 8B models with chain-of-thought choose agent, model and whether RAG is needed
  • RAG on Qdrant — internal regulations and knowledge bases
  • Function-calling agents and a SQL agent — REST access to search, stock and orders
  • Canary rollout — 15, then 30 employees
Continue reading — it's free

Google or a one-time email code. No password.