60k tickets a month handled by AI

Leboncoin's customer service faces a massive volume of requests. The Data & AI team, already mature with 30 ML models in production, wanted to move up a gear by bringing in LLMs to automate handling of the most common tickets.

With Hymaia, leboncoin designed and deployed a system of specialized multi-agents, each trained on a precise business scope and able to resolve tickets end to end.

A system of 3 specialized agents

The architecture relies on a dispatcher that routes each ticket to the most relevant agent:

Generalist agent: handles common questions through a RAG connected to the Zendesk FAQ, with similarity search (FAISS) and LLM rephrasing

Buyer transactions agent: handles purchase-related requests, accessing Mondial Relay data and payment information

Seller transactions agent: handles seller-side cases with access to the same transactional data sources

When no agent can answer with a sufficient confidence level, the ticket is escalated to a human agent, along with the information already gathered, to speed up handling.

A robust architecture in production

The system runs on Claude (now on its 3rd generation) via AWS, orchestrated with LangChain, containerized on Kubernetes/Docker and integrated with Zendesk. Monitoring combines Datadog for infrastructure and Langfuse for tracking LLM chains.

Guardrails control and moderate the assistants' outputs to guarantee response quality. The pragmatic RAG/DAG approach favors maintainability: every component can be tested and replaced independently.

Measurable results

After the rollout of the 3 agents, shipping roughly one new agent to production every 3 months:

60,000 tickets handled per month with AI assistance

10% of tickets fully automated, with a mid-term target of 30%

30 ML models in production, 6 of them incorporating LLMs, run by a team of 10

Lower customer service costs and faster response times

A lean team that scales through good practices

The key to the setup: a team of about ten people able to run 30 models in production thanks to a well-designed architecture. Rapid prototyping makes it possible to test new model versions and adapt the system as technology evolves. Good GenAI architecture practices (monitoring, evaluation, hallucination management) are spread across the organization so every new AI project benefits from the lessons already learned.