Agentic Systems at Scale with Vilva Athiban

🎧 New Señors @ Scale Episode

This week, I spoke with Vilva Athiban, Lead AI Engineer at Omio, one of Europe's leading travel platforms, where he architects and ships agentic AI systems in production, including Omio AI and the MCP integrations that connect AI agents to the company's systems. We met at a JavaScript conference in October 2023, back when he was a senior JavaScript developer on the talk circuit. Eight months later his company put him on a two-person AI team reporting to the CEO and CTO, and he has spent the two years since relearning his job in production.

In this episode, we dig into why supervisor agents failed his CEO demos and what took their place, how Omio AI went from LangChain to LangGraph to no framework at all, the cost optimizations he found in the bills, what MCP replaced, why he expects loop engineering to die within months, and whether fine tuning is finally worth the money.

⚙️ Main Takeaways

1. AI arrived at his desk before he went looking for it

Omio picked two people in June 2024 and had them report straight to the CEO and CTO.

• The timing: "Around June 2024, my company wanted to start a small AI team because AI was getting bigger. And they directly picked me and another person, two people. And we started working directly with the CEO and CTO."
• The starting point: "When I started, I was blank. All I knew was how to make API calls to LLMs." Everything after that came from handling production from day one.
• Staying in TypeScript: He tried Mastra and LangChain in TypeScript, then checked whether the Python frameworks did anything different. "Theoretically you don't have to switch to Python for agentic AI. All the good things are in TypeScript as well."
• The first project: An ad campaign workflow for marketing that cut the turnaround "from six to eight weeks to two to three hours." He calls it technically simple and complicated on the product side.

2. LangChain is React for LLM calls, with LangGraph routing between agents

Both got explained from scratch, with React as the comparison.

• What agentic AI is: "Calling LLMs and giving them access to certain APIs or tools or files or database or resources. So they decide what to do." They read the query, decide what is needed, fetch it, and use it to produce the output.
• The React analogy: You can build a site in plain JavaScript and React gives you comfort over it. "LangChain is similar to what React is to JavaScript," a wrapper over plain LLM API calls that talks to the different providers and models.
• Where LangGraph comes in: Production control. It's a graph that defines the pathway and the conditional flow once a query arrives.
• The split in one line: "LangGraph is for routing into the right graph and right agent, and LangChain is the execution of the agent itself."

3. Asking for intents fixed the demos that supervisor routing kept failing

Most of the industry ran the supervisor pattern last year, until failed CEO demos pushed him off it.

• The supervisor model: A supervisor classifies the query and hands it to a predefined sub agent. If the user asks for addition and the supervisor picks subtraction, the subtraction agent subtracts. "It doesn't care about the user's query."
• The demo problem: "That was the most pain in the ass thing I had last year." His numbers: "90% of the time it works, and the 10% of the time when I demo it to my CEO, it doesn't."
• Why it breaks: "When I ask the LLM to binary decide one of the things based on how the input query is formed, it messes it up."
• The fix: Ask for intent instead of a decision. For "the multiplication of 2 and 3 plus 2", a supervisor can't choose between two agents. An orchestrator returns multiplication true, addition true, and sends the parameters to both. LangGraph then loops until the result is right.
• The 2026 caveat: "This was the world where LLMs were not smart enough. Today they are far better, where most of the use cases would suffice with a supervisor agent."
• Stacking them: You can put an orchestrator in front of a supervisor if you want. "The more steps you have, it would increase the latency and the token usage."

4. From controlled workflows to no framework at all

Omio AI's backend ended up on MCP alone, after the team dropped LangChain and then LangGraph.

• The first bet: LangChain, then LangGraph for control, because last year an LLM "either gets it right or it gets it wrong." "We wanted full control, so we couldn't go 100% autonomous. We wanted 100% accuracy." So they built controlled workflows over autonomous agents.
• The bill: "The controlled workflows always had an issue with latency and cost, mainly with the token usage."
• Where it landed: "Right now the application doesn't use any framework. It doesn't use LangChain, doesn't use LangGraph, Mastra, none of the fancy agentic stuff." The current backend is powered by MCP. "MCP is the only thing that drives the complete application."
• The meta app: He built the ChatGPT apps model inside Omio AI, so in theory any ChatGPT app can run there too. External MCP apps plug in as tools plus data, which he describes as UI plus data.
• He thinks they are alone in this: "I think we are the only person who have done this architecture, which is currently working."
• How agents are wired: Early on, each agent was a file in the same repo. The team created it with LangChain's create agent function and passed it to the graph. Once other teams need to plug in, each one becomes an MCP server, living in the same repo or behind a single URL on a third party.

5. The bigger model with medium thinking cost less than the mini

He found this one by reading the invoice, and it's the one I'm taking back to my own team.

• The assumption: "Usually we think mini models are cheaper. However, bigger models with medium thinking were cheaper to us."
• Why: The mini's per-token price is lower, but it generates far more tokens. With the bigger model and thinking on, "the amount of tokens are crisp and limited, and it directly gives the output better."
• His read on the pricing: "That's where these AI companies make profit. They are hiding behind the wall."
• The result at scale: Switching from the mini to the bigger model brought the cost down. "Theoretically it is observed as a different thing, but a bigger model with medium thinking brought our cost down."
• The wider point: "These are the small nuances which differentiate people who go through tutorials and who work in production."

6. LLMs are built for token maxing, so cap the tool calls

He designs the agent around a model that wants to spend tokens.

• The diagnosis: "LLMs are made for token maxing, because that's how they make money. They are not made for efficiency."
• What that looks like: An agent does its job, and instead of stopping it pulls the full output into context and returns extra inferences on top. Left autonomous, "it could run like 30, 40 tools sometimes based on how it perceives the output or the user input."
• Fix one: A hard cap on the number of tool calls allowed per agent. "If you don't limit it, it keeps looping sometimes."
• Fix two: For agents whose raw output should never flow back into the loop, replace the tool response with a smaller summary of the results.
• Fix three: Prompt caching, as much of it as possible. "In a travel product, you would expect hardly 20 or 30 types of queries to be predominant," so when the semantics are similar the cache keeps bringing the cost down.

7. Whale hunting showed him guardrails are subjective

An argument inside his own team changed who decides what the agent refuses.

• The architecture: Guardrails sit on top of the application. The top layer rejects anything out of bounds before the agent calls a single tool.
• The bug report: Someone asked the agent whether they could go whale hunting. His team in Singapore filed it as a bug, on the grounds that whale hunting is violent and should be blocked. He had seen whale hunting in Iceland and the Faroe Islands, where it is legal and common, and couldn't see the problem.
• The conclusion: "Guardrails can be subjective. Things which look okay to me might not be good for everybody else."
• Who decides now: The legal team sets the list and the guardrail layer enforces it.
• The layers: They don't own the model, so the frontier models deny most things themselves. On top of that sits a list of topics the agent won't touch, politics included.

8. The agent is as powerful as the tools you give it

I asked what stops a user telling the agent to delete every train from Berlin to Munich.

• The rule: "The agent is as powerful as the tools you give access to." Omio gives the agent no delete tools.
• The posture: "We are stingy about what an agent can access in our application. If you let it access the database, anybody can override it with being smart."
• The 1% number: "Guardrails are good 99% of the time, but there is 1% of the time where people can be extremely smart and bypass the LLM."
• The design for that 1%: "Even if the LLM is bypassed, all they could do is read their particular data or a general LLM response. Nothing else is accessible for a user."

9. MCP is to agents what REST is to APIs

He explained MCP for anyone who woke up from a coma this year, then argued the gains have moved from models to context.

• The definition: A standard, playing the role for agents that REST plays for APIs. At runtime it understands the query and gives the right context to the model, through TypeScript and Python libraries.
• The saying he builds on: "Your model is as good as the context it gets."
• Why everyone is doing context engineering in 2026: "They are out of options to make the models themselves better. The models are already at a good stage, and it's already costing them a lot." Results are stagnant, so context is where the gains are left.
• Harness versus runtime: Harness engineering gives the right context to local coding agents, while MCP handles that job at runtime inside a product.

10. Compaction beats summarizing, and a smaller window helps accuracy

He hasn't built a harness, but he read the leaked Claude Code source closely enough to write three posts on it. Then we compared context rules.

• The posts: One on how it uses React to render the terminal UI, which went viral, and one on how it manages memory.
• What a harness needs: Web search, local file system access, and the ability to run bash commands, plus memory management, which he calls "one of the most trickiest things, because different harnesses handle memory differently."
• Summary versus compaction: Some harnesses summarize the conversation and lose information. The summary itself can get large too. "Claude Code uses something called compaction. It gets the whole context, compacts it in a way that it doesn't lose any critical information."
• The cap detail: Compaction doesn't wait for the 1 million limit. "When it reaches a certain amount, it starts already compacting," so the harness always keeps budget for the compacted area and leaves space for the current conversation.
• Our shared rule: At my company we cap a 1 million window at 200K to avoid context rot. His take goes further: "Keeping it even lower would help the accuracy even better." The model does fine on the past four or five turns, and that reduces hallucination.
• What counts as a harness: For him, local first. "If it runs in your computer, does your job based on what you tell with the files you give and actions you say, I see that as harness," which he keeps separate from an agentic system built to run a defined set of tasks.

11. Loop engineering is a sugar-coated while loop

He gives this trend a few months to live, because plan mode in Claude Code already does the job for him.

• The definition: "What is loop engineering? It's a sugar-coated while loop." You give it an end goal and tell it to keep going until it gets there.
• Objection one, control: "As a developer, I will lose control completely of the system. I wouldn't know what's happening." He calls it good for prototyping and demos and stays skeptical about production.
• Objection two, cost: "Loop engineering can be 10x or 20x more expensive than what you do normally." He half suspects the AI companies push it through their celebrity engineers for the token revenue.
• What he does instead: Plan mode in Claude Code. Plan step by step, execute, and "at least 90% of the plan is achieved. And based on how good you are, sometimes 100% of the plan is achieved." Going block by block produces better code than a loop writing crap and fixing itself.
• The prediction: "I would say it would die in two, three months. That's my personal guess."
• Where loops do belong: Agentic systems have always looped. Given multiple tools, the model can call another one if it's unhappy with an answer.

12. MCP took over the retrieval RAG was doing

When I asked whether RAG is dead, he started by arguing it was never a vector database question.

• The misunderstanding: "When RAG was out, everybody thought it should be a vector DB." People with structured databases migrated into vector stores so they could say they had RAG.
• His definition: "RAG is more of a concept. Retrieval doesn't have to be from a vector DB." Retrieve something, process it so the LLM generates better output.
• Why MCP wins: Context windows got bigger, so you query less. And "in RAG you decide, and when you decide, you can decide only for certain cases. But in MCP, the LLM decides what information it needs for this user query."
• The similarity search problem: Search for a short mustache and similarity search can hand you Hitler. "RAG is not smart. It can't understand things."
• His verdict: "MCP beats everything of it, because that has become the standard. I don't think MCP would be replaced by anything else."

13. In 2026 the model matters less than the harness around it

A year ago he mixed three or four providers in one pipeline. Now he says it barely matters which one you pick.

• The 2026 answer: "It doesn't matter much. Whatever you have access to, whatever is cost benefit, you can go with those models." The latest models from each provider are in balance with each other.
• The 2025 answer: "If there are 10 steps, I had to use three or four models to do these 10 steps." OpenAI models wrote nicer text and Claude was better at supervisor decisions. Groq (the inference company, "not the Twitter Grok") ran open source models at a fraction of the latency.
• What differentiates now: "It's the context window and the tool calling capability and the harness around it that makes the difference, not the model itself."
• Big versus small by task: "If a task requires thinking or an analytical process, you use a bigger model. If a task has all the context and you have to generate the text out of it, even a nano model is completely fine."
• The ladder: He ranks Anthropic's lineup from Opus down through Sonnet to Haiku. OpenAI's main model with high thinking matches Opus, and with low thinking it matches Sonnet. The mini is close to Haiku, while the nano sits below the mini. "It can't do thinking, but for generation it's so good."

14. Fine tuning was a trap for two years, and the window is about to open

He has fine tuned open source models for specific tasks, and each one was overtaken by a frontier release before it earned its cost back.

• The pattern: "When I have a decent fine-tuned model, the next model release already would be better than my fine-tuned model."
• The hidden cost: "Anybody who did it in the last couple of years, they might not tell you, but they would have cried inside."
• Why it changes: The models are saturated. "Now fine tuning makes a good investment. We don't have to switch models because the next model would be better than our fine-tuned model."
• His timeline: Fine tuning becomes worth it "in the next six months or in early 2027."

15. Coding agents are a celebrated auto-complete that still shrank teams

I asked whether developers are cooked. He rates the tools lower than the marketing does and still says teams got smaller.

• On the tools: "Even you use Fable, what they say, it's super awesome. I tried it and it does make mistakes." For a non-technical person who only cares about the output, he says it can work. "90% of the time it works, but 10% of the time it doesn't work, and they don't know what to do."
• His description: "It is a highly celebrated auto-complete for software developers. Usually the IDE auto-complete gives one or two words. This gives 10 or 15 files."
• The headcount math: "A team that required five people is comfortably running with one person taking care of two such teams right now."
• Why he still likes the job: "We can get rid of all the boring work and focus on the cool stuff and interesting stuff to solve."

16. The path from JavaScript into AI engineering has two tiers

He runs interviews, and that shapes what he tells engineers who want his job.

• Tier one, adding AI features: A frontend or backend engineer can start integrating tools and learn prompting on the job. "In one, two months, they would learn how to handle a prompt, how to make it accurate." No new language needed.
• Tier two, working on models: Deploying and fine tuning models, or distilling them. "Then you need Python. TypeScript alone wouldn't help. There is no argument there."
• Who needs it: Most use cases are solved by frontier models or open source models from a provider like OpenRouter plus minimal tooling. "Most people will not have the need to become an AI engineer. They will have a higher need to become a better software engineer, because the interviews are going to be tougher."
• The surprising advantage: Application developers, whether they write JavaScript or Go, find it easier to become AI engineers than data scientists do. "Most people I have met, mainly in machine learning and data science, they don't have a clue on how applications work. They only know how numbers work."
• Why that holds: "Right now we don't even need to build ML models from scratch for most of the use cases."

17. The token subsidy ends while per-developer budgets arrive

I asked whether this is a bubble. He expects prices to rise while companies start managing their usage.

• Why prices rise: "These companies can't subsidize tokens throughout their life. They need to answer some of their investors." The price graph was high, dropped so people would adopt, and is climbing again.
• Where he thinks companies go: Open source models, which he puts at 10 to 20 times cheaper than frontier models at almost the same accuracy. The blocker for many companies is that the good ones are Chinese. He's hoping for a non-Chinese open source model from Meta within six months.
• The abuse on the user side: "I know people who use LLMs even to do a commit." Two or three words of command that cost tokens when a terminal would do.
• What comes next: Budgeting tools. He mentioned pura.sh, a CodeRabbit-style tool with routing and budgeting, and expects the same optimization wave that hit Kubernetes and AWS. "I don't think that would be a point where they say this is more expensive than humans, so we stop using it. We crossed that line already. It's here to stay."

🧠 What I Learned

• A bigger model with medium thinking can cost less than the mini, because the mini generates far more tokens for the same job.
• Supervisor agents fail on binary routing decisions; asking for intents and sending the query to every agent that matches is what made Omio AI accurate last year.
• Capping the number of tool calls per agent is a cost control and a loop control at the same time.
• When an agent's raw output should stay out of the loop, replace the tool response with a summary.
• Prompt caching pays off in a product with 20 or 30 dominant query types.
• The legal team should own the guardrail list, because what counts as out of bounds differs by team and country.
• Without delete tools the agent can't delete anything, and a bypassed LLM should only reach the user's own data.
• Claude Code compacts the context before it hits the limit instead of summarizing after, and he would cap a 1 million window below the 200K we use.
• Loop engineering costs 10x to 20x more tokens than plan mode and takes away the developer's view of what happened.
• RAG was always retrieval, and in 2026 the retrieval is a tool call the model decides to make.
• Fine tuning lost to the next frontier release for two years; with models saturated, he expects a fine-tuned model to stay ahead long enough to matter.
• Application developers have an easier path into AI engineering than data scientists, because the job is building systems.

💬 Favorite Quotes

"I didn't make the switch. AI suddenly started taking over."
Vilva Athiban

"90% of the time it works, and the 10% of the time when I demo it to my CEO, it doesn't."
Vilva Athiban

"Usually we think mini models are cheaper. However, bigger models with medium thinking were cheaper to us."
Vilva Athiban

"LLMs are made for token maxing, because that's how they make money. They are not made for efficiency."
Vilva Athiban

"The agent is as powerful as the tools you give access to."
Vilva Athiban

"What is loop engineering? It's a sugar-coated while loop."
Vilva Athiban

"Anybody who did it in the last couple of years, they might not tell you, but they would have cried inside."
Vilva Athiban

"It is a highly celebrated auto-complete for software developers. Usually the IDE auto-complete gives one or two words. This gives 10 or 15 files."
Vilva Athiban

🎯 Also in this Episode

• How we met at a JavaScript conference in October 2023, and why this is his first community appearance in three years after going quiet on conferences and LinkedIn while he got up to speed
• His prediction that AI content from full-time influencers who never shipped a production system will lose its audience, while non-technical people who still get excited by a generated video are about to move in
• Why "our median knowledge of AI is always the median of knowledge of people around us," and what that hides from those of us working in tech
• The one learning resource he recommends, Anthropic's learning courses, because conceptually they hold up whatever API or model you use
• Why he will not recommend an AI book: "By the time somebody writes a book, it gets printed and it comes to publication, everything in that book is already obsolete"
• The book he did recommend, The Psychology of Money
• My own observation that the token maxing culture of six months ago has flipped into per-developer budgets at companies like Uber

Resources

More from Vilva Athiban:

• Vilva on LinkedIn: posting again after two quiet years
• His blog posts on the leaked Claude Code source: how it renders the terminal UI with React, and how it manages memory

Tools & concepts mentioned:

• Omio AI (omio.ai): the consumer-facing agent app whose backend runs entirely on MCP
• Model Context Protocol (MCP): the standard behind the whole architecture, with TypeScript and Python SDKs
• LangChain and LangGraph: the frameworks Omio AI started on and later dropped
• Mastra: the TypeScript agent framework he evaluated
• Claude Code: compaction, plan mode, and the leaked source he wrote three posts about
• Groq: the fast inference provider he used for open source models last year
• OpenRouter: his suggested route to open source models for most use cases
• pura.sh: a CodeRabbit-style tool with routing and token budgeting
• Anthropic's learning courses: the one resource he recommends

Books mentioned:

• The Psychology of Money by Morgan Housel: the one book he recommended, with the caveat that he came to it late

🎧 Listen Now

🎧 Spotify
📺 YouTube
🍏 Apple Podcasts

Episode Length: 52 minutes on running a production travel agent on MCP alone, supervisor versus orchestration agents, the cost optimizations that only show up at scale, subjective guardrails, compaction, loop engineering, and where RAG and fine tuning stand in 2026.

Whether you're shipping your first agent or staring at an LLM bill that doesn't match the price list, this one comes from someone who has been running it in production for two years.

Happy orchestrating,
Dan