Case Study
November 10, 2026 · 5 min read
Mason Pierce · Jake O’Rielly · Jacopo Cirica · Onur Guleryuz
At a glance
Measure
Result
Performance
92%, vs 82.1% for the best frontier model
Cost
about $15k, down from $60k on Sonnet
Model size
27B parameters
Reliability
1.29 million requests, 99.995% uptime
Speed
up to 178 tok/s per user
About Cartage
Cartage runs Wilson, an AI agent that moves freight for dozens of manufacturers. Wilson quotes loads, books carriers, chases follow-ups, and handles payments and communicates across email, Slack, Teams, and spreadsheets.
The challenge
In many AI applications, humans are usually there to protect against a model doing the wrong thing in production. In Cartage’s case, however, the sheer volume of shipments makes that infeasible for the work that Wilson does.
There are many common failure modes that the Cartage team has noticed even the smartest frontier models run into. Sometimes models will make up weights and counts that nobody gave them. For example, a buyer emailed Wilson for a quote on drums going from Memphis to Little Rock but forgot to include how many drums there were and how much they weighed. In this scenario, the model should have asked, but instead it guessed six drums weighing about 3,100 lbs total. In reality, if it had stopped to ask, it would have learned that the order was actually two drums weighing 980 lbs. An experienced coordinator would have asked for the missing numbers before quoting anything.
Frontier models also consistently mess up dates. For example, a human knows that “next business day” when asked on a Friday, means Monday. Frontier models will often book it for the weekend instead, even when it’s explicitly clear that the next day is a weekend.
These kinds of mistakes are expensive because in freight, every hour costs money. When a truck shows up at the wrong place or at the right place at the wrong time, the driver can miss the drop-off window which can cascade into missing the next load.
In addition to mistakes, running Wilson on Anthropic’s Claude Sonnet 5.5 was costing the Cartage team about $60k a month, and every customer they added made Wilson more expensive to run.
“When we were using Sonnet, our costs climbed dramatically as we onboarded more customers. We also had to be very deliberate with our cache controls, and any mistake would immediately raise costs.”
Jake O’Rielly, CTO of Cartage
The solution
Using the TrainLoop platform, the Cartage team was able to train Cartage-1, a state-of-the-art model in logistics.
Most of what makes a good freight coordinator is learned through experience, and thus lives in their head. They likely have an understanding of what is important when defining a load for a shipper and how that changes across customers. The TrainLoop platform is built around getting that knowledge out of people’s heads and into tests and training data.
The TrainLoop platform monitors how Wilson behaves in production and identifies places where it keeps getting things wrong. For each of those failure modes it writes a test to Cartage’s internal benchmark that recreates the situation. The Cartage ops team monitors and confirms tests before they are added to the benchmark.
From there, the platform generates synthetic curriculum tasks from those tests, so no customer data goes into training, and trains a new version. That new version is then released into the wild and the mistakes it makes become the training objective for the next iteration.
The first Cartage-1 went live for one Cartage customer in June 2026. Cartage moved the rest of its customers over in waves through August and finished on September 17, 2026.
“When we moved to Cartage-1, our costs came way down and got a lot less variable, because caching just worked. I was surprised at how easy it was to move over. We expected issues when we swapped the model into production customer environments, and instead it just worked out of the box.”
Jake O’Rielly, CTO of Cartage
The results
Today, every Cartage customer runs on Cartage-1. On Cartage’s internal benchmark, Cartage-1 scores 92%, which is higher than every frontier model we tested. It makes around half the mistakes GPT-6 Astra makes. Cutting mistakes in half matters a lot because a truck that shows up for a load at the wrong location or a shipment that is booked for the wrong day means that the Cartage team has to fix the issue by hand.
The base model for Cartage-1 scored 74.4% on Cartage’s internal benchmark. With training the TrainLoop platform was able to add 18 percentage points on top of the base model, 10 points above the most expensive frontier models.
It’s also a lot cheaper to run. Cartage-1 costs about $0.45 to $0.90 per million output tokens to serve, versus $2 per million input and $10 per million output for Sonnet 5.5, even with caching. Because Cartage-1 runs on its own GPUs at a fixed hourly cost, adding new customers increases the bill sub-linearly, and that’s what lets Cartage offer Wilson for free.
In production, the Cartage team went from spending about $60k a month on Sonnet to about $15k, a quarter of the price.
Since September 17, Cartage-1 has served at 99.995% uptime. Out of the 1.29M requests served, only 66 requests have failed.
“Since we moved to Cartage-1, the model has stopped being something my team worries about. It just works.”
Jake O’Rielly, CTO of Cartage
Wilson now uses about 3.3B tokens a day, and that number keeps growing as Cartage adds customers and workflows. Over the last ten weeks, traffic grew about 8x while the cost of running Cartage-1 roughly doubled. Since frontier models operate on token pricing, the cost would have been 8x what it was before, which would have forced Cartage to limit or remove Wilson’s capabilities.
“TrainLoop helped us build a model that’s an expert in freight at a fraction of the cost of Sonnet. Our inference is now so efficient that we were able to change our entire business model. We can give Wilson away for free and make money on the freight he moves.”
Abdul Basharat, CEO of Cartage
What’s next
Cartage-1 will keep improving as Cartage’s scope and needs continue to evolve. As the Cartage team takes on new customers and new kinds of work, Wilson is going to run into things it hasn’t seen before. Because they use the TrainLoop platform, the model evolves with them, so they never have to give away their secrets to the world to improve the frontier on logistics.
If you’re running an agent on a frontier model, chances are you have your own long tail of edge cases hiding in your production traces too. We’d love to help you turn those into a model that’s yours, so reach out at trainloop.ai. And if you want to see Wilson in action, check out cartage.ai.
Training reasoning models aligned with your goals.
Email: founders@trainloop.ai
LEGAL
© 2026 TrainLoop. All rights reserved.
North Beach, San Francisco, CA