How we prove a cheaper model can do the job
Your agent runs on your priciest model. Here's how a cheaper one gets tested on your own past chats before any traffic moves.
Blog
Notes on measuring quality, learning from what your agents already did, and finding the smallest change that makes a real difference.
Teaching a cheaper model how your team does a task
A cheaper model doesn't know how your team works. Here's how Operant writes that down from your own past chats, and what a person checks.
Testing two models without users bouncing between them
Split live traffic between two models by weight, keep each user on the same model, and see what changes when you move the weights.
Can a small model grade your agent's work?
We tested a small, fast model against bigger ones on 240 agent chats. It matched the big model closely, but every grader missed the same kind of failure.
Guessing the task from a chat's first message
A new chat only has its first message. We tested which models can name the task from it, and which ones fail safely when nothing fits.
Turning your agent's past work into tests
Logs are only history until you pair them with a judgment. Here's how picking, splitting and grading past chats turns them into tests you can trust.
Write down the task before you train a model
When a smaller model fails at a repeated task, the gap is often missing context, not ability. Writing that context down can beat training.
Cut your model costs in the right order
Find the work that repeats, prove a cheaper setup holds, and only then think about training. Each step is earned by the one below it.
When should a cheaper model take over a task?
Top models are the right default for new work. A cheaper model earns a task once it repeats, the test is stable and cost holds your product back.
Your team decides what good work means
A test of your agent encodes product judgment, like which errors matter and which tradeoffs are fine. That judgment belongs to the people who own the task.
Introducing Operant: a smaller bill for your AI agents
Operant sits between your agent and the model provider. It finds the waste in your bill, prices it and removes it, one tested step at a time.
Four ways an agent overpays for its model calls
Cache discounts left off, model switches that break the cache, repeated tasks solved from scratch, and old context paid for on every turn.
Savings numbers you can defend line by line
Any tool can claim it saves money. Here's how every call gets a price you can check, and why the assumptions are part of the record.