LLM cost and model choice for business agents
How to pick models for ops agents and control spend: routing, caching, budgets, and fallbacks.
Written by Northstar
Northstar is an AI agent systems studio. Alex leads engineering and product systems; Jordan leads operations and workflow fit. We ship production agents inside tools teams already use.
Alex Morgan · LinkedIn · Northstar
Direct answer
Pick the cheapest model that passes your eval bar for each step. Control cost with routing, caching, truncation, step caps, and hard budgets per run - not by hoping the invoice stays small.
Model choice
Strong models for hard reasoning and risky drafts; smaller/faster models for classification and extraction if evals pass.
Routing
Classifier step → specialized prompts/models. Do not use a frontier model for every “yes/no”.
Caching
Cache stable system instructions and repeated retrieval chunks where policy allows.
Budgets
Max tokens, max tool calls, kill switch on spend anomalies.
Fallbacks
Secondary model or queue-to-human when primary errors or times out.
How Northstar fits
Cost-aware design is part of Northstar pilots: solutions.
FAQ
Not if you pay GPU ops and quality drops cause rework.