AI agent agency vs building in-house: how to choose
A practical decision framework for production AI agents: when an agency wins, when in-house wins, and the failure modes of each path.
Written by Northstar
Northstar is an AI agent systems studio. Alex leads engineering and product systems; Jordan leads operations and workflow fit. We ship production agents inside tools teams already use.
Alex Morgan · LinkedIn · Northstar
Direct answer
Choose an AI agent agency when you need production systems mapped to real workflows fast, with engineering plus operations discipline, and you do not yet have in-house agent ops capacity. Build in-house when agents are a core product moat, you can staff ongoing ownership, and you already have clear process maps and approval rules.
Most teams fail between those poles: they hire a freelancer for a demo chatbot, or they launch an internal "AI initiative" without workflow discovery. Production agents live inside tools your team already uses - inbox, CRM, sheets, chat - with human gates on risky actions.
Decision criteria
| Criterion | Agency fit | In-house fit |
|---|---|---|
| Time to first production path | Faster if discovery is included | Slower until hiring + process catch up |
| Knowledge retention | Risk if vendor holds all context | Stronger if team documents systems |
| Control and approvals | Good when gates are designed up front | Best when security owns the stack |
| Cost shape | Project + optional retainers | Salaries + infra + management load |
| Differentiation | Best for ops leverage, not unique IP | Best when agents are the product |
| Hiring reality | No agent-eng hiring needed | Senior agent engineers are scarce and contested |
| LLM cost ownership | Vendor estimates, you hold the keys | You own budgets, caps, and optimization |
| Evals and regression | Should ship with the pilot | You must build the discipline yourself |
| Incident response | Defined in the retainer, or absent | Your on-call rotation, your pager |
What each path really costs
The honest comparison is total first-year cost, not the pilot invoice against a salary line.
Agency path: project fees plus an optional retainer plus LLM usage. Typical market ranges: audits around $2,000-15,000, single-workflow pilots $10,000-50,000, ops retainers $1,500-10,000 per month. These are typical market ranges, not Northstar quotes.
In-house path: salaries plus management time plus infrastructure plus LLM usage plus on-call. A minimal credible team is one senior engineer with real agent experience plus a fraction of a product owner. In most markets that is a significant six-figure annual commitment before any model spend.
On both paths: LLM call costs are a permanent budget line, not a launch cost. Estimate per workflow at real volume, hold your own API keys, and set caps so a retry loop cannot surprise finance.
The 12-month view
| Period | Agency path | In-house path |
|---|---|---|
| Months 1-3 | Audit, pilot, first gated workflow in production | Hiring, tool selection, first prototypes |
| Months 4-6 | Second workflow, retainer ops, team training | First production path, if hiring landed |
| Months 7-12 | Handoff to an internal owner, or steady retainer | Eval discipline, on-call rotation, roadmap |
The agency path buys speed and transfers risk early. The in-house path buys ownership but pays for it in calendar time and hiring risk.
When an agency is the right call
- You need a custom agent system for intake, support, ops, or knowledge work, not a slide deck.
- You want agent-first product work with delivery ownership.
- You need help defining approval boundaries before anything autonomous ships.
- You also care about AI visibility (SEO/GEO) so the business is discoverable while systems ship.
Northstar approaches this as an agent systems studio: process discovery first, then implementation paths that fit existing tools. See solutions and start an audit.
When in-house is the right call
- Agents are the product customers pay for, and the roadmap is multi-year.
- You already run platform eng, evals, and on-call for automation.
- Compliance requires all logic and data residency under your entity only.
Even then, short agency sprints for discovery or architecture reviews can reduce expensive rework.
The hybrid path most teams should consider
For most mid-size companies the strongest pattern is sequential, not either-or:
- An agency runs discovery and ships the first gated workflow to production.
- Your team operates it during a defined support window, with the agency on call.
- An internal owner takes over the runbook, eval set, and gate rules.
- The agency exits to advisory, or ships the next workflow while your owner runs the first.
Write the exit criteria into the first contract: which artifacts transfer, who gets trained, and when the retainer can end without penalty. A vendor who resists defined exit criteria is selling dependency, not capability.
Failure modes (both sides)
Agency path fails when the engagement skips workflow mapping, ships chat UI without tool integrations, or never defines who approves irreversible actions. It also fails quietly when the vendor holds all context and every change becomes a ticket.
In-house path fails when leadership funds models and prompts but not ownership, evals, or change management. Tool sprawl grows; nobody maintains the agent after the pilot; the one engineer who understood the system leaves.
A practical sequence
- Map 3-5 high-volume workflows and their failure cost.
- Mark which steps need a human gate.
- Pick one path to production in existing tools.
- Measure cycle time and error rate for 2-4 weeks.
- Only then expand or hire a permanent agent team.
How Northstar helps
Northstar combines engineering and operations: custom agent systems, agent-first products, consulting, and AI visibility systems. Public work includes products like Atlas and Dev Platform. If you want a grounded audit of where agents belong in your stack, use the site CTA for a free consultation.
FAQ
No. Chatbots answer questions. Production agent systems execute multi-step work with tools, rules, and human review.