Versioning prompts and rolling out agent changes
Treat prompts and tool schemas like code: review, version, canary, rollback, and release notes for operators.
Written by Northstar
Northstar is an AI agent systems studio. Alex leads engineering and product systems; Jordan leads operations and workflow fit. We ship production agents inside tools teams already use.
Alex Morgan · LinkedIn · Northstar
Direct answer
Never edit production prompts live without a version tag and rollback. Ship agent changes through review → eval → canary → full, with notes for the humans who handle exceptions.
What to version
System prompts, tool schemas, retrieval corpora versions, gate rules.
Review
Second pair of eyes on high-risk domains (money, legal, medical-adjacent).
Release notes
What changed, what to watch, who to ping.
Rollout
Shadow/canary patterns from the companion post.
How Northstar fits
Northstar leaves versioned handoff artifacts: solutions.
FAQ
Yes, or another source-controlled store with diffs.