top of page

How to Change an AI Prompt Without Losing the Old Behavior

Writer: Branden Bell
Branden Bell
Nov 12
4 min read

Updated: Nov 13

Keep the old version. An olive paper pocket holds the old instruction slip beside a new cream slip with a terracotta corner.

A team shortens its intake prompt because the task summaries are hard to skim. The new summaries fit on one line. Then someone notices what’s missing: if the customer hasn’t given a delivery address, the task no longer asks for one.


Save the working prompt as a version before you change it. Decide which behavior should improve and which rules must survive. Run both versions on the same examples, including the awkward ones, and keep a copy you can restore. A smaller prompt is useful only if the next person still gets what they need.


Write down what can change


“Make it shorter” is a reasonable request. It’s a poor release check. Describe the observable change instead: “Put the request in the first line and make the rest easy to scan.” Then add a separate rule: “If the shipping address is missing, label the request as waiting and ask for the address.”


The Claude Platform guide to building evaluations recommends clear, measurable success criteria and test cases that reflect real work, including edge cases. For a small manual pilot, use a short set of anonymized requests your team can grade. Decide which failures block the change before you see the outputs.


One line saved, one handoff lost


Consider a fictional supplier’s intake workflow. Its prompt prepares a task draft for a coordinator. The coordinator checks the result before assigning any work. The team runs both prompt versions against three sample requests:


  • Complete request: Both drafts show the correct address. The candidate is easier to skim.

  • Missing address: The current prompt says “Waiting for address” and asks the customer for it in a draft. The candidate says “Ready for fulfillment” and leaves the field empty.

  • Conflicting addresses: Both drafts route the request to the coordinator to confirm which address is correct.


The candidate doesn’t pass. The complete request is easier to read, but the missing-address case could send an incomplete task to fulfillment. Keep the current version active. Add the required-field rule to the candidate and rerun all three cases. Recheck the source document version before testing if the intake rules have also changed.


A change card you can copy


Keep this card alongside the prompt so a second person can repeat the review without a meeting:


  • Working version: v1.4, with its prompt, model/settings and source references.

  • Proposed change: v1.5 candidate, shorter first line; retain the waiting status when required details are missing.

  • Comparison set: Three sanitized requests: complete, missing address and conflicting addresses. Save expected outputs or pass rules beside each example.

  • Release decision: Coordinator records pass or hold. A wrong readiness status blocks release regardless of shorter text.

  • Restore plan: Point the workflow back to v1.4. Rerun the missing-address case in draft mode and check for “Waiting for address.” Have the coordinator review affected tasks before retrying anything.


The NIST Generative AI Profile recommends reviewing generated output against defined expectations and planning how to respond to incidents. A small team can apply that idea with a short change note and a restore step it has tested. Restoring a prompt affects future runs; it doesn’t undo tasks or messages already created.


Run it on a change you already want


Pick one prompt your team edits often. Copy the current prompt into your normal version history. Choose one ordinary case, one missing-input case and one case where somebody has to make a judgment. Run the current prompt first, then the candidate on the same inputs and settings where practical. Keep both outputs and compare them against the written rules. Repeat important edge cases if outputs vary.


Once the candidate passes, try it in draft mode with a named reviewer. Track how often the reviewer has to correct a critical field or reopen a case. Add any missed case to your AI workflow testing checklist before the next revision.


Prompt change FAQs


If the revision also changes what an assistant can see or do, use the connected workflow permission check to test both permitted access and an expected denial.


Do I need special software to version a prompt?


Start with version history in a tool your team already uses. Record the exact instructions, version, owner, test cases and any settings or source changes that could affect behavior. Keep the active version easy to identify.


Should I change the prompt and model together?


When you can, change one thing at a time so you can tell what caused a difference. If both must change, record both and run the full check before release.


How many examples are enough?


Three examples can start a small manual trial. They aren’t proof that the workflow is safe on every input. Add cases for expensive mistakes, common exceptions and every real failure you want to prevent from recurring.


What if the new prompt has already created a bad task?


Pause the affected routine, preserve the output and identify which tasks used the new version. Restore the prior version for future runs and let the owner review each affected task before changing or replaying it.


Bring one prompt that keeps changing


If a prompt keeps changing in ways nobody can explain, bring a recent edit and a draft it affected. Ask Bell Consulting Solutions to review the workflow, or see how a process review can help find where the checks belong.

 
 
 

Comments


bottom of page