An AI workflow passed its pilot. A month later, someone updates the model, adds a document source or enables a connector action. The screen looks familiar, but the evidence behind the original approval may no longer describe the system people are using.
For a business owner, the useful question is straightforward: what changed, what consequences could that change affect and what proof is needed before expanding use?
Treat an AI agent as an operating service with a change record. Keep review proportional to the change, and connect each test to a specific business or security consequence.
Record more than the model name
The model is one part of the workflow. Record the application version, system instructions, connected tools, tool permissions, approved information sources and relevant operating settings. Include the business purpose and the person who owns it.
When something changes, compare the proposed configuration with the last accepted one. Adding a new mailbox, changing a retrieval filter or allowing an action can matter even when the model stays the same.
NIST’s Secure Software Development Framework offers a common vocabulary for secure development and supplier discussions. Apply that discipline to the software and integrations around an agent, while adding evaluations for the AI behavior your workflow depends on.
Match the test to the consequence
A revised writing instruction calls for checking output quality and required disclosures. A new information source calls for checking access, source selection and answer support. A connector update calls for checking the operations it exposes and the identity under which they run.
Changes that affect money, external communications, sensitive information or access rights deserve explicit attention to authorization and business review. OWASP’s excessive agency guidance identifies unnecessary functionality, permissions and autonomy as sources of risk. It recommends enforcing authorization in downstream systems and limiting tool capabilities.
Do not assume that a previous refusal proves a boundary remains enforced. Verify the changed path using the actual permissions and controls proposed for release.
Keep a small, durable evaluation set
Build a set of representative tasks from the approved workflow. Include normal work, missing information, conflicting sources, unauthorized requests and failure conditions. Use approved test data and record the expected outcome for each case.
Check both the result and the action taken. A polished answer can conceal an unnecessary tool call. An agent may correctly decline a request while having already retrieved information the user should not see.
Separate mandatory controls from quality preferences. An unauthorized action is a release blocker even if the writing improves. Minor phrasing differences may be acceptable when the answer stays accurate, supported and useful.
Keep the test inputs, relevant configuration and results together. Repeat runs where variability could change the decision, with the number and acceptance rule chosen for the workflow’s risk.
An illustrative connector change
Imagine an internal assistant that drafts supplier follow-up messages for employee review. This example is hypothetical and describes no Z7 client.
A connector update adds a send operation alongside draft creation. The change owner must determine whether the application can call it, which mailbox identity it uses and whether the approved workflow permits sending at all.
If the accepted scope is drafting, disable or remove sending through an enforceable control. Test a direct request to send, a request embedded in retrieved material and a normal draft task. Inspect the action records to confirm what happened.
If the business wants automatic sending, that is a scope decision. Define recipients, review requirements, limits, evidence and recovery before accepting the new behavior.
Make the release decision readable
Use a concise record: the change, affected workflows, tests performed, observed results, remaining limitations and decision owner. Preserve the last accepted configuration and a practical rollback path.
Confirm how to stop tool actions without losing the evidence needed to investigate. A rollback may restore application behavior while leaving messages already sent or records already changed. Those consequences need a separate handling plan.
For a vendor-managed service, ask which changes the vendor can make, what notice is available and which controls you can verify yourself. Where versions cannot be pinned, agree on monitoring and reevaluation triggers rather than promising an unchanging system.
Budget for keeping the approval current
Evaluation takes staff time, test data preparation and sometimes model or infrastructure usage. Include those costs in the operating plan. Ask who maintains the cases, reviews failures and decides when a change needs broader testing.
A focused retest can reuse evidence whose inputs and controls remain unchanged. Record that reasoning so the next reviewer understands the boundary of the conclusion. Passing the changed cases supports a specific release decision; it does not establish that every possible behavior is safe.
Book an AI security review conversation. Bring one agent workflow and its proposed change so the review can focus on the controls and evidence that decision requires.