A private AI demonstration can answer a question beautifully while leaving the buying decision unresolved. Where did the documents go? Can employees retrieve information outside their department? What happens when the system cannot find an answer? Who operates it after the pilot?
Before comparing servers or model names, write down what the finished service must prove. A useful proposal makes those tests part of acceptance, with named owners and evidence the buyer can inspect.
This matters whether you lead a public agency, a school, a professional services firm or a growing business. The deployment boundary should follow the workflow, its information and its consequences.
1. Follow a document through the entire system
Ask the supplier to trace a representative document from ingestion to deletion. Include the original file, extracted text, search index, model requests, answers, logs, backups and support access. Identify every service involved and any outbound connection.
Write down who controls each location, how long information remains there and what deletion actually removes. Confirm whether external processing is optional, required or prohibited. A diagram that ends at the model server leaves too much unexplained.
Treat this as a demonstration: provide an approved test document, observe processing and inspect the relevant configuration. Avoid using real sensitive records until the proposed handling has been reviewed.
2. Test the boundary between two employees
Place approved test documents in two departments with different access rights. Have each employee ask questions that would require the other department’s material. The expected result is an enforced denial of unauthorized retrieval, including snippets and citations.
Then change an employee’s access and repeat the test. Ask how quickly permissions propagate and what happens to cached answers or previously created conversation history.
NIST’s Zero Trust Architecture rejects implicit trust based solely on network location or ownership. Applying that principle to private AI means testing authorization even when every server sits inside your organization.
3. Make uncertainty visible
Build a small evaluation set from the actual job. Include answerable questions, outdated documents, conflicting procedures and questions whose answers are absent. Define what a reviewer needs: a useful response, the supporting source and a clear signal when evidence is insufficient.
Do not score only whether the answer sounds persuasive. Check whether the cited passage supports it and whether the document is current. Separate document retrieval failures from reasoning errors so the team fixes the right problem.
For consequential workflows, decide which outputs require human review. An internal policy search and a decision affecting a person’s services need different acceptance criteria.
4. Test the busy period and the failure path
Ask for a workload test that resembles your expected use: document sizes, simultaneous users, response length and required turnaround. Record measured results against agreed requirements. Avoid treating a single-user demonstration as a capacity plan.
Next, interrupt an approved test dependency. Confirm that users receive a useful failure message, work remains recoverable and the operating team can identify the fault. Require a demonstrated backup and restore procedure for the components needed to resume the service.
The buyer should know who handles patches, model changes, access reviews and incidents. Those responsibilities belong in the proposal alongside the hardware.
5. Compare the cost of an accepted task
Request separate estimates for initial setup, infrastructure, licensing, integration, support, evaluation and ongoing maintenance. Include internal staff time for document cleanup, permission repair and reviewing answers.
Then compare the same workflow across the proposed options. A useful planning measure is total operating cost divided by tasks accepted by the business reviewer. Include failed attempts and human correction time. Define an accepted task before measuring it.
There is no honest universal price for private AI. Required capacity, availability, integration complexity and operating responsibilities change the economics. Ask suppliers to state assumptions and show how cost changes when those assumptions change.
An illustrative buying decision
Consider an organization evaluating an assistant for internal purchasing procedures. This is a hypothetical example, not a client result.
The first release could cover approved procedures for one department, exclude supplier banking records and produce cited draft answers. Procurement owns answer acceptance; IT owns access and recovery; the sponsor approves the operating budget.
The purchase decision follows demonstrated results: authorized users receive supported answers, excluded material stays inaccessible, failure handling works and the operating estimate includes review time. Expansion becomes a separate decision with new tests.
That approach turns “private AI” into something a buyer can inspect. It also gives the supplier a concrete target and a fair way to demonstrate value.
Book a Private AI planning conversation. Bring one workflow, its information restrictions and the evidence you would need to approve a pilot.