01. Start with a bounded healthcare workflow
Reduce a specific burden while keeping patient safety, professional judgment and the existing care process in view.
Choose the first task carefully
Start with approved-policy training, public patient education resources or a nonclinical operations brief. Use synthetic data until the workflow, contracts and controls are approved. Familiarity does not establish suitability for patient information.
Keep diagnosis, triage, prescribing, treatment selection and other clinical decisions outside this starter pilot. Clinical use requires its own safety, regulatory and workflow assessment. This guide does not authorize autonomous clinical decisions. A clinician’s name on a workflow is insufficient unless that clinician can inspect and reject the output.
Define a pilot charter
Scroll sideways to see the whole table.
| Review area | Practical check |
|---|---|
| Problem | Which repeated task creates delay, rework or avoidable burden? |
| Scope | Which staff, data, department and outputs are allowed? |
| Review | Who checks accuracy, safety and appropriateness? |
| Boundary | What may the tool never decide, send or change? |
| Stop | What event pauses the pilot, and who can act? |
Make the current process the baseline
Observe representative tasks before introducing AI. Record total effort, correction rates and handoffs. For example, a patient education pilot could compare staff preparation and review time using approved public material. Ask whether the final handout remains accurate and understandable.
Include nursing, clinical leadership, privacy, security and the operational owner. Ask staff to identify failure points before purchasing or connecting a system.
02. Approve the patient data path
Determine what information is processed, who handles it and what controls apply before allowing protected health information into the workflow.
Follow the complete flow
Map prompts, uploads, recordings, source systems, generated text, logs, backups and support access. Include integrations and subcontractors. Minimize information at every step. Removing names alone does not establish HIPAA de-identification; use the organization’s approved method and review.
HHS cloud guidance explains that a provider handling electronic protected health information on a regulated entity’s behalf can be a business associate even when it cannot decrypt the data. Determine the relationship and required business associate agreement before use. A signed agreement does not replace the organization’s risk analysis or operational controls.
Verify the control responsibilities
- Identity: unique accounts, appropriate access by role, prompt removal when access is no longer needed.
- Data use: enforceable limits on training, secondary use, support access and disclosure.
- Retention: documented handling of prompts, recordings, outputs, backups and termination.
- Audit: enough protected evidence to investigate activity without unnecessarily copying clinical content.
- Recovery: tested continuity procedures and a practical way to disable the integration.
Choose by requirements, not labels
On-premises systems need patching, access controls and support. Cloud services require appropriate contracts, configuration and review. HHS describes the Security Rule as technology neutral. Evaluate the workflow against applicable requirements.
When information enters an unapproved service, stop that use and contact the organization’s privacy and security teams. Follow their incident process, preserve necessary evidence and avoid further dissemination.
03. Use templates that support professional review
Begin with approved nonpatient material or synthetic scenarios. These examples produce drafts for qualified staff.
Patient education preparation
“Using only these approved patient education materials, draft a plain-language explanation of [topic] for staff review. Preserve the source’s warnings and instructions. Include questions the reader could ask their care team. Do not add a diagnosis, recommend a treatment, change a medication instruction or invent a citation. List unclear passages and identify the source for each section.”
A qualified reviewer checks medical meaning, reading level, accessibility and local contact details. Use the organization’s language-assistance process for translations. Do not send generated material directly to a patient without the required review.
Staff policy training
“Turn this approved policy dated [date] into a ten-minute staff training outline. Include the stated process, responsible roles, escalation contact and five knowledge-check questions with source references. Distinguish the policy’s requirements from suggested teaching examples. Flag missing or conflicting instructions. Do not create a new policy.”
The policy owner validates the outline and answers. If the source conflicts with current practice, resolve that conflict before training staff. A model should not silently decide which version governs.
Operations briefing
“Using these approved aggregate figures and nonpatient notes, prepare an operations briefing with observed issues, possible explanations, data limitations and questions for the team. Do not infer causation or individual performance. Label proposals as proposals.”
Verify that small groups or detailed narratives do not expose individuals. Check calculations independently. Keep scheduling, staffing, billing and record changes outside the assistant’s authority unless separately assessed and explicitly approved.
04. Test the workflow before expanding it
Safety depends on the application, its users and its integration, as well as the model’s performance.
Build a representative review set
Use synthetic cases and approved source materials to test ordinary requests, incomplete information, conflicting instructions and misleading statements. Have the appropriate specialists define unacceptable results before the demonstration. Include language and accessibility needs relevant to the people who will use the service.
Check that the system identifies unsupported answers and routes uncertainty to a person. Test whether an account can retrieve another department’s restricted information, whether sensitive content appears in logs, and whether an uploaded document can manipulate tool behavior. Fix the controls before repeating the affected tests.
Keep clinical regulation a separate question
FDA guidance addresses when clinical decision support software functions may fall outside the device definition and when device oversight may apply. Have qualified regulatory and clinical reviewers evaluate the intended use and actual function. Neither a generic “assistant” label nor human review alone resolves every regulatory question.
For clinical software, examine evidence for the specific population and setting, foreseeable harm, user training, monitoring and failure response. A cleared device or vendor study does not remove the need to evaluate local implementation.
Practice the fallback
- Disable the tool and confirm staff can continue the underlying workflow.
- Withdraw or correct a flawed draft before it reaches patients or a clinical record.
- Revoke a connector’s credentials and verify the resulting access boundary.
- Record who investigates a safety concern and who approves restarting.
- Recheck after material changes to the source collection, model, interface or integration.
05. Measure benefits without hiding review work
Use operational evidence and safety findings to decide whether a pilot should continue.
Measure the complete task
Count preparation, review, correction, documentation and support time. Compare similar work with the baseline and describe sample limits. Track inaccurate statements, missing warnings, unsupported references, access failures and staff workarounds. A quick draft that needs extensive correction may not improve the workflow.
For patient-facing material, check meaning and whether readers understand the next step. Staff satisfaction and faster drafting do not establish clinical benefit. Clinical outcome claims require appropriate evidence.
Use a concise decision record
Scroll sideways to see the whole table.
| Review area | Practical check |
|---|---|
| Quality | Reviewed examples, corrections and unresolved failure patterns. |
| Privacy and safety | Incidents, access findings and completed corrective actions. |
| Effort and cost | Total staff effort plus licenses, infrastructure and support. |
| Readiness | Owner, training, fallback, maintenance and review schedule. |
| Decision | Continue, narrow, change or stop, with reasons and approver. |
Expand through another bounded review
If the pilot meets its criteria, change one boundary at a time. Adding patient information, a new department, a clinical function or write access is a new assessment, not a routine feature toggle. Update training and documentation before the expanded workflow starts.
Bring a workflow, safe example, data-flow map and results to leadership. Keep a named owner responsible for ongoing operation, review and pausing when evidence changes.