'Human in the loop' sounds reassuring. But the phrase means very little if the reviewer receives hundreds of outputs, cannot understand their limitations and has no authority or time to challenge them. In a pharmaceutical organization, the real question is who owns the consequence of a recommendation.
Separate information from authority
An AI system may summarize permitted information, recommend a possible action or execute a bounded task. These are different levels of operational authority. They should not share the same approval process simply because they use similar technology.
Inform
A system that summarizes internal material still requires source traceability, access controls, accuracy checks and a way to report errors. The human reader must understand that a fluent summary is not necessarily a reliable one.
Recommend
A system proposing a commercial action requires more: clear criteria, an accountable decision owner, explanation of important limitations and a process for rejecting the recommendation. The owner must be able to exercise judgment rather than rubber-stamp a result.
Act within guardrails
Automated execution raises the bar. Define permitted actions, approval thresholds, monitoring, exception handling, auditability and the ability to stop. Decisions involving patient safety, regulated communications or significant compliance risk require the appropriate specialist oversight and governance.
Worked example
Consider a illustrative system that suggests which approved educational material might be relevant to an HCP segment. Before use, teams should clarify the data and permissions involved, whether the content is appropriate for the intended audience, who approves the selection logic and how inappropriate recommendations are detected. A suggestion is not an authorization to communicate.
The accountability test
Ask who can explain the recommendation's role, challenge it, override it, report a problem and suspend its use. If those responsibilities are ambiguous, the system's governance is incomplete even if the technology performs well in testing.
AI accountability is an operating-model question. The aim is not to place a human signature on every output; it is to preserve meaningful human authority where consequences require it.
Follow the decision
What evidence would justify deploying the system? This question connects to two other parts of the operating model:
Illustrative scenario. The views I share are my own and do not represent those of my employer.
Put this into practice
AI oversight must be executable, not ceremonial.
Test authority, evidence and escalation using the AI Accountability Decision Laboratory.
Test the AI accountability boundary