Frame the risk in operating terms
An AI feature becomes a security boundary when it can read internal documents, call tools, write records or influence a consequential decision. Direct and indirect prompt injection can arrive through user input, retrieved webpages, files or connected systems. The model should therefore never be treated as the authority that decides its own permissions, validates its own output or determines whether an action is safe.
Treat the subject as an operating system rather than a product setting. The accountable owner is the business owner of the AI-enabled workflow, supported by application security and data owners. Important decisions include which information the system may retrieve, which tools it may invoke, which actions require human approval, how unsafe output is contained. Record assumptions, dependencies and accepted exceptions so a future reviewer can distinguish deliberate design from accidental exposure.
Engineer a controlled path
Keep identity, authorisation and transaction rules in deterministic application code. Give each tool a narrow schema, scoped credential and explicit consequence rating. Separate untrusted retrieved content from trusted policy, validate model output before another system consumes it and require human confirmation for high-impact or irreversible actions. Limit context to the minimum data needed for the task and prevent one user or tenant from influencing another.
Build the control path in layers: server-side authorisation, least-privileged tool credentials, input and output validation, human approval for consequential actions, bounded context and consumption. Apply stronger safeguards where a failure could change production, expose sensitive information or block recovery. Pilot changes with representative users and services, define rollback criteria and make emergency access narrower and more observable than ordinary administration.
Operate from evidence, not reassurance
Log the request path, retrieved sources, model and policy version, tool decision, approval and resulting action without storing unnecessary secrets. Test direct injection, poisoned documents, excessive tool calls, data leakage and misleading output. Rate limits and spending controls should bound consumption, while kill switches and credential revocation should stop an unsafe integration without disabling unrelated services.
Useful evidence includes tool-call audit records, source provenance, policy decisions, approval events, red-team results. Keep collection protected from the systems being observed, test that records arrive when expected and assign a person who can act on a meaningful signal. Measures such as high-impact actions stopped or approved, untrusted-source coverage, tool exception age, time to revoke an integration should lead to a decision, not merely decorate a dashboard.
Exercise failure and improve the design
Use adversarial test cases that hide instructions in documents, webpages and structured fields, then observe whether the system crosses an authorisation boundary. Practise revoking a tool credential, removing a compromised source and reconstructing the actions taken during one session. Reassess whenever the model, system prompt, retrieval source, tool set or decision threshold changes.
Run a bounded exercise that forces the team to use real contacts, authority, evidence and recovery paths. Capture what was known, what was assumed and which action changed the outcome. Give every improvement an owner and validation method, then repeat the difficult part after the change instead of closing the work on written assurance.
What to do next.
- ✓Inventory every model, data source and tool.
- ✓Keep permission checks outside the model.
- ✓Separate trusted policy from untrusted content.
- ✓Require approval for consequential actions.
- ✓Record tool decisions and resulting changes.
- ✓Re-test after model, prompt or tool changes.
Use guidance responsibly
Technology, provider behaviour and legal requirements change. Validate implementation details against current official documentation, test changes safely and obtain qualified legal advice where required.