Content Safety and Data Handling
AI Gateway can inspect model inputs and outputs. Enterprise deployments can configure sensitive-word inspection chains, an optional LLM content guard, and security scanning services integrated through plugins to suit their application requirements.
Content inspection
| Mode | Purpose |
|---|---|
| Streaming | Inspect content during streaming output |
| Non-streaming | Inspect complete requests or responses |
| Trusted-request exception | Skip checks for explicitly configured trusted requests where supported |
Plugins and third-party security scanning
AI Gateway supports integrating security scanning services through plugins. Enterprises can select a scanning service that meets their application requirements and connect it to the gateway using a plugin.
Custom plugins can integrate additional third-party security scanning services, allowing enterprises to reuse their existing scanning capabilities. Refer to the selected plugin's documentation for integration instructions and configuration parameters.
Configure content inspection
For Kubernetes Helm deployments, configure gateway-level content inspection under aigateway.moderation in the deployment values. To use the optional LLM content guard, also configure aigateway.moderation.llm to connect the appropriate guard-model service.
| Setting | Purpose |
|---|---|
aigateway.moderation.enabled | Enable or disable gateway-level content inspection |
aigateway.moderation.checkChain | Configure the inspection chain |
aigateway.moderation.streamCheckMode | Configure streaming inspection mode |
aigateway.moderation.llm.enabled | Enable or disable the optional LLM content guard |
aigateway.moderation.llm.endpoint / apiKey | Configure the guard service address and credentials |
aigateway.moderation.llm.guard.model / guardStream.model | Configure the standard and streaming guard models |
See Moderation in the Kubernetes Configuration Guide for configuration, defaults, and deployment instructions. To use the platform’s sensitive-word list, maintain its rules in Asset Management, then configure the gateway’s inspection switch and chain.
After configuration, check ordinary input, rule-matching input, and streaming and non-streaming responses. Applications using streaming inspection should handle inspection failures or stream interruptions and manage any content already received.
Exceptions and access permissions
Use inspection exceptions to specify which request sources and scopes skip content checks. Verify behavior for requests inside and outside each exception. Model access remains subject to the service’s authorization rules.
Identity and Access describes who can call a service. This page addresses how its request content is handled.
Request bodies and logs
Request records can include prompts, tool calls, and streamed outputs for troubleshooting, auditing, and analysis. Determine the deployment's collection scope, storage location, access controls, and retention rules according to configuration and enterprise requirements.
Before using request records for training or fine-tuning, follow the enterprise data-authorization process and select data within the approved scope.
Self-hosted and external destinations
Request content is sent to the upstream selected by routing. To keep data within the enterprise, review both primary and backup upstreams and select services that meet enterprise data requirements.
See Network Requirements and Deployment Architecture.
Related guides
Monitoring, Logs, and Troubleshooting · Routing and Reliability