AI API Gateway

Bring model traffic
under one policy layer

Route supported OpenAI-compatible traffic through a controlled gateway for DLP, scoped keys, provider routing and audit context.

Request an AI Security Assessment Talk to Sales
4
LLM providers: OpenAI, Anthropic, Gemini, Azure
7
ML DLP engines on both prompt and response
<1ms
Regex detection latency
~35ms
Average ML detection overhead
OpenAI-Compatible API

Change one line. Get
full policy enforcement.

AIGuard implements the OpenAI REST API spec exactly. Applications using the OpenAI SDK, LangChain, or any OpenAI-compatible library route through AIGuard with a single configuration change.

Integration — Python SDK
Before
from openai import OpenAI

client = OpenAI(
  api_key="sk-..."
)
After — one line changed
from openai import OpenAI

client = OpenAI(
  api_key="ag-app-key-...",  # AIGuard key
  base_url="https://gw.aiguard.io/v1"
)
✓ DLP scanning — both directions
✓ Model allow/block list enforced
✓ Rate limits applied per key
✓ Full audit log recorded
✓ Circuit-breaker active
  • Full OpenAI API compatibility — completions, chat completions, embeddings, and streaming (SSE). Any client that works with the OpenAI SDK works with AIGuard — no wrapper libraries, no code changes beyond base URL and key.
  • Per-application gateway keys — SHA-256 hashed, scoped to specific models and providers, rate-limited (requests/minute via Redis), and instantly revocable without redeploying your application.
  • Central provider key management — your OpenAI, Anthropic, Google, and Azure keys are stored encrypted in AIGuard's vault. Application teams never see the underlying provider keys — they only see their scoped gateway key.
  • Proof of Possession (Ed25519) — optional key binding that prevents replay attacks. Gateway keys can be bound to a deployment identity; a stolen key is useless on a different host.
▲
OpenAI

GPT-4o, GPT-4o-mini, o1, o3

◆
Anthropic

Claude 3.5, 3.7, 4 Sonnet/Opus

✦
Google Gemini

Gemini 1.5, 2.0, 2.5 Pro/Flash

⬡
Azure OpenAI

All deployments, all regions

Policy Enforcement

Every token scanned.
In both directions.

The same seven on-premises ML engines that power SaaS AI DLP scan every API prompt before it reaches the LLM provider — and every response before it reaches the application.

  • Bidirectional DLP — scan outbound prompts for PII, secrets, and injection, and scan inbound responses for sensitive data hallucinated or retrieved by the model. Block or redact before either direction reaches its destination.
  • Multi-provider routing — each gateway key specifies which provider and model it targets. Add a fallback: if the primary provider is unavailable, AIGuard routes to the secondary. Model allow/block lists per provider prevent applications from being upgraded to more expensive models unintentionally.
  • Circuit breaker — configurable fail-open or fail-closed mode when the AIGuard policy service is unreachable. Fail-closed blocks requests for sensitive applications; fail-open passes them through for low-risk use cases. Cached policy bundle (5-minute TTL) ensures the circuit breaker almost never needs to fire.
  • Full LLM audit log — every request is logged with: application key, provider, model, prompt token count, response token count, end-to-end latency, policy decision, and detection details. Searchable, filterable, exportable to SIEM.
  • Image OCR on multimodal requests — for applications sending images with prompts (GPT-4o, Claude 3, Gemini 1.5 Pro), AIGuard's built-in OCR engine extracts text from image attachments before DLP evaluation. Secrets in screenshots are caught before they reach the model.
Gateway · Live Traffic
Req/min
284
Blocked
12
P50 overhead
37ms
App KeyModelTokensDLPStatus
finance-app gpt-4o 1,240 Clean ✓ 201
hr-assist claude-4-sonnet 890 SSN ⊘ 403
code-review gpt-4o-mini 412 Clean ✓ 201
data-pipeline gemini-2.5-pro 3,100 M&A topic ! 200
legal-summarize gpt-4o 4,820 Redacted ↻ 200
Key Management

Scoped keys. Encrypted vault.
Instant revocation.

AIGuard's key management layer gives every application a scoped gateway key with its own model allowlist, rate limit, and DLP policy — while keeping underlying provider keys locked in an encrypted vault that no application can access.

SHA-256 Hashed Keys

Gateway keys are stored as SHA-256 hashes — the plaintext key is shown once on creation and never again. Even a full database compromise exposes no usable keys.

Model Allowlists

Each key can be locked to a specific model or set of models. An application key scoped to gpt-4o-mini cannot call gpt-4o — preventing cost overruns and unauthorized capability access.

Rate Limiting

Per-key request-per-minute limits enforced via Redis. When a key exceeds its rate limit, the gateway returns 429. Burst allowances configurable per key.

Ed25519 Proof of Possession

Optional cryptographic binding of a key to a deployment identity (host fingerprint). A stolen key used from the wrong host returns 401 — preventing replay attacks.

Gateway Keys · 4 active
finance-app-prod
ag-sha-3f9a…c21b
Active
gpt-4o only 300 req/min DLP: strict PoP: enabled
code-review-bot
ag-sha-8b2c…91af
Active
gpt-4o-mini, claude-4-haiku 1,000 req/min DLP: standard
hr-assist-staging
ag-sha-1e4d…77c3
Rate limited
claude-4-sonnet 50 / 50 req/min DLP: strict
How It Works

Gateway enforcement in four steps

01

Application sends request with gateway key

The application sends a standard OpenAI API request to the AIGuard gateway endpoint with its scoped gateway key. The gateway authenticates the key (SHA-256 hash comparison + optional PoP check), identifies the application, and looks up its policy configuration.

02

Prompt scanned by all 7 DLP engines

The prompt text (and any image attachments via OCR) is run through the full DLP pipeline on-premises. The regex engine returns in under 1ms; ML engines average ~35ms. If a violation is found matching an active rule, the gateway returns an error immediately — the prompt never reaches the LLM provider.

03

Request forwarded to the LLM provider

After DLP clearance, the gateway replaces the application's gateway key with the encrypted provider key (decrypted from the vault at runtime) and forwards the request to the configured LLM provider. Model allow/block lists are enforced — requests for disallowed models are rejected with a 403.

04

Response scanned and returned

The LLM response is scanned by the DLP pipeline before being returned to the application. Sensitive content in responses (hallucinated PII, model-retrieved secrets) is blocked or redacted. The full event — request, response, token counts, latency, and policy decision — is written to the audit log.

Capability Comparison

AIGuard Gateway vs. alternatives

Capability AIGuard Gateway LiteLLM AWS Bedrock Gateway Custom Proxy
OpenAI API compatibility✓ Full spec✓PartialManual
On-premises ML DLP (7 engines)✓———
Bidirectional scanning (prompt + response)✓—Guardrails only—
Per-key model allowlists✓✓—Manual
Encrypted provider key vault✓Plaintext env✓ IAMManual
Proof of Possession (Ed25519)✓———
Circuit breaker (fail-open/closed)✓ ConfigurableLimited—Manual
Full LLM audit log (tokens, latency)✓✓CloudWatch onlyManual
Multi-provider routing with fallback✓✓AWS onlyManual
FAQ

Common questions

Yes. AIGuard implements the OpenAI REST API specification exactly — the same endpoints, request/response formats, and error codes. Any client that targets the OpenAI API works without modification: the official OpenAI Python/Node/Go SDKs, LangChain, LlamaIndex, AutoGen, CrewAI, and any custom HTTP client. You change the base URL and the API key; everything else is unchanged.
Provider keys (OpenAI, Anthropic, Google, Azure) are stored AES-256 encrypted in AIGuard's key vault. They are decrypted in memory only at request time and are never exposed in logs, responses, or the management console. Application teams interact only with their scoped gateway key — they have no access to the underlying provider keys. If a developer workstation is compromised, the attacker cannot retrieve provider keys from it.
The gateway adds a P50 overhead of ~37ms (dominated by ML DLP inference). Regex-only rules add under 1ms. Image OCR adds 500ms–1s per image in multimodal requests. For most chat and completion use cases where the LLM itself takes 1–30 seconds to respond, the gateway overhead is imperceptible. For latency-critical applications, you can configure gateway keys to skip ML DLP and use regex-only policies, reducing overhead to under 2ms.
Yes. The AI API Gateway is independently useful as a key management, routing, and audit logging layer even without DLP enabled. A common deployment pattern: start with gateway-only (no DLP rules) to gain visibility into LLM usage and establish audit logging, then incrementally add DLP rules as you understand your applications' data patterns. DLP can be toggled per gateway key — strict enforcement for sensitive applications, none for low-risk internal tools.
Each gateway instance caches the current policy bundle with a 5-minute TTL. If the central AIGuard cloud service is unreachable, the gateway continues enforcing the cached policy bundle — so there is no enforcement gap during brief outages. For complete gateway failure scenarios, the circuit breaker behavior (fail-open or fail-closed) is configured per gateway key based on the application's risk profile. Fail-closed blocks all requests; fail-open passes them through without DLP. Most customers configure fail-closed for production applications.

One base URL change.
Every LLM API secured.

See AIGuard Gateway in a live demo — we'll integrate with your existing OpenAI SDK setup and run DLP against real traffic in under 15 minutes.

Request an AI Security Assessment Talk to Sales
Zero application code changes
Provider keys encrypted at rest
~37ms P50 overhead