Azure API Management AI gateway

Microsoft

Vendor

AI gateway policies in Azure API Management for LLM, MCP and A2A APIs: token limits, semantic caching, content safety and load balancing.

Curated by Tiberiu ArvaVerified 2026-09-30

Proprietary

Docs Website

Strengths

Per-consumer token limits and quotas
PTU-aware priority load balancing
Foundry integration for models, agents, tools

Practitioner note

The natural choice for Azure OpenAI / Foundry estates that already run APIM, especially to spread PTU and pay-as-you-go capacity across many apps. Check that the policies you need are available in your APIM tier, and note that semantic caching needs Azure Managed Redis or another RediSearch-compatible cache.

Warning

Azure API Management AI gateway: The AI gateway is part of API Management, not a separate offering, and availability varies by service tier. The unified model API and the AI gateway in Microsoft Foundry integration are in preview (as of 2026-09-30).

Governance posture

Reviewed 2026-09-30

YesData residency

Instances deploy to chosen Azure regions; the self-hosted gateway (Developer/Premium) keeps traffic in customer environments, but it still sends config, heartbeat and optional telemetry traffic to Azure.

Source

YesDeployment model

Managed Azure service; the containerized self-hosted gateway enables hybrid and multicloud data planes. (saas, hybrid)

Source

YesAudit logging

Logs prompts, completions and token usage to Azure Monitor / Application Insights, with a built-in token dashboard.

Source

YesSOC 2

Azure is covered by the Azure SOC 2 Type 2 attestation; per-service scope is in the audit report.

Source

YesISO 27001

Azure is covered by the Azure ISO/IEC 27001:2022 certificate; per-service scope is in the certificate.

Source

UnknownISO 42001

ISO 42001 scope for API Management not confirmed in the sources reviewed.

N/AEU AI Act

API gateway infrastructure, not an AI system placed on the market; obligations rest with the deployer and model providers. (role: not-applicable)

YesLicense risk

Proprietary Azure service; policies are APIM-specific, so there is Azure lock-in. (medium)

Source

EU AI Act obligations

Risk tier: Not applicable · as of 2026-08-23

Source-backed information mapped from Azure API Management AI gateway's tracked risk tier — not legal advice. Obligations depend on how your organisation deploys the system; see the full obligation reference and timeline.

This record's EU AI Act risk tier is marked not applicable, so no tier-specific obligations attach. Deployers embedding it in an AI system in scope of the Act should assess that system's own tier.

Explore the category

Compare this tool against the rest of its category and the cloud platform foundation layer.

AI & MCP Gateways

Return to the category grid to compare governance posture across every tracked tool.

Platforms

Review Microsoft Foundry, Amazon Bedrock, and Gemini Enterprise Agent Platform as the foundation layer.

Weekly updates

Track releases, deprecations, license changes, and other market movements.