ORGN Gateway vs Cloud-Native AI Gateways (AWS / Azure / GCP)

June 17, 2026

TL;DR

  • Cloud native gateways offer tight identity and billing hooks, but force you into proprietary model IDs and region hopping data paths. You trade long term provider flexibility for immediate convenience within a single vendor service mesh.
  • While cloud giants promise privacy through contracts, ORGN Gateway shifts the burden of proof to hardware attested environments. Cryptographic verification ensures prompts remain unreadable even to the infrastructure operator, moving beyond mere policy-based trust.
  • Azure default abuse monitoring allows human review of flagged prompts, a critical compliance detail often overlooked. Similarly, AWS geographic profiles do not guarantee single region execution, often moving data between regions for processing.
  • Apigee leads in multi-cloud routing, but its best safety features force data to detour through Google infrastructure. This creates a hidden dependency that can break strict data residency requirements for non GCP workloads.
  • ORGN Gateway decouples governance from the cloud provider, using Intel TDX and NVIDIA GPU attestation to verify execution boundaries. It acts as a neutral enforcement layer that maintains zero data retention across hundreds of open and frontier models.
  • Choosing a gateway is now an infrastructure commitment where Bedrock or Foundry specific schemas make migration expensive. A provider agnostic approach using standard API formats prevents the technical debt of rewriting agent logic when shifting vendors.

What Are Cloud-Native AI Gateways?

Cloud-native AI gateways sit between applications and large language model providers, acting as the control plane for authentication, routing, governance, observability, and policy enforcement across AI workloads. Instead of every application integrating directly with OpenAI, Anthropic, Gemini, or self-hosted models, the gateway centralizes authentication, inspection, routing, logging, rate limiting, and auditing.

Modern AI gateways solve operational problems that traditional API gateways were never designed to address. They manage token-aware rate limiting, semantic caching, model failover, prompt inspection, agent tool governance, and cross-provider routing. In production environments, a single-agent workflow may invoke multiple models across several providers during a single request lifecycle, making the gateway the primary enforcement layer for both infrastructure operations and compliance controls.

Cloud-native AI gateways such as AWS Bedrock, Azure AI Foundry, and GCP Apigee integrate deeply with their respective cloud ecosystems. They inherit native Identity and Access Management (IAM), billing, networking, logging, and policy systems directly from the cloud provider. That integration simplifies operations for organizations already committed to a single cloud environment, but it also increases coupling to provider-specific APIs, model catalogs, routing logic, and governance mechanisms.

Provider-neutral gateways such as ORGN take a different approach. Instead of anchoring governance within a single cloud ecosystem, they operate as an abstraction layer across multiple model providers and infrastructure environments. This becomes important for organizations managing cross-cloud AI deployments, sovereign AI requirements, or confidential inference workloads where hardware-backed isolation and provider portability matter more than native cloud integration.

Why Your LLM Gateway Choice Has Become an Infrastructure and Compliance Decision

AI gateways are no longer optional glue code. According to Gartner's October 2025 Market Guide for AI Gateways, by 2028, 70% of software engineering teams building multimodel applications will use AI gateways to manage reliability and costs, up from 25% in 2025. Gartner also predicts that 40% of enterprise applications will embed task-specific AI agents by the end of 2026. When those agents call LLMs hundreds of times per workflow, the gateway between your application and those models becomes the primary enforcement point for data governance, cost attribution, provider failover, and regulatory compliance.

The EU AI Act entered full enforcement for high-risk systems in August 2026. NIST AI RMF and ISO/IEC 42001 are now procurement-grade requirements. Your gateway choice determines what data leaves your trust boundary, how verifiable that boundary is, and whether you retain provider flexibility when a model is deprecated or a vendor raises prices without warning.

This article examines how AWS Bedrock, Azure AI Foundry, and GCP Apigee function as AI gateways, where each performs well and where each hits its architectural ceiling, then compares all three against ORGN's confidential gateway model across the dimensions that drive enterprise gateway selection.

The Structural Problem With Direct LLM API Integration at Scale

Calling LLM providers directly works until it doesn't. Every provider ships its own authentication scheme, rate-limit format, and model ID structure. A team integrating three providers manages three separate auth flows, three sets of retry logic, and three quota dashboards with no unified cost view. When a provider goes down, requests fail with no automatic fallback. Cost attribution by team, project, or tenant requires custom instrumentation for each provider.

By 2026, most large-scale engineering organizations will route every LLM request through a gateway that handles failover, policy enforcement, cost metering, and audit logging from a single control point. The question isn't whether to use a gateway; it's which category of gateway fits your architecture.

The Trust Model Problem That None of the Cloud Vendors Fully Solve

There is a meaningful architectural difference between policy-based privacy and hardware-enforced privacy. Policy-based privacy means the vendor has agreed not to use your prompts for training, and their access controls prevent unauthorized access. Hardware-enforced privacy means the inference runs inside a hardware-isolated environment whose properties can be cryptographically verified, and no one, including the cloud operator, can access prompt data in readable form during execution.

Article illustration

AWS, Azure, and GCP operate on a policy-based trust model. AWS's documented cross-region inference behavior states directly: "your input prompts and output results might move outside of your source Region during cross-Region inference."

The following example policy grants the required permissions to use the Claude Sonnet 4.5 foundation model with a Geographic cross-Region inference profile for the US, where the source Region is us-east-1, and the destination Regions are us-east-1, us-east-2, and us-west-2:

{ "Version": "2012-10-17", "Statement": [ { "Sid": "GrantGeoCrisInferenceProfileAccess", "Effect": "Allow", "Action": "bedrock:InvokeModel", "Resource": [ "arn:aws:bedrock:us-east-1:<ACCOUNT_ID>:inference-profile/us.anthropic.claude-sonnet-4-5-20250929-v1:0" ] }, { "Sid": "GrantGeoCrisModelAccess", "Effect": "Allow", "Action": "bedrock:InvokeModel", "Resource": [ "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0", "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0", "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0" ], "Condition": { "StringEquals": { "bedrock:InferenceProfileArn": "arn:aws:bedrock:us-east-1:<ACCOUNT_ID>:inference-profile/us.anthropic.claude-sonnet-4-5-20250929-v1:0" } } } ]}

The first statement grants bedrock:InvokeModel API access to the Geographic cross-Region inference profile for requests originating from the requesting Region. The second statement grants bedrock:InvokeModel API access to the foundation model in both the requesting Region and all destination Regions listed in the inference profile.

How AWS Bedrock Functions as an AI Gateway and Where Its Architecture Draws Hard Lines

Bedrock has evolved from a serverless model access layer into a reasonably complete AI platform with AgentCore, Guardrails, and cross-region inference profiles. The capabilities are real, and they're well-suited for AWS-committed teams, but the architectural boundaries are equally real.

Bedrock's Inference Profiles, Cross-Region Routing, and Data Residency Boundaries

Bedrock's geographic cross-region inference keeps data processing within defined geographic boundaries (US, EU, APAC, Japan, Australia) while routing requests across multiple AWS regions for throughput. An inference profile ARN takes the form:

arn:aws:bedrock:us-east-1:<ACCOUNT_ID>:inference-profile/us.anthropic.claude-sonnet-4-5-20250929-v1:0

The IAM policy must grant bedrock:InvokeModel on both the inference profile ARN and each destination region's foundation model ARN. For organizations with SCPs blocking unused regions, every destination region in the cross-region profile must appear in the allow list.

The compliance caveat matters: even geographic profiles don't guarantee single-region prompt execution. Prompts are encrypted in transit to destination regions for processing before returning to the source region. Teams requiring strict single-region execution must implement a deny SCP for geographic and global inference profile ARNs; the default configuration doesn't enforce this.

AgentCore Gateway as an MCP and Tool Orchestration Layer

Amazon Bedrock AgentCore Gateway converts REST APIs, Lambda functions, and Smithy models into MCP-compatible tool servers. The dual-sided auth model covers inbound (custom JWT) and outbound (IAM role assumption, OAuth 2LO for third-party APIs). Interceptors: Lambda functions attached to the request or response side that provide fine-grained access control without touching the underlying MCP servers. Semantic tool search and CloudWatch/CloudTrail integration round out the observability surface.

Article illustration

To create a gateway, use Amazon Cognito for inbound auth using the AWS Boto3:

gateway_client = boto3.client('bedrock-agentcore-control')auth_config = { "customJWTAuthorizer": { "allowedClients": '<cognito_client_id>', # Client MUST match with the ClientId configured in Cognito. "discoveryUrl": '<cognito_oauth_discovery_url>' }}create_response = gateway_client.create_gateway(name='DemoGateway', roleArn = '<IAM Role>' # The IAM Role must have permissions to create/list/get/delete Gateway protocolType='MCP', authorizerType='CUSTOM_JWT', authorizerConfiguration=auth_config, description='Demo AgentCore Gateway')# Values with < > needs to be replaced with real values

Create gateway targets

Create a target for an existing API using the OpenAPI specification with API key as an outbound auth:

# Create outbound credentials provider in AgentCore Identityacps boto3client(service_name"bedrock-agentcore-control")responseacpscreate_api_key_credential_provider(name"APIKey",apiKey"<your secret API key")credentialProviderARN response['credentialProviderArn']# Specify OpenAPI spec file via S3 or inlineopenapi_s3_target_config = { "mcp": { "openApiSchema": { "s3": { "uri": openapi_s3_uri } } }}# API Key credentials provider configurationapi_key_credential_config = [ { "credentialProviderType" : "API_KEY", "credentialProvider": { "apiKeyCredentialProvider": { "credentialParameterName": "api_key", # Replace this with the name of the api key name expected by the respective API provider. For passing token in the header, use "Authorization" "providerArn": credentialProviderARN, "credentialLocation":"QUERY_PARAMETER", # Location of api key. Possible values are "HEADER" and "QUERY_PARAMETER". #"credentialPrefix": " " # Prefix for the token. Valid values are "Basic". Applies only for tokens. } } } ]# Add the OpenAPI target to the gatewaytargetname='DemoOpenAPITarget'response = gateway_client.create_gateway_target( gatewayIdentifier=gatewayID, name=targetname, description='OpenAPI Target with S3Uri using SDK', targetConfiguration=openapi_s3_target_config, credentialProviderConfigurations=api_key_credential_config)

Bedrock's Guardrails, Vendor Lock-In Surface, and Real Cost of AWS-Only Architecture

Bedrock Guardrails provides PII redaction and content filtering across all models in the platform. The Reserved Service Tier (currently available only for Claude Sonnet 4.5) offers pre-reserved token capacity with a 99.5% uptime target and automatic overflow to on-demand.

The lock-in surface is real: Bedrock-specific model IDs, AgentCore agent definitions, Guardrails configuration schemas, and IAM policies tuned to Bedrock resource ARNs. A team that starts on Bedrock and later needs to route traffic to a self-hosted model on GKE, or to a model not yet in Bedrock's catalog, faces a full re-architecture of its routing layer, auth model, and observability stack. Provider-agnostic gateways that accept BYOK and route to any OpenAI-compatible endpoint handle the same transition with a configuration change.

Azure AI Foundry's Model Router and Gateway Capabilities: Deep Integration With Deep Constraints

Azure AI Foundry combines model deployment, agent service, and a model router into a single Azure-native platform. The native Entra ID and APIM integration makes it attractive for organizations already running on Azure, but the data-handling boundaries warrant close reading before treating Foundry as a complete compliance solution.

Model Router Architecture: Quality, Cost, and Balanced Routing Modes and Their Limits

Azure Foundry's model router supports three routing modes: Quality (prefers the highest-capability model), Cost (prefers cheaper models while maintaining acceptable quality), and Balanced (the default). The November 2025 release added support for 18 models, including DeepSeek, Grok, Llama 4, and Claude variants from Anthropic.

The model router is packaged as a single Foundry model that you deploy. To deploy programmatically without the portal, use the REST API examples in the following deployment sections. By default, the model router deploys with the Balanced routing mode and routes across the full supported model set. You only need to change the routing mode or select a model subset when you want custom routing behavior.

Article illustration

Go to the Microsoft Foundry portal and navigate to the model catalog. Find model-router in the Models list and select it. Choose Default settings for the Balanced routing mode and route between all supported models.

Before you run the REST examples, sign in with Azure CLI and save a management-plane bearer token as AZURE_AI_AUTH_TOKEN.

export AZURE_AI_AUTH_TOKEN=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv)

Deploy a model router programmatically with the Azure Management REST API. The following example creates a default deployment and relies on the built-in Balanced routing mode and fully supported model set.

curl -X PUT "https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.CognitiveServices/accounts/my-foundry-account/deployments/model-router-deployment?api-version=2025-10-01-preview" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $AZURE_AI_AUTH_TOKEN" \ -d '{ "sku": {"name": "GlobalStandard", "capacity": 10}, "properties": { "model": {"format": "OpenAI", "name": "model-router", "version": "2025-11-18"} }}'

The architectural constraint that matters in production: the context window limit for a given model router deployment is governed by the smallest model in the selected subset. A prompt larger than the smallest model's context window only succeeds if it happens to route to a larger model, and routing is non-deterministic from the application's perspective. Teams with variable prompt lengths need to either constrain prompt size, use the model subset feature to exclude small-context models, or accept that some requests will fail intermittently.

Claude models require separate pre-deployment from the model catalog before they become available to the router. Unlike Azure-native models, they don't become available automatically upon router configuration. Adding a new Claude version to the routing pool requires a separate deployment step.

Azure APIM as the AI Gateway Layer and the BYOM Compliance Gap

Azure Foundry's AI gateway layer for MCP tool governance routes through Azure API Management (APIM) and is still in preview. The routing applies only at tool creation: existing tools aren't automatically enrolled. Governance is activated at the Foundry resource level, and applying APIM policies requires the API Management Service Contributor or Owner role, not just Foundry-level permissions.

The Bring Your Own Model (BYOM) pattern allows Foundry Agent Service to connect to models behind APIM or non-Azure gateways. Microsoft's documentation is explicit about what BYOM means for compliance: "You are responsible for ensuring that use of the BYOM model complies with your data handling requirements." For BYOM models, Microsoft makes no data residency or processing guarantees. A team connecting a self-hosted DeepSeek endpoint or a non-Azure third-party model through Foundry's BYOM is solely responsible for validating how prompt data is handled at that endpoint.

Azure's Data Processing Scope and the Abuse Monitoring Carve-Out

Most comparison articles stop at "Azure doesn't use your prompts for training." The full picture in Microsoft's documentation is more nuanced. Azure Direct Models run abuse monitoring by default. When active, prompts and completions are stored in a geographically scoped data store and human reviewers can access content flagged by the automated system.

Turning off abuse monitoring requires explicit approval from Microsoft and isn't available on request. Teams can verify its status via Azure CLI by checking whether the ContentLogging attribute reads false on the Azure subscription hosting the Foundry resource; if the attribute isn't present, monitoring is still on.

There are two ways for customers, once approved, to turn off abuse monitoring to verify that data storage for abuse monitoring has been turned off in their approved Azure subscription:

  • Using the Azure portal, or
  • Azure CLI (or any management API).

Note:

The value of "false" for the "ContentLogging" attribute appears only if data storage for abuse monitoring is turned off. Otherwise, this property will not appear in either Azure portal or Azure CLI's output.

Prerequisites

  • Sign in to Azure
  • Select the Azure Subscription that hosts the Foundry resource.
  • Navigate to the Overview page of the Foundry resource.

Using the Azure portal

  1. Go to the resource Overview page
  2. Click on the JSON view link in the top right corner, as shown in the image below.
Article illustration

There will be a value in the Capabilities list called "ContentLogging" that will appear and be set to FALSE when abuse monitoring logging is off.

{ "name":"ContentLogging", "value":"false"}

Using the Azure CLI

Execute the following command in Azure CLI to see the same JSON data as shown in the Azure portal above.

az cognitiveservices account show -n resource\_name -g resource \_group

GCP Apigee as an LLM Gateway: API Management Strengths, Semantic Caching, and the Gemini Coupling Problem

Apigee is a mature API management platform with a decade of enterprise deployment history. Its AI-specific policy extensions are genuinely useful, and its multi-cloud routing support is broader than Bedrock's. The architectural tension lies between its multi-cloud claims and the GCP dependencies inherent in its deepest AI governance features.

Apigee's Policy Engine for AI Traffic: Token Quotas, Semantic Caching, and Model Armor Integration

Apigee's AI-specific policy suite covers meaningful ground. The LLMTokenQuota policy enforces token-based rate limiting using o200k_base encoding, while the PromptTokenLimit policy blocks requests that exceed per-prompt token thresholds before they reach the model, returning HTTP 429. SemanticCacheLookup and SemanticCachePopulate use Vertex AI Vector Search and the Embeddings API to cache and retrieve responses to semantically similar prompts: which reduces both latency and per-token cost for high-repetition workloads.

Model Armor implements LLM-specific enforcement in Apigee's XML-based policy language: prompt injection detection, jailbreak mitigation, PII redaction, and OWASP LLM Top 10 coverage at the proxy layer. The policy logs metadata for each evaluation, including triggered filters, feeding into Apigee's analytics and alerting pipelines.

Article illustration

To create a Model Armor template for sanitizing user prompts and LLM responses:

gcloud model-armor templates create --location $LOCATION TEMPLATE_NAME --rai-settings-filters='[{ "filterType":"HATE_SPEECH", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "HARASSMENT", "confidenceLevel": "MEDIUM_AND_ABOVE" },{ "filterType": "SEXUALLY_EXPLICIT", "confidenceLevel": "MEDIUM_AND_ABOVE" }]' --basic-config-filter-enforcement=enabled --pi-and-jailbreak-filter-settings-enforcement=enabled --pi-and-jailbreak-filter-settings-confidence-level=LOW_AND_ABOVE --malicious-uri-filter-settings-enforcement=enabled --template-metadata-custom-llm-response-safety-error-code=798 --template-metadata-custom-llm-response-safety-error-message="test template llm response evaluation failed" --template-metadata-custom-prompt-safety-error-code=799 --template-metadata-custom-prompt-safety-error-message="test template prompt evaluation failed" --template-metadata-ignore-partial-invocation-failures --template-metadata-log-operations --template-metadata-log-sanitize-operations

Substitute TEMPLATE_NAME with the name of the template you want to create. The template name can have letters, digits, or hyphens. It must not exceed 63 characters and cannot have spaces or start with a hyphen.

This command creates a Model Armor template that uses all the available Model Armor filters and settings. To learn more about the available filters, see Model Armor filters.

Confirm that the Model Armor template is created:

gcloud model-armor templates describe TEMPLATE_NAME --location $LOCATION

Replace TEMPLATE_NAME with the name of the template you created in the previous step.

Here is the dependency that teams need to account for before designing around it: Model Armor runs on Google Cloud infrastructure. Applying it to traffic destined for OpenAI, Anthropic Direct, or self-hosted models requires routing those calls through Apigee, which adds a GCP processing step to what might otherwise be a cloud-agnostic architecture. A team with data residency requirements that exclude GCP cannot use Model Armor for non-Vertex endpoints.

Apigee's MCP Proxy Support and the ApigeeLLM Wrapper Scope Limitation

Apigee's MCP proxy support, currently in preview, converts existing APIs into MCP tool servers by creating an Apigee proxy with /mcp as the base path and an OpenAPI specification as the tool definition source. Apigee manages the MCP servers, transcoding, and protocol handling: the client code doesn't need a separately deployed MCP server. The proxies register automatically in Apigee API Hub, and the standard Apigee policy set applies to MCP traffic, including quota limits and identity controls.

The documented scope limit is that the ApigeeLLM wrapper in ADK is currently designed for use with Vertex AI and the Gemini API's generateContent interface. Teams routing Anthropic, OpenAI, or self-hosted model calls through Apigee must write their own routing logic; the native ADK integration doesn't provide full-depth governance for non-Gemini models. GCP has indicated broader support is planned, but at the time of writing, there is a real gap for multi-provider agentic architectures.

ORGN's Confidential Gateway Architecture: Hardware-Enforced Privacy vs Policy-Based Trust

ORGN's architectural differentiation from the three cloud-native gateways lies not in routing features or policy syntax, but in the trust model. Where AWS, Azure, and GCP rely on access controls and contractual commitments, ORGN anchors trust in hardware attestation.

The TEE Data Plane: Intel TDX, NVIDIA GPU Attestation, and What "Verifiable Privacy" Actually Means

ORGN's architecture separates three layers: a router control plane that handles authentication, validates model availability, and enforces security constraints without touching prompt data; a TEE data plane where actual inference runs inside hardware-backed Trusted Execution Environments; and an attestation layer that generates cryptographic proof per request.

The router authenticates and forwards requests but never accesses plaintext prompt or response content. Inference runs inside Intel TDX confidential virtual machines with NVIDIA H100/H200 GPU attestation via Phala's confidential AI cloud. Each request generates attestation artifacts proving: the specified model ran inside a valid TEE, the execution environment matched expected measurements, and the response was generated within the trusted boundary. Those artifacts return with the response, enabling independent verification.

Contrast this with AWS's geographic inference profiles (a routing boundary enforced by policy and SCPs) and Azure's abuse-monitoring default (prompts stored with human-review access): both are policy commitments by the cloud operator. ORGN's attestation is a cryptographic fact that can be independently verified against the hardware measurements. The performance cost of running inside a TEE is approximately 0.5–5% overhead, making it viable for production inference workloads.

Zero Data Retention Models, User-Controlled Model Selection, and the No-Substitution Guarantee

Beyond TEE-backed inference, ORGN offers a ZDR model path in which inference runs on Vercel infrastructure under zero-data-retention provider agreements. Neither Vercel nor the underlying model provider stores or logs prompts and responses.

One guarantee worth noting: ORGN does not perform automatic model selection or dynamic routing. The model specified in the request is the model executed, with no substitution. For compliance workflows mapping specific requests to specific model versions, determinism matters.

ORGN's catalog includes DeepSeek 3.2, Qwen3, GPT-OSS-120B, GLM 4.6, and additional models in progress, all accessible through an OpenAI-compatible API. Migrating from direct OpenAI SDK usage means changing a base URL and API key, nothing else.

Step 1: To test this, you need to create an API key from the ORGN Dashboard:

Your API key will be shown once. Copy it and store it securely.

Step 2: ORGN uses Bearer token authentication. Send your API key in the Authorization header.

Required headers

Authorization: Bearer YOUR_API_KEYContent-Type: application/json

Step 3: Make Your First Request

ORGN is OpenAI-compatible. You explicitly choose the model in your request.

Example: Chat completion (curl)

curl -X POST https://api.gateway.orgn.com/v1/chat/completions \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "phala/gemma-3-27b-it", "messages": [ { "role": "user", "content": "Hello" } ] }'

If the request is successful, you’ll receive a model response along with metadata tied to that specific request.

Step 4: View and Verify the Request

After sending a request, you can inspect it directly in the ORGN dashboard.

The ORGN + Phala Confidential AI Cloud: Architecture, Token Throughput, and Production Scale Evidence

The Phala partnership underpins ORGN's TEE infrastructure. Phala operates GPU TEEs using Intel TDX and AMD SEV, keeping prompts, outputs, and model weights encrypted during execution, with every workload generating attestation verifiable via Phala's Trust Center.

The production-scale evidence addresses the standard objection that confidential computing is a research-grade feature: Phala has processed over 3.05 billion LLM tokens in a single day across partners. In December 2025, ORGN announced a partnership with AILO AI and MBK Holding to purpose-build Qatar's and Europe's first Confidential AI Factories: infrastructure designed for hardware-encrypted AI inference at a national scale.

Side-by-Side Comparison Across Architectural Dimensions That Actually Determine Gateway Selection

The following comparison covers the dimensions that platform engineers and security architects care about when making gateway decisions

Dimension

AWS Bedrock

Azure AI Foundry

GCP Apigee

ORGN

Trust Model

Contractual: Relies on IAM policies and legal agreements.

Contractual: Policy-based; includes default abuse monitoring.

Contractual: Native for Vertex; customer-managed for others.

Hardware-Enforced: Cryptographic attestation via TEEs.

Data Privacy & Verification

Encrypted in transit; cross-region movement allowed by default.

Prompts stored for human review unless monitoring is disabled.

Safety features require data to transit Google infrastructure.

Hardware-isolated; plaintext data is invisible to the operator.

Audit Mechanism

CloudTrail logs with inference region metadata.

Azure Monitor and CLI-verifiable logging status.

Apigee Analytics and standard Cloud Logging.

Per-request cryptographic receipts for hardware proof.

Multi-Cloud Routing

Limited to AWS-hosted models (Anthropic, Meta, etc.).

Azure-only router; external models lack native governance.

Broadest cloud support; can route to OpenAI and Anthropic.

Provider-agnostic; unified endpoint for hundreds of models.

Core Dependency

Deeply coupled with AWS IAM and Account-level SCPs.

Tied to Azure Entra ID and APIM structures.

Integrated with Vertex AI Search and GCP KMS.

Infrastructure-neutral; runs on Phala Confidential Cloud.

Vendor Lock-in Surface

High: Proprietary model IDs and Guardrail schemas.

High: XML-based APIM policies and Entra ID tokens.

Medium: XML policy bundles and Gemini-specific wrappers.

Low: OpenAI-compatible API; standard SDK support.

Migration Effort

Requires full rewrite of auth, routing, and agent logic.

Significant re-architecture of project and hub structures.

High effort to move XML policy logic to other systems.

Minimal; involves changing the base URL and API key.

How Teams Actually Deploy These Gateways in Production

Different gateway architectures solve different operational problems. Some teams optimize for native cloud integration and centralized IAM. Others need multi-provider routing, confidential inference, or audit-ready prompt isolation. The deployment pattern usually depends less on feature checklists and more on where prompts travel, how models are governed, and whether requests must stay inside a verifiable trust boundary.

Example: Using Bedrock as the Primary Gateway Inside AWS

# bedrock-routing-policy.yamlroutes: - match: team: internal-platform provider: bedrock model: anthropic.claude-sonnet-4 - match: workload: support-chatbot provider: bedrock model: amazon.nova-profallback: provider: azure-foundry model: gpt-4.1

A deployment like this works well for organizations already standardized on AWS networking, IAM, CloudTrail, and SCP enforcement. AgentCore can expose internal APIs as MCP-compatible tools, while Bedrock centralizes quota management and model access through AWS-native identity controls. The limitation appears once teams need to route traffic outside AWS-managed models or prove where prompts were processed during inference.

Example: Routing Regulated Workloads Through ORGN

from openai import OpenAIclient = OpenAI( api_key=os.environ["ORGN_API_KEY"], base_url="https://api.gateway.orgn.com/v1")response = client.chat.completions.create( model="deepseek-r1-confidential", messages=[ { "role": "user", "content": "Analyze this patient discharge summary" } ])print(response)

Teams handling regulated prompts often entirely separate confidential inference from standard workloads. In this setup, requests containing legal documents, patient records, financial analyses, or internal strategy data are routed through ORGN's confidential inference layer rather than directly through a cloud-native provider gateway. The request executes inside a Trusted Execution Environment (TEE), and the response includes attestation metadata that can be forwarded into audit pipelines or compliance evidence stores.

Example: Combining ORGN With Existing Gateway Infrastructure

# llm-router.yamlroutes: - match: sensitivity: high target: provider: ORGN model: qwen3-confidential - match: sensitivity: standard target: provider: bedrock model: anthropic.claude-sonnet-4telemetry: exporter: otlp endpoint: tempo.monitoring.svc.cluster.local

Some organizations keep their existing Bedrock, Foundry, LiteLLM, or Portkey deployment for standard application traffic while routing sensitive requests through ORGN separately. In practice, this usually means adding ORGN as another provider target inside the existing routing layer rather than replacing the entire gateway stack. The attestation receipts returned by ORGN can then feed into observability or compliance systems independently from the cloud-native gateway logs.

Conclusion

AWS Bedrock, Azure AI Foundry, and GCP Apigee are capable gateways for teams whose AI workloads live inside a single cloud vendor's ecosystem. Bedrock offers tight IAM integration and AgentCore's depth in tool orchestration. Foundry provides Azure-committed teams with a model router that supports a broad range of models and APIM-backed governance. Apigee brings the strongest multi-cloud routing config support and a mature policy engine, with semantic caching and Model Armor as genuine additions for GCP-native workloads.

Each makes credible contractual commitments about data handling, but all three commitments rely on access controls, policy enforcement, and organizational trust in the cloud operator. ORGN operates on a different trust model: hardware-isolated inference with per-request cryptographic attestation. The trust isn't contractual; it's verifiable. For platform engineers and security architects deciding between these options, the deciding variable isn't features; it's whether your compliance posture, threat model, and provider diversity requirements fit within one cloud vendor's perimeter, or require something the cloud-native gateway was never designed to provide.

FAQs

1. Can ORGN Be Used as a Drop-In Replacement for AWS Bedrock or Azure AI Foundry If My Application Already Uses an OpenAI-Compatible SDK?

Yes. ORGN exposes an OpenAI-compatible API, so applications that already use the OpenAI Python SDK or the Vercel AI SDK can switch by updating the base URL and API key. No new SDK installation or request format changes are required. ORGN documentation covers the migration path in its quickstart guide.

2. How Does AWS Bedrock's Geographic Cross-Region Inference Handle Data Residency for EU-Regulated Workloads, and What Are Its Documented Exceptions?

AWS's geographic profiles keep data processing within specified regions (the EU in this case), but AWS explicitly states that prompts and outputs may travel between regions within that geography for inference. Data isn't stored in destination regions, but it does transit them. For strict single-region residency under GDPR or national data laws, teams must implement deny SCPs to block geographic inference profiles entirely.

3. What Is the Difference Between Zero Data Retention and Confidential Computing in the Context of LLM Gateways, and When Does Each Provide Sufficient Protection?

ZDR means the provider agrees not to store or log your prompts. Confidential computing means inference runs inside a hardware-isolated environment where even the infrastructure operator cannot access data in use. ZDR is a contractual guarantee; confidential computing is a hardware property. For most commercial workloads, ZDR is sufficient.

4. Does GCP Apigee's Model Armor Policy Work for Non-Vertex AI LLM Endpoints, and What GCP Infrastructure Does It Depend On?

Model Armor supports multiple LLM providers, including Anthropic, OpenAI, and self-hosted models when routed through Apigee. However, Model Armor's enforcement engine runs on Google Cloud infrastructure: enforcing it on non-GCP model endpoints requires routing those calls through Apigee first, adding GCP as a processing dependency.