Private AI for Confidentiality: A Practical Guide for Engineering Teams

TL;DR

• Treating zero-data retention as the end of the security discussion leaves a gap during computation, when prompts, source code, and terminal context exist in plaintext memory. If your compliance requirements demand independently verifiable confidentiality, contractual guarantees alone won't satisfy procurement or audit reviews.
• Running models inside your own VPC changes who owns the infrastructure, but it doesn't eliminate privileged access to memory during inference. Choosing between self-hosted deployments and Trusted Execution Environments is ultimately a decision about who can inspect data in use, not where the servers are located.
• Mixing up sandbox attestation with inference attestation creates compliance evidence that answers the wrong question. One proves the development environment launched inside genuine TDX hardware, while the other proves a specific AI request executed inside a verified confidential environment.
• Selecting a model becomes a security architecture decision when different inference routes provide different assurance levels. Wider model availability comes through contractual zero-retention paths, whereas hardware-backed TEE routes trade model choice for cryptographically verifiable execution that auditors can independently validate.
• Expecting a confidential AI platform to eliminate every security risk leads to deployment mistakes and unrealistic compliance claims. Hardware isolation protects data during execution, but compromised endpoints, leaked credentials, unsafe prompts, and post-response handling remain the organization's responsibility.

Why Policy-Based AI Confidentiality Fails at the Code Layer

Every AI IDE extension that sends your prompt to vendor infrastructure decrypts your context outside your security perimeter. That is the gap TLS was never designed to close.

The security industry has three distinct categories for data protection: at rest, in transit, and in use. Disk encryption covers the first. TLS covers the second. The third, data in use (computation actively happening in memory), went largely unaddressed until the Confidential Computing Consortium formalized the problem in 2019. That formalization brought founding members including Intel, Microsoft, and Google under one effort specifically to address what happens when a workload is running: the moment a prompt, a file diff, a terminal session, or an agent tool call is being actively processed, the data is decrypted inside a conventional VM and visible to the host OS, the hypervisor, and any privileged path into the underlying infrastructure.

In a recent r/AskNetsec discussion, a security team described a developer accidentally pasting an entire .env file into ChatGPT, concluding that acceptable-use policies and blocking public AI tools didn't solve the problem because they offered no technical control over what left the browser. The incident reinforced a common enterprise concern: policy and contractual assurances reduce risk, but they don't provide verifiable enforcement when sensitive data is in use.

The obvious enterprise response has been to ban public AI tools or negotiate zero-data-retention agreements with providers. Both approaches share a structural weakness: they're contractual, not technical. A ZDR agreement tells you a vendor won't retain your prompts. It gives you no mechanism to verify that claim after the fact. For teams working on defense contracts, regulated financial workflows, or classified infrastructure, "trust the ToS" doesn't hold up in a procurement questionnaire or a third-party audit.

This guide walks through the architecture decisions behind hardware-enforced AI confidentiality: from how Intel TDX Trust Domains differ from standard VMs, through how ORGN's inference gateway routes requests across two distinct assurance tiers, and where attestation receipts fit into compliance documentation. The specific architecture we worked with is a cloud coding session in a TDX sandbox, with inference routed through ORGN Gateway and verification evidence surfaced in ORGN Scanner.

What "Private AI" Means Across the Stack

Private AI means different things depending on which layer of the stack you're talking about. Getting the terminology wrong in a security review produces gaps that no policy can close.

Data-in-Use vs. The Two Categories It Isn't

Self-hosted deployments, on-premises clusters, and VPC-isolated inference environments all address data residency. They control where your data lives and who owns the infrastructure. What they don't address: a conventional VM host still has privileged access to memory while code is running inside it.

Whoever controls the host OS on your on-premises inference box can, in principle, read the model's inputs and outputs in memory. The CCC's definition makes this explicit: confidential computing protects data in use by performing computation in a hardware-based, attested Trusted Execution Environment. The "hardware-based, attested" clause is doing real work in that sentence. Disk encryption and network isolation don't satisfy it.

TEE vs. Self-Hosted: Two Architectures With Different Assurance Properties

This diagram highlights where the trust boundary exists in each deployment model. A self-hosted deployment removes the cloud provider from the equation but still requires trusting whoever manages the host, while a TEE-based deployment moves that trust into hardware so even the host operating system and hypervisor cannot inspect data while the model is running.

A self-hosted model like Ollama or vLLM inside your VPC removes the cloud vendor from the trust chain, which is meaningful. But it transfers the trust problem to whoever administers the host. A TEE deployment isolates computation from the host at the hardware level, so the hypervisor and OS genuinely can't inspect memory contents, regardless of who runs the infrastructure.

The evolution from Intel SGX to TDX matters here. SGX required application rewrites and had roughly 1 GB of usable protected memory, which ruled out any serious LLM inference. TDX secures an entire virtual machine without code modifications, making it operationally viable for the kind of workloads that matter in development environments. The tradeoff is that the model catalog inside TEE paths is narrower than that of frontier providers, and attestation verification adds per-request overhead.

How Intel TDX Trust Domains Differ From Standard VMs

A Trust Domain is a VM with hardware-encrypted memory that the hypervisor and host OS can't read, verified by Intel silicon rather than by software controls. The attestation report a TDX sandbox produces contains four fields: the TDX quote (hardware-signed evidence from Intel), a measured launch digest that proves the software image hasn't been tampered with, a sandbox ID that binds the report to a specific execution context, and an issued timestamp. Critically, as noted in ORGN's platform trust documentation, TDX uses Trust Domains, not SGX enclaves. These terms aren't interchangeable in security reviews, and conflating them signals to a security architect that your team hasn't read the hardware specification.

Confidential AI Development Environments: What Changes for Engineers

The threat model for an AI IDE differs from that for an AI inference API. Both matter, and they need separate controls.

The Threat Model Inside a Conventional AI IDE

A typical AI coding extension sends your prompt context to the vendor's infrastructure to generate completions: open files, terminal history, diff state, and project structure. Enterprise agreements and "no training" policies reduce what the vendor does with that data after receipt. They don't change where the inference computation happens or who has privileged access to it. In 2023, Samsung engineers pasting sensitive semiconductor code into ChatGPT triggered what became a widely cited internal security incident, and the structural problem that incident exposed persists, whether the tool you're using is a consumer product or an enterprise-licensed API with contractual zero-retention.

The diff shows that AnalyticsChart.tsx is modified across 4 files, with the left pane showing the deleted version and the right pane showing the replacement. The header reads "Session Changes — 4 files +9 -12." Nothing is committed; the developer reviews every line before promoting. The split-mode toggle is visible in the top-right.

How a Confidential Development Environment Changes the Trust Boundary

ORGN CDE flips the location of the trust boundary, i.e., when a developer opens a cloud worktree, the repository branch runs inside a TDX Trust Domain rather than on general cloud infrastructure. File diffs, terminal sessions, branch state, and project context stay inside that hardware-isolated sandbox; they don't transit through ORGN's control plane in plaintext. The first SSH attach to a cloud worktree takes 30–60 seconds. After that, the TDX attestation report is available directly from the status bar's TDX shield or via the command palette's "Show TDX Sandbox Attestation" command.

This is the complete Studio interface: task backlog on the left (with priority labels like HIGH and URGENT), the chat session in the center showing the active branch (trial-96acac35 branched from main), and the file tree on the right. The status bar at the bottom shows TDX Sandbox in green, the live signal that the session is running inside hardware-isolated compute.

The report proves one specific thing: that the runtime environment is genuine Intel TDX hardware running an untampered measured image. It doesn't prove that a specific inference call ran in a TEE. Those are separate artifacts: sandbox attestation covers the development runtime; inference receipts cover individual model calls. Conflating them produces exactly the kind of incomplete security claim that fails a procurement audit.

Origin Agent: AI Assistance Inside the Sandbox Boundary

Origin Agent is CDE's in-IDE assistant. File edits and terminal commands from tool calls execute inside the cloud sandbox, not on external infrastructure. The model selection at inference time determines the assurance tier: TEE models produce per-request cryptographic attestation receipts that are verifiable against Intel and NVIDIA public PKIs; ZDR models provide the broadest frontier catalog with contractual zero retention and no hardware proof. Both route through ORGN Gateway, which logs only non-sensitive operational metadata (model ID, token counts, latency, attestation status) and never logs prompt contents or outputs.

For teams building their own tooling on top of the same inference layer Origin Agent uses, the SDK's `streamText` call routes directly through a confidential model without buffering the full response:

import { createOLLM } from '@ofoundation/ollm';
import { streamText } from 'ai';

const ollm = createOLLM({ apiKey: 'YOUR_OLLM_API_KEY' });

const result = streamText({
  model: ollm.chatModel('near/GLM-4.7'),
  prompt: 'Summarize this function and flag any obvious memory risks.',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

The model ID `near/GLM-4.7` routes to NEAR AI infrastructure inside a TEE. Swapping it for a ZDR frontier model ID changes the execution path and drops the attestation receipt, nothing else in the call changes.

Bash commands Origin Agent runs no longer have to block the chat, send a command to the background, and keep working while it finishes. Commands still running in the foreground show the full inline orchestration card in chat, so you can follow command and subagent progress without switching to a separate terminal tab.

Confidential Inference: How ORGN Gateway Routes AI Workloads

The routing decision in ORGN Gateway isn't a UX choice. It's a security architecture decision that needs to be made before the tool goes into production.

The Two-Path Model and When Each Applies

ORGN Gateway is OpenAI-compatible, with a base URL at https://api.gateway.orgn.com/v1. Model IDs use underscores, not slashes: phala_deepseek_r1 for a TEE path on Phala Network infrastructure, near_qwen3_30b for a TEE path on NEAR AI. ZDR paths go through Vercel AI Gateway to frontier providers.

The following TEE request, taken directly from ORGN's Gateway quickstart, shows what a verified inference call looks like at the API level:

curl -X POST https://api.gateway.orgn.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "phala_deepseek_r1",
    "messages": [
      { "role": "user", "content": "Hello" }
    ]
  }'

After sending a TEE request, ORGN Scanner surfaces the attestation result. The status field returns one of three values, as documented in Gateway's quickstart:

Scanner shows attestation metadata and cryptographic hashes. It doesn't display prompt contents or model outputs. That separation is intentional, as explicitly mentioned in the Gateway security model.

Teams integrating Gateway into an existing TypeScript codebase don't need to manage raw HTTP calls. The @orgn/gateway package exposes an OpenAI-compatible provider for the Vercel AI SDK, wrapping the same confidential inference paths with a typed provider instance. Setup is two lines:

import { createORGN } from '@orgn/gateway';

const gateway = createORGN({
  apiKey: process.env.ORGN_API_KEY,
});

The API key is available in the Gateway console at gateway.orgn.com and exported as ORGN_API_KEY, a legacy convention from an earlier SDK version; the sk-orgn- key prefix remains unchanged. The provider instance binds to your key and passes every request through Gateway's routing layer; the model ID you pass to gateway.chatModel() determines whether the request hits a TEE or ZDR path.

What Gateway Cannot Protect Against

The scope exclusions in ORGN's security documentation are worth reading as carefully as the guarantees. Gateway doesn't protect against compromised client machines, leaked API keys, data exposed before the request enters the API or after the response returns, or vulnerabilities in model weights. These aren't design gaps. They're explicit shared-responsibility decisions that let security architects reason cleanly about what the tool covers and what their own controls need to cover.

Attestation Receipts as Compliance Artifacts

Per the ORGN trust model, Scanner surfaces per-request operational metadata: request ID, model, provider, token usage, latency, attestation status, and cryptographic hashes, and never prompt contents or outputs. For teams filling out procurement questionnaires or preparing for third-party audits, attestation artifacts are exportable and can be included directly in security documentation packages. The engineering discipline requires that your team know which requests are routed through TEE paths so they can reference the correct receipts. Scanner provides that lookup.

Deploying a Confidential AI Stack: Architecture Decisions and Limits

Getting the shared-responsibility boundary wrong at deployment time produces the same problem as choosing the wrong assurance tier: you don't discover the error until a review surfaces it.

Air-Gap and Private Deployment Considerations

ORGN's enterprise tier supports air-gap and private deployment for classified or restricted environments. The engineering implications are specific: the Gateway API endpoint changes, the model catalog available on TEE paths depends on the deployment configuration, and attestation verification against Intel's public PKI requires network access unless you configure a local verification path.

For teams evaluating offline verification, the attestation artifacts are signed with hardware keys rooted in Intel's PKI, enabling reproducible offline verification using the relevant public certificates. The scanner.orgn.com web interface won't be reachable in a fully air-gapped deployment, which affects how teams access audit trails.

Where the Stack Shares Responsibility

ORGN's platform trust documentation explicitly names the shared-responsibility boundary. ORGN mitigates unauthorized access inside TEE paths, infrastructure-level compromise of Trust Domain memory, unverifiable provider claims through hardware attestation, and insider access to plaintext inference data on attested routes.

The shell prompt (daytona@e6562e08-c9b7-4255-b940-c8b6ad9b8463) confirms execution is scoped to an isolated worktree, not the developer's local machine. The npm i output shows 381 packages installed and 15 vulnerabilities flagged, all of which run inside the confidential sandbox boundary ORGN provisions per project.

ORGN doesn't guarantee protection against compromised client machines, malicious prompts, unsafe model behavior, or data exposed before it enters the stack or after it leaves the stack. The ORGN stack closes the data-in-use gap at the inference and development layers. Endpoint security, secrets management, and model safety controls remain the team's own responsibility.

When Contractual Guarantees Aren't Enough: Choosing the Right Assurance Tier

The choice between TEE and ZDR isn't about how much you trust ORGN; rather, it's more about what your auditor can verify independently.

This decision tree helps determine whether a workload should use a TEE-backed or Zero Data Retention (ZDR) inference path. Rather than choosing based on model preference alone, the flow starts with compliance, audit, and evidence requirements, showing that the required level of verifiability should drive the architecture decision.

TEE vs. ZDR: The Decision Is About Verifiability, Not Privacy Level

Both TEE and ZDR paths in ORGN Gateway are zero-retention: neither stores prompt contents nor model outputs. What separates them is verifiability. TEE paths produce a cryptographic attestation receipt that exists independently of ORGN's own claims and can be verified against Intel's and NVIDIA's public PKI without involving ORGN at all. ZDR paths rely on Vercel's and the upstream provider's adherence to their contractual commitments.

For most regulated workloads, contractual zero retention satisfies procurement sign-off. For defense contexts with CMMC requirements, financial workflows touching material non-public information, or any scenario where a supplier's assertion fails a third-party audit on its own, TEE paths are the right choice. Not because ZDR is weaker in practice, but because the attestation receipt gives the auditor something to verify without contacting the vendor. Pick the assurance tier at system design. Changing it after the tool is embedded in your CI/CD pipeline or agent framework costs significantly more than getting it right the first time.

The Procurement Questionnaire Translates Directly to Gateway Configuration

Enterprise security questionnaires for AI tools ask five questions: data retention, model training on prompts, encryption in use, audit trails, and vendor access to inference data. Each maps to a specific ORGN configuration answer.

• Data retention: zero on both TEE and ZDR paths, per Gateway's security model.
• Model training: none; ORGN doesn't train on prompt or completion data.
• Encryption in use: hardware-enforced on TEE routes (prompts and responses encrypted in memory, inaccessible to host OS and ORGN operators), policy-enforced on ZDR.
• Audit trails: Scanner provides attestation status and operational metadata without surfacing inference content.
• Vendor access to inference data: the Gateway control plane handles routing and metadata; on TEE paths, prompt contents sit outside its access boundary.

The answer to "vendor access to inference data" on a procurement questionnaire doesn't end at the Gateway configuration. It also includes how your application scopes the model's input. A system prompt that constrains the model's reference boundary is a second control layer on top of the TEE path:

import { createORGN } from '@ofoundation/orgn';
import { generateText } from 'ai';

const orgn = createORGN({ apiKey: 'YOUR_ORGN_API_KEY' });

const { text } = await generateText({
  model: orgn.chatModel('near/GLM-4.6'),
  system: 'You are a code review assistant. Reference only the code provided. Do not request, infer, or describe external systems, credentials, or network configuration.',
  prompt: userCode,
});

This doesn't replace the TEE guarantee; prompts still run inside hardware-isolated memory. It prevents the model from producing outputs that inadvertently describe the system architecture that the developer didn't intend to share in the response.

Conclusion: Private AI Confidentiality Is an Architecture Decision, Not a Feature Toggle

The data-in-use gap is the one that TLS and disk encryption have never touched, and it's the gap that matters most for teams whose work can't leave the security perimeter. Picking an AI coding tool without specifying the assurance tier doesn't produce an error. It produces a tool that's live in your development workflow before anyone has checked whether its confidentiality properties hold up in a security review.

This article covered three decisions that determine whether a private AI stack closes the data-in-use gap: the choice between TEE and ZDR at the inference tier, the separation between sandbox attestation and inference receipts (two distinct artifacts that answer different questions for a procurement audit), and the shared-responsibility boundary defining what ORGN's stack handles vs. what the team's own endpoint security and secrets management must cover. Every claim about verifiability in a TEE deployment reduces to whether the attestation receipt exists and whether your team can verify it independently. If the answer to the second part is no, the tool belongs on the ZDR path.

FAQs

1. What's the difference between zero data retention and a confidential AI environment?

Zero data retention is a contractual commitment: the vendor agrees not to store, log, or train on your inference data. A confidential AI environment is a hardware guarantee: your workload runs inside a TEE where memory is encrypted, and the host OS can't inspect it. ZDR tells you what happens after the fact; a confidential environment limits what's accessible during computation. A confidential environment can also be zero-retention, but the two properties aren't the same thing.

2. Can you verify that your AI coding tool isn't sending your code somewhere?

With a conventional AI tool, you can't, short of network traffic inspection on every request. With a TEE-based tool like ORGN CDE, you can fetch a TDX sandbox attestation report directly from the status bar after attaching to a cloud worktree, and verify it against Intel's public PKI independently. TEE inference requests produce separate per-request receipts verifiable in ORGN Scanner. That verification doesn't require trusting the vendor's assurances.

3. Does running AI on-premises mean your code is confidential?

Not by itself. An on-premises deployment removes the cloud vendor from the data path, but whoever administers the host OS on your inference server still has privileged access to memory during computation. Confidentiality in the technical sense requires hardware-enforced isolation: the computation runs inside a Trust Domain where memory is encrypted even from the host. On-premises addresses data residency; it doesn't address data in use.

4. How does Intel TDX attestation prove where an AI workload ran?

TDX attestation works by having the hardware generate a signed quote containing a measured launch digest of the software image and a hardware-signed certificate rooted in Intel's PKI. The digest proves the image running in the Trust Domain hasn't been tampered with; the hardware signature proves it's running on genuine TDX silicon. Both are verifiable against Intel's public certificate chain without contacting the vendor. In ORGN's stack, sandbox attestation covers the runtime environment; Gateway's per-request inference receipts cover individual model calls.