Confidential compute

ColdVault Platform is a shared confidential-inference gateway. In Vault Mode (ColdVault’s default; the alternative, Partner Mode, routes the request to an outside provider instead — see below), every request runs end-to-end on hardware-encrypted infrastructure — plaintext exists only inside the silicon, never in RAM, on the bus, or on the wire. The full request path:

The path is built so plaintext never leaves the encrypted enclave (the hardware-protected region where data can exist in the clear). The public endpoint sits behind an L4 passthrough load balancer that forwards only encrypted bytes — it holds no Transport Layer Security (TLS) key, parses no HTTP, and sees no plaintext. TLS terminates inside an AMD SEV-SNP (Secure Encrypted Virtualization–Secure Nested Paging) Confidential Virtual Machine (CVM), in an nginx process in encrypted memory, with the certificate pulled from Secret Manager at boot. SEV-SNP encrypts the VM’s memory and CPU register state (building on SEV-ES, the earlier AMD layer that also encrypts the CPU registers) in hardware with a key the hypervisor cannot see, and adds integrity protection and hardware attestation — a hardware-signed proof of exactly which genuine chip and firmware are running — on top, so no host operator or co-tenant (another customer’s VM sharing the same physical host) can read or tamper with it.

From the inference gateway, Vault Mode runs the model on self-hosted, confidential GPU clusters where NVIDIA Confidential Computing (NVIDIA-CC) keeps weights and activations (the model’s learned parameters and the intermediate values it computes as it runs) encrypted outside the GPU silicon and encrypts the CPU↔GPU link — matching what SEV-SNP provides on the CPU side. The only place customer plaintext ever exists is inside the CPU and GPU silicon themselves. The gateway reaches those clusters over a mutually-attested (each end proves its genuine hardware to the other) TLS 1.3 channel — it verifies each GPU node’s SEV-SNP and NVIDIA-CC attestation before sending any data, and both ends generate the channel keys inside the enclave and bind them to that attestation, so the private key never leaves confidential memory and there is no external certificate authority to trust. The deep-dives below open up how the AMD and NVIDIA chips each encrypt their part.

The gateway fleet runs on a Managed Instance Group (MIG) of Confidential VMs that autoscales under load and auto-heals any VM that fails health checks; rolling updates deploy one VM at a time, so a release has no downtime. Certificate rotation is automated end-to-end — a scheduled renewal over ACME (Automatic Certificate Management Environment — the protocol Let’s Encrypt uses) using a DNS-01 challenge (which proves domain control via a DNS record) writes the new cert to Secret Manager and an event-driven workflow rolls the MIG so every VM picks it up at boot, no human in the loop.

Verify the confidential VM

ColdVault’s confidential boundary is not “trust us” — it is verifiable. An authenticated endpoint takes a client nonce (a one-time random value the client supplies, so the chip cannot return an old, pre-recorded proof) and returns a fresh AMD SEV-SNP attestation report bound to that nonce, read directly from the SEV-SNP hardware inside the Confidential VM. The platform verifies the report’s certificate chain — AMD VCEK → ASK → ARK (Versioned Chip Endorsement Key → AMD SEV Key → AMD Root Key — the chain tracing the chip’s signature back to AMD’s root) — and the Trusted Computing Base (TCB) version locally, inside the enclave, and fails closed — it refuses to serve the request — if verification does not pass.

Beyond the platform’s own verdict, the endpoint returns the raw, nonce-bound evidence (the report plus the AMD certificate chain) so a third party can verify it independently and offline — no trust in ColdVault, no call back to any ColdVault or cloud endpoint:

go-sev-guest check -in evidence.bin -inform proto -network=false
# exit 0  → genuine AMD SEV-SNP silicon, bound to your nonce, verified offline against AMD’s root

A short guide at coldvault.ai/verify.html walks through it end to end — exit 0 for genuine silicon bound to your nonce, non-zero for anything else.

This proves a genuine, fresh AMD SEV-SNP processor produced a report bound to your nonce — a silicon-level proof. It is not a transport-channel binding (it does not prove the encrypted connection itself ends inside that chip) or a workload-measurement pin (it does not pin which exact code or container image is running).

Two inference modes, one API

One API, two inference modes, declared per request. Vault Mode is the default described above — end-to-end on hardware-encrypted infrastructure. Partner Mode is the option: every other proprietary frontier model is available through the same API, routed to its provider, where inference runs in plaintext at the provider — for the cases where the customer is fine with the provider seeing the payload, or only that provider has the capability. Providers are not named here by design.