Cohere Hides Patient And Bank Data From Outsiders On Azure

Cohere Model Vault, a new lockbox from AI company Cohere, lets banks and hospitals use its software on Microsoft Azure while keeping patient files and financial records hidden from outsiders.

Key Takeaways

• Cohere said the beta combines dedicated inference with NVIDIA confidential computing on Azure GPUs
• Cohere said its confidential environment spans both CPU and GPU memory, with encrypted traffic between them
• The beta adds remote attestation so customer software can verify hardware and software configuration before sending sensitive data
• Cohere sells enterprise AI models and tools for businesses that want private deployments rather than consumer chat products

Cohere posted the release on X, saying the beta combines dedicated inference with NVIDIA confidential computing on Azure GPUs. Cohere sells enterprise AI models and tools for businesses that want private deployments rather than consumer chat products.

Cohere Model Vault Targets Data In Use

The pitch is narrower than a general model launch, with Cohere trying to solve the part of enterprise AI that lawyers, banks and hospitals worry about most, namely data exposed during inference. Most security stacks already protect data at rest and in transit, but the harder problem is data in use, the short window when a model must read plain text to answer a question or generate code.

Cohere said its confidential environment spans both CPU and GPU memory, with encrypted traffic between them, and that matters for customers deciding whether managed AI can touch clinical notes, financial records or source code.

Before this beta, many firms either kept those workloads off shared AI services or built costly self‑hosted systems to control who could see raw prompts and outputs.

Also Read: OpenAI Locks Down AI Training As Stolen Models Cost Near Zero To Copy

From Private Deployments To Verifiable Isolation

Friday’s beta extends a longer shift in enterprise AI buying. Companies first asked whether a model was accurate enough for internal work.

They now ask who can inspect the prompts, weights and memory during runtime, and that shift has pushed vendors toward dedicated inference, virtual private environments and hardware‑backed isolation.

The new beta adds remote attestation, a cryptographic check that lets customer software verify the hardware and software configuration before sending sensitive data, which matters because “trust us” is no longer enough for regulated sectors.

A bank compliance team can accept encrypted storage on paper and still reject a service if operators retain privileged access while the model runs. Attestation turns that question into a machine check instead of a contract clause.

Enterprise AI Security Gets More Specific

The launch also sharpens Cohere’s position against larger rivals, as the company is not trying to win the broadest chatbot market, but is leaning into private enterprise inference, where procurement teams care less about viral adoption and more about audit trails.

That could make the offering attractive to firms already on Azure that want managed infrastructure without exposing plaintext prompts to cloud operators, and it also puts pressure on rivals to show not only that their models are private by policy, but that the infrastructure can prove it by design.

How Remote Attestation Checks The Hardware

Cohere Model Vault depends on remote attestation, which is a signed hardware report that proves where inference will run.

Before a company sends a prompt, its software checks evidence from NVIDIA GPUs, the host CPU and the surrounding firmware against an approved policy.

If the configuration matches, the prompt is released into the enclave, and if it does not, the request can fail before any sensitive text leaves the customer side.

For a hospital using Microsoft Azure, that is different from ordinary encryption, which protects data on disk and in transit but not once software starts reading it. The practical win is for security and compliance teams, not just machine learning engineers.

A bank can let employees query contracts or code through Cohere Model Vault while narrowing the number of systems and staff that can inspect raw data.

The next thing to watch is whether Cohere extends the same attestation flow to more models and regions, because enterprise buyers usually need consistent controls across multiple jurisdictions before moving high‑value workloads.

Read Next: Software Firm Triples AI Success On 114 Business Tasks