Last updated on September 19th, 2026 at 07:50 am
A quiet crisis is unfolding across most big corporations.
Staff are entering confidential information into public AI tools. Consumers check quotes with refreshable chatbots, and finance teams conduct client projections. Consumers check quotes with refreshable bots, and finance departments run client projections using consumer-grade chatbots. Poor session information is a problem because health workers ask LLMs with real patient context and often don’t even realize it’s the issue.
IT and security staff know. However, as they’ve seen, they haven’t been able to stop it, since blocking it altogether means people find a way around it.
On one side, this is the big challenge of enterprise AI adoption in 2025: Useful enough that people will find a way to use it; on the other, regulated industries can’t afford to be exposed to the show. There has to be a compromise.
This “something” is the infrastructure itself.
Table of Contents
What the Shift from “Public Experiment” to Confidential AI Actually Means
For most companies, the path to AI adoption began with giving a handful of teams access to a publicly available LLM that generated summaries and drafts of their content. It was low-risk because the data it had to transfer was low-sensitivity. It was good enough to foster internal demand.
Then a legal person or maybe a risk compliance person asked the seemingly simple question: what happens if this is utilized for something that really matters?
That’s when the cracks start to show.
Public models send data to third-party infrastructure, even via API. You can’t tell if your prompt was used or not, and you can’t be certain who used it or whether it was used for training. That’s a reality, not just a possibility, in a healthcare system, bank, or government department. This is not an easy-to-break regulatory wall.
Confidential Infrastructure” is a particular technical solution to the problem. It is equivalent to running AI workload applications within hardware-enforced trusted execution environments (TEEs) — memory regions even the cloud administrator can’t reach. The data stays encrypted at rest, in transit, and during processing. And then it can create a cryptographic proof of what software was used, on what hardware, and under what policies? An attestation report.
This is AI infrastructure that a regulator can audit.
Why Traditional Security Wasn’t Enough
Based on the time in which most enterprise security frameworks were created. All of these actually mean something. However, you can’t use them to protect data when a model is running on it.
This is the ‘data-in-use’ exposure that confidential computing addresses. You can discover what is inside a normal computation via a root user on a cloud host or a memory-scraping attack. Inside a TEE, you can’t do that cryptographically.
For those who want to dive deeper into the underlying technology before enterprise strategy, Confidential Computing 101 is a great place to start, and it offers a valuable introduction to TEEs, attestation, and the underlying hardware stack in simpler terms.
What’s Already Shipping – Not Just on Slides
Many of the “Confidential AI” pieces read like a roadmap document. Here is a list of what is currently available and being used in the real world:
Cloud-native confidential compute: Azure’s confidential GPU VMs are a production service, as are Google’s Confidential Space. You can deploy RAG pipelines or use fine-tuned models now, with encrypted memory and attestation built into the infrastructure. This isn’t beta. Enterprises are using it.
High-Throughput Application (HTA) GPU TEEs for high-performance workloads: Most demanding AI tasks require GPUs, and the H100S can run them in a TEE. Early confidential AI pilots were forced to make use of CPU-only TEs – slower and more limited. The restriction is being removed.
Mutual attestation patterns: If you want to share data containing sensitive information with a vendor-provided AI model, you’ll need to verify who you are before you share it. One of the harder parts of confidential AI is verifying the data owner and the AI model. Example model patterns exist, such as this one that Red Hat published. The technical work is in place; it’s not widespread, but it exists.
Governance takes AI products and processes to the next step: enterprise AI governance frameworks. Now, organizations can get AI-specific risk guidance from organizations such as ISACA. Liminal’s enterprise AI governance guide details data exposure and IP leakage, along with compliance controls, in enough detail for regulated industries to use effectively. These aren’t just concepts; they’re checklists tied to tangible controls.
I’ve seen some of this stuff marinating myself, and it’s really been faster than I thought it was going to be moving from the ‘Should we use AI?’ question to the ‘How do we make sure that we can do it safely?’ The audit trail is now the prominent focus.
The Threats That Changed Everything
It’s worth pointing out what scares enterprise CISOs enough to take this seriously.
Valuable injections proved harder to patch than expected, since a model’s behavior can be compromised by malicious injection. Model extraction attacks became a real threat to IP rights (when an adversary queries the model enough to reconstruct it), which is a new IP issue. We also see documented examples of LLMs exposing internal passwords or confidential information from their context window, often in ways the enterprise can’t easily detect or audit.
These aren’t hypothetical. They’ve shifted from governance-as-policy to governance-as-infrastructure.
What’s Still Early and Where Things Are Heading
Other things are just starting, and it’s okay to admit it.
End-to-end trust fabrics: The vision is to provide a platform that ensures data governance, model registries, model policy enforcement, and confidential runtime with auditable controls across multi-cloud and hybrid scenarios. Some sellers are promoting this. Few have managed to ship it. The pieces don’t live together, and integrating them is difficult.
At garden scale, mutual attestation doesn’t exist. At pilot scale, bilateral verification between data owners and model vendors has proven it. Much work remains if we want to make it work across and between organizations and across all cloud providers—particularly with high-volume inference.
Regulatory clarity: Technical standards are moving faster than legal standards. These are all in flux, with developing rules in various sectors (HIPAA, Basel III for banking AI). Companies building confidential AI platforms now are taking risks on how the laws will eventually turn out.
One of the more practical technical guides that would be helpful to explore in detail, if it’s actually going to be implemented in the cloud, is Architecting Confidential AI in the Cloud, Google’s reference architecture. It includes real architectural diagrams for confidential RAG, federated learning, and analytics pipelines.
As I was looking through this material, I was aware that the most significant challenge isn’t the technical aspect; it’s the organizational aspect of bringing the legal, security, and engineering departments together. Technology is still a few steps ahead of the process in most businesses.
Where Confidential AI Is Actually Being Used
In research papers, no. In production pilots.
Banks are utilizing cross-institution fraud detection models inside TEEs without sharing customer data. Confidential computing is being applied to clinical analytics where PHI doesn’t leave a hospital’s trusted network.
Government agencies are experimenting with AI that interacts directly with citizens and encrypts sensitive case information to avoid the public cloud.
These aren’t moonshots. They’re certain cases in which the data disclosure proved to be an insurmountable obstacle, but the contribution and utility of AI were evident. To overcome this hurdle and confirm the blocker can be addressed without a revolutionary shift in ML-based methods, Confidential AI developed its solution.
My Take: The Playbook That Actually Works
What it suggests to put into practice is as follows:
Step 1: Identify and categorize AI workloads based on data sensitivity. First, inventory and classify AI workloads by data sensitivity.
Many businesses have already experimented with AI. Which of those types of workloads do you interact with that involve regulated data? Identify flag “blocked but valuable” use cases – the uses where there is clearly a need for AI, but there is a hard “no” on exposing data.
Step 2: Select one high-value workload for a confidential AI pilot. Avoid trying to cook the entire ocean! Select one use case requirement that is blocked, and rough out a prototype of that requirement using cloud confidential computing services. Consider not only performance, but also the amount of risk reduction, and whether risk reduction can be shown to a regulator in an “audit trail.
Step 3: Develop the governance layer (in parallel with the technical layer). TEEs protect data-in-use. They do not automatically address bias, explain it, or secure the supply chain. The governance system should also include attestation policies, runtime logs, and model provenance – beyond the infrastructure checklist.
Step 4: Identify a pattern with the pilot to repeat. The objective isn’t just a working pilot. It’s creating the diagrams, threat models, and attestation reports that let other business areas follow along without reinventing the process.
So my observation and experience across the deployments I have seen is that the teams that aren’t the most technologically advanced are the ones who identified a specific issue, won buy-in from stakeholders early on, and brought that evidence back to the compliance team, where they could use it.
Free Resources Worth Bookmarking
Fortunately, that’s not one for a paid course!
- Confidential Computing for Secure AI Pipelines: Practical micro-content on end-to-end pipeline security, Linux Foundation. Free.
- Zero-Trust Architecture for Confidential AI Factories: Covers the GPU threat model and design patterns, on the NVIDIA Developer Blog. Unexpectedly readable.
- ISACA AI Resources: Risk-centric and governance-centric resources, including most articles are free and open to access without membership.
- Red Hat: Improving AI Inference Security with confidential computing: dives into mutual attestation and confidential containers on Kubernetes. Technical but well-structured.
An interesting thing to look at for the wider Confidential AI market – at its future trajectory of standards organizations, open-source organizations, etc. – is the Confidential Computing Consortium. It offers the closest neutral/vendor independent perspective on the space.
FAQs: What People Actually Ask About This
Is confidential AI the same as using a private model?
No. A private model means building and running it yourself, without an agent or another company. In confidential AI, inference is cryptographically protected so even the IaaS infrastructure cannot access the data or model internals.
Do you always need TEEs, or is this overkill for most use cases?
If it’s internal productivity tools that rely on non-sensitive information — that’s often redundant. For financial records, health information, legal documents, or government information, TEEs are no longer a luxury; they’re becoming the standard.
Does confidential computing fix bias or explainability problems?
Not directly. It’s about confidentiality and integrity (NOT fairness). But it does enable better provenance (the model version deployed and the data it was trained on) for the audit dimension of responsible AI.
Where should a bank or hospital actually start?
Select one workload that has a significant impact, but is presently held up due to data accessibility issues. Prototype it in a cloud Confidential VM. Create a report of attestation. Demonstrate that to your compliance staff. This is a better argument than an architecture diagram.
Who Should Be Paying Attention to This Right Now
This is no longer a niche subject if you’re involved in tech and work with ‘regulated data’ in any capacity, whether at your own company or advising companies that do. It is emerging as the infrastructure on which enterprise AI is developed.
Security architects, cloud architects, and AI product managers in Banking, Healthcare, Insurance, and Government are already asking these questions. Confidential infrastructure falls somewhere between “we use AI” and “we can prove we use AI safely”.
The transition from public experimentation to AI that’s ready for use and production isn’t happening all at once. But most of them are taking place at a rate that was faster than expected, and it is the business that realizes that the technical and governance layers are the same, rather than separate, that will have the ability to implement the use of AI in the right place at the right time.
This is the actual possibility. Not only great AI, but trusted AI – thanks to industries.
I’m a technology writer passionate about AI and digital marketing. I create engaging and useful content that bridges the gap between complex technology concepts and digital technologies. My writing makes the process easy and engaging. I encourage participation I continue to research innovation and technology. Let’s connect and talk technology!
