GuidesNews

Google Gemini in the Enterprise: What the AI Act Mandates

Google Gemini in the enterprise cloud meets the AI Act. Vertex AI only covers a part, the rest remains with the deployer.

By Alec Chizhik May 23, 2026 4 min read

Google Gemini in the Enterprise: What the AI Act Mandates

8 min read

Google Gemini has arrived in the enterprise cloud, and the AI Act has come into effect. Those building inference pipelines without accounting for general-purpose AI model obligations are building architectures that will fail during audit.

Key Takeaways

AI Act obligations are not a GDPR extension. General-purpose AI models have their own transparency, documentation, and risk obligations. A DPA with Google does not cover them.
Vertex AI is not automatically compliant. The managed service simplifies logging and region pinning, but leaves model cards, use-case evaluations, and Fundamental Rights Impact Assessment (FRIA) to the client’s responsibility.
Self-hosted Gemma costs GPU hours plus cluster overhead. Those deploying Gemini via Model Garden on their own GPUs take on compliance overhead and ongoing GPU OPEX. It only pays off at significant volume.

How Google Gemini Changes Regulatory Requirements in the Enterprise Cloud

What is an AI Act-compliant inference architecture? An inference architecture is AI Act-compliant if it technically implements the obligations from the EU AI Act for general-purpose AI models and high-risk applications. This includes model transparency, logging of inputs and outputs in regulated use cases, risk assessment per use case, Fundamental Rights Impact Assessment for public authorities, and clear separation between training and inference data.

The AI Act addresses Google Gemini not as a product but as a general-purpose AI model. Obligations apply to both the provider and the deployer. Google is the provider. The deployer is any company using Gemini in a specific use case. This shifts part of the responsibility back to DACH cloud teams consuming the model. During an audit, saying “Google is responsible” is insufficient. Those calling the model must categorize, document, and classify the call into a risk category.

This is not theoretical. The first fine proceedings are underway, and EU AI Office publications are explicit. Those planning inference architectures without AI Act considerations are setting themselves up for fines.

Where Vertex AI Simplifies Compliance and Where It Doesn’t

Vertex AI is Google’s managed inference platform for Gemini models. It automatically solves three issues: region pinning to EU locations, logging via Cloud Audit Logs, and contractual standard clauses in the Data Processing Addendum. What Vertex AI doesn’t solve: model transparency for your specific use case, risk classification against the AI Act, and Fundamental Rights Impact Assessment for high-risk applications.

This means: An HR application using Gemini for resume screening is a high-risk application under Annex III of the AI Act. Vertex AI doesn’t provide this classification. The deployer must do it themselves, ensure logging of model decisions per application, and document false positive and false negative rates. If regulators ask, “Vertex AI logs all this” isn’t enough.

// Key point

Vertex AI provides the infrastructure; the AI Act requires classification. Only the deployer can fill the gap.

Self-Hosted Gemini Endpoints: When They Make Sense

Google offers Gemini models in several variants, including open model families like Gemma 2 and 3 and proprietary endpoints that only run in Vertex AI. Self-hosted is therefore not a one-to-one choice. For Gemma variants, there are genuine on-premises paths via Triton Inference Server or vLLM on your own GPUs. For the proprietary Gemini Pro classes, this doesn’t exist–they remain bound to Vertex.

The question of “self-hosted or managed” thus reduces for most workloads to Gemma. For proprietary models, only Vertex remains, and the compliance question shifts to the endpoint setup in the project: region, logging, IAM bindings, audit trail.

Dimension	Vertex AI Managed	Self-Hosted Gemma on GKE
Pricing Model	Per token, starting at around 0.3 cents per 1,000 output tokens for Gemini Flash, significantly higher for Gemini Pro	GPU hourly rate plus cluster overhead, starting at around 3 Euro per H100 hour on-demand
Compliance Effort	Region and logging out-of-the-box, AI Act classification at customer end	Complete compliance pipeline at customer end, no vendor logging
Model Portfolio	Published by Google, to be classified by deployer	Gemma model portfolio plus own custom tuning documentation
Data Sovereignty	Region pinning to EU, government access possible under US law	Full control, provided GPU provider isn’t subject to US legal jurisdiction
Worthwhile From	Immediately, for most use cases	From around 10 million tokens per day or with clear data sovereignty requirements

Source: Google Cloud pricing pages and own analysis of hyperscaler GPU prices as of May 2026.

In practice, the self-hosted path rarely pays off below 10 million tokens per day. Below this threshold, the additional work for DevOps, MLOps and compliance costs more than the extra token costs with Vertex. Above this threshold, the ratios flip, especially when data sovereignty is required or specific tuning paths that Vertex doesn’t allow become necessary.

The Five AI Act Obligations That Must Be Addressed in Every Gemini Architecture

Obligation Clusters for the Deployer

Risk Class

Each use case is assigned to a class: prohibited, high-risk, transparency-obligated, minimal risk. This classification is a prerequisite for all subsequent obligations.

Logging

For high-risk applications, at least six months; in the financial industry or healthcare, longer. Inputs, outputs, model versions, and confidence values must be preserved.

FRIA

Fundamental Rights Impact Assessment for public entities and regulated sectors. This assessment is created before production deployment and updated before any changes.

When users interact with AI or view AI-generated content, this must be disclosed. Synthetic content must be clearly labeled; watermarked content is the state of the art.

Model drift, bias indicators, and error rates are continuously measured. The responsibility for this lies not with Google, but with the deployer.

These five obligations can be technically addressed, but not without effort. Logging is the cheapest discipline; FRIA and risk classification are the most expensive, as they require legal involvement. An inference architecture that does not incorporate these obligations into its setup will face pressure to add them later.

Why GPU Costs Drive Architectural Decisions

€3.80

This is the cost of an H100 hour on-demand on GKE in European regions. For a 24/7 setup with high availability, this amounts to a six-figure GPU OPEX per year–before cluster overhead and personnel costs are factored in.

Source: Google Cloud Pricing Page, On-Demand List Prices May 2026.

Anyone hosting Gemma on their own purchases GPU capacity that must never fail. An H100 as a single node is not a productive architecture. High availability requires at least three nodes in two zones, backup capacity for training and tuning runs, and a load balancer with affinity. The nominal €3.80 per hour effectively becomes €6–8 in real production, depending on utilization.

That’s not wrong–it’s just expensive. Those who run a clear inference workload with six- to seven-figure token numbers per day can consider the self-hosted path. Those who start experimentally can go with Vertex faster and cheaper, gaining time to establish the compliance foundation before the GPU question is decided.

What DACH Cloud Teams Must Anchor in the Next Six Months

Three key decisions determine whether a company’s Gemini integration complies with the AI Act. First: a use-case inventory, with risk classification for each application, documented and readily accessible during audits. Second: a platform decision–whether Vertex AI serves as the default or if certain use cases are offloaded to self-hosted Gemma. Third: a responsibility matrix that clearly delineates what Google delivers and what the deployer is accountable for.

Without these three foundational decisions, an architecture emerges that expands in response to every new requirement, without the underlying compliance framework keeping pace. This is the most common source of failure. Those who establish these parameters before implementing the second use case build a robust foundation; those who delay end up rebuilding later.

Frequently Asked Questions

Is Vertex AI’s EU Data Location Sufficient for GDPR Compliance?

The EU data location is a necessary but not sufficient condition. As a U.S.-based company, Google remains subject to the CLOUD Act, necessitating additional contractual and technical safeguards. Standard Contractual Clauses combined with encryption using customer-managed keys via Cloud KMS or an External Key Manager represent the standard approach. Without these supplementary measures, GDPR compliance for sensitive data remains vulnerable.

Which Gemini Use Cases Are Classified as High-Risk Under the AI Act?

Annex III of the AI Act specifically lists critical areas: critical infrastructure, education and vocational training, employment and personnel decisions, access to public services and social benefits, law enforcement, border control, and judicial administration. Deploying Gemini in any of these domains creates a high-risk application, triggering FRIA obligations, enhanced logging requirements, and mandatory reporting to authorities.

Do Companies with Fewer Than 250 Employees Need to Comply with the AI Act?

Yes–the AI Act does not primarily differentiate based on company size but rather on the risk category of the specific use case. SMEs receive some relief regarding documentation requirements, yet core obligations still apply. Even a three-person startup using Gemini for preliminary applicant screening would be considered a high-risk application, subject to all associated duties.

How Does the AI Act Differ from the GDPR in Logging Requirements?

While the GDPR mandates a register of processing activities and audit logs for sensitive data, the AI Act additionally requires model activity logging–including inputs, outputs, confidence scores, and version details–to ensure transparency and traceability of AI-driven decisions. These obligations overlap but are not identical. Simply maintaining GDPR-compliant logging does not automatically meet AI Act requirements.