AI Infrastructure Made in Germany: GPU Clusters, Sovereign AI, and the European Path

Anyone wanting to train AI models needs computing power – lots of it. Until now, that has almost always meant US cloud providers, US hardware, and US jurisdiction for European companies. That’s changing – and Germany is playing a central role.

TL;DR

• Germany is investing over €3 billion in AI compute infrastructure – from the Jülich Supercomputing Centre to private GPU farms in Frankfurt and Scandinavia.
• European GPU capacity tripled in 2025: Providers like Nebius, CoreWeave, and Northern Data are scaling up aggressively.
• Sovereign AI – AI models trained on European infrastructure – is no longer a niche topic but a strategic necessity.
• The German Federal Ministry for Economic Affairs is supporting decentralized AI compute capacity through its AI Computing Hub programme.
• The bottleneck is shifting: GPU availability is no longer the main constraint – energy, cooling, and skilled personnel are.

The dependency is real: Nvidia controls over 80% of the GPU market for AI training. AWS, Azure, and Google Cloud supply the bulk of AI compute capacity. For a German company fine-tuning a large language model on sensitive corporate data, this means the data leaves the European legal jurisdiction – at least theoretically.

In 2025, a counter-movement has begun – one that goes far beyond symbolism. GPU clusters are emerging across multiple locations in Germany, offering European companies sovereign AI infrastructure.

The GPU Offensive: Who’s Building What

The Jülich Supercomputing Centre has brought online JUPITER – the first European exascale supercomputer. With more than 24,000 Nvidia GH200 chips, it delivers one exaflop of computing power: enough to train foundation models of European scale. Access is open to research institutions and enterprises via the EuroHPC programme.

Meanwhile, Northern Data is building one of Europe’s largest commercial GPU clusters, with sites in Frankfurt and Scandinavia. Its infrastructure comprises 8,200 Nvidia H100 GPUs, available as a cloud service for enterprises. The investment totals around €330 million. “Our goal is to offer European companies a viable alternative to US hyperscalers for AI workloads,” says CEO Aroosh Thillainathan.

Nebius – the cloud division spun off from Yandex – is constructing an AI data centre in Mäntsälä, Finland, with capacity for up to 60,000 GPUs. Power will come from Scandinavian hydropower. And CoreWeave – originally a US startup – has operated a European site in London since 2025, with plans to expand into Frankfurt.

Why Sovereign AI Matters Now

The EU AI Act requires companies deploying high-risk AI applications to fully document both their training data and underlying infrastructure. If a German insurance company trains an AI model for claims processing, it must be able to prove where the data was processed – and who had access.

That proof is difficult to provide on a US hyperscaler – not because AWS or Azure are insecure, but because US laws like the CLOUD Act theoretically permit access by US authorities. On European infrastructure, that concern vanishes.

There’s also an economic rationale: Companies regularly training large models often find dedicated or rented GPU clusters cheaper than on-demand cloud instances. Once utilization reaches 60-70%, dedicated hardware typically pays for itself within 18-24 months.

The Bottleneck: Energy and Cooling

A 1,000-GPU H100 cluster consumes as much electricity as a small village – roughly 700 kilowatts continuously. Data centres in Frankfurt, Europe’s largest internet hub, are already hitting local grid limits. Grid expansion is lagging behind demand.

Many are looking to Scandinavia for solutions: cheap hydropower, cold ambient air for natural cooling, and high-speed fibre connectivity to Central Europe. Yet for latency-sensitive applications – such as real-time AI inference – locations in Germany or the Benelux countries remain preferable.

The BMWK has recognized this and is backing decentralized AI compute capacity through its “AI Computing Hubs” programme. Five sites are slated to go live by 2027 – distributed across Germany to ensure redundancy and local availability.

What This Means for CIOs

Europe’s AI infrastructure landscape is diversifying. In 2026, CIOs have – for the first time – genuine alternatives to the three major US hyperscalers for AI workloads. The strategic choice is clear: keep experimental training and prototyping in the public cloud – but run production, regulated AI applications on European infrastructure.

CIOs who architect this separation early – using containerized ML pipelines portable across cloud providers – gain flexibility without locking in.

Frequently Asked Questions

Header Image Source: Pexels / Brett Sayles

Translated from the German original using artificial intelligence. The German version is authoritative.