7 min. reading time<\/p>\n
Artificial intelligence is becoming a strategic infrastructure issue for companies. In conversation with Alexander Hendorf, AI consultant and open-source expert, it becomes clear: if you want to use AI sovereignly, you need to understand, operate and control what happens within your own system. Open source is not an afterthought here, but a prerequisite for control, operational capability and long-term competitiveness.<\/strong><\/p>\n Key takeaways<\/p>\n Related:<\/span>Platform engineering is no longer just a DevEx project<\/a> \/<\/span> SAP Sovereign Cloud France<\/a><\/p>\n On 3 June 2026, the European Commission presented its European Tech Sovereignty Package: a package comprising Chips Act 2.0, a Cloud and AI Development Act and its own open-source strategy. The planned Cloud and AI Development Act is set to define graduated sovereignty levels for sensitive workloads and expand European data-centre capacity. The open-source strategy explicitly treats open software as a lever to reduce dependence on third countries in cloud, AI and cybersecurity.<\/p>\n For Hendorf, the direction is right, but the focus is incomplete. Capacity alone does not create control. What counts is whether companies and public authorities can also understand, operate and audit the systems running on that infrastructure.<\/p>\n \u201cEurope\u2019s bottleneck is not merely models or data centres. Europe\u2019s bottleneck is the ability to build AI systems under control, operate them productively, assess them securely and adapt them to real-world use cases – locally, vendor-neutrally and on the basis of open source. Only open models and open software let you truly penetrate, audit, adapt and run AI systems in-house. Proprietary black boxes cannot be controlled sovereignly.\u201d<\/p>\n Alexander Hendorf, AI expert and head of the Open Source working group at the German AI Association, in his statement on the EU Tech Sovereignty Package<\/p>\n<\/div>\n What is AI sovereignty?<\/strong> AI sovereignty describes a company\u2019s ability to independently select, integrate, operate, and audit AI systems. This includes control over data flows, models, infrastructure, quality, security, and migration paths. What matters is not the model\u2019s name, but the operational capability behind it.<\/p>\n Many businesses are currently debating sovereign AI, European models like Mistral, and the use of open source. Yet the conversation has long since moved beyond individual applications or chatbots. AI is now deeply embedded in business processes, data platforms, and cloud architectures: from contract analysis and customer communication to internal knowledge search.<\/p>\n Even at leading AI conferences such as PyCon DE and PyData, the focus has shifted. Today\u2019s discussions center not only on models and frameworks, but also on AI agents, API standards, data architectures, and software engineering. That\u2019s precisely why the question is shifting from model selection to operational capability.<\/p>\n The debate around sovereign AI is often framed too narrowly. Many companies feel pressured to choose between cloud and on-premise as if it were a binary decision. According to Hendorf, that\u2019s missing the point.<\/p>\n The real issue is whether companies truly understand and control their infrastructure – regardless of where it runs. Relying exclusively on hyperscalers and SaaS providers can quickly lead to dependencies: model versions change without notice, pricing and quotas are adjusted unilaterally, and data flows end up in regions that compliance teams can\u2019t approve. At the same time, local operation only makes sense if the necessary expertise exists in-house.<\/p>\n \u201cThe software isn\u2019t the asset – it\u2019s everywhere available. Even the hyperscalers run on open source and actively promote it,\u201d says Hendorf. \u201cThe real asset is operational capability: the ability to understand a system, set it up yourself, and switch between cloud and your own hardware when needed. That skill set is what\u2019s missing in many organizations.\u201d<\/p>\n What operational capability looks like in practice can be summed up by a term that\u2019s standard in engineering circles but rarely heard in boardrooms: the Harness. It refers to an in-house test and evaluation environment where every AI model is systematically tested against your own use cases, data samples, and quality criteria before going live.<\/p>\n \u201cThe Harness is the real asset,\u201d says Hendorf. \u201cIt\u2019s the safeguard for model changes and upgrades. Without it, every switch becomes a leap of faith. You only find out weeks later whether the new model still does what the old one did. With it, switching becomes a routine technical decision.\u201d<\/p>\n This gap is one of the biggest vulnerabilities when companies want to change providers or update a model. Without your own Harness, you\u2019re forced to trust what release notes promise – and hope it works in production.<\/p>\n Models, frameworks and infrastructure components are more widely available today than ever before: from open language models like Llama or Mistral to inference servers such as vLLM and Ollama, and vector databases for in-house knowledge search. This shifts the AI debate away from the narrow question of who has access to the most powerful model. What becomes more important is who can understand, adapt and reliably operate AI systems.<\/p>\n The gap to proprietary leaders is shrinking. Hendorf points to assessments from the PyCon community indicating that commercial models are only months ahead of open alternatives rather than years. For companies this is a meaningful shift: they can increasingly configure, run locally, integrate into their own processes and exert stronger control over AI systems. This new freedom, however, also increases responsibility within their own organisation.<\/p>\n Because open source does not automatically remove vendor lock-in; it merely relocates the requirements. To leverage open AI, businesses need technical know-how, clear architectural decisions, governance and the ability to realistically assess quality, security and operating costs.<\/p>\n This brings a question to the forefront that often gets lost in the hype around European or American models: what criteria should companies use to select AI systems at all? Decisive is not merely the origin of a model, but whether it matches the specific use case, existing data assets, regulatory obligations and in-house operational capability.<\/p>\n \u201cWhich of today\u2019s widely available models a company adopts is decided not by origin, but by application,\u201d says Hendorf. \u201cModels from China, for example from DeepSeek or Qwen, are technically world-class, openly available and in many tasks equal to or ahead of Western open-source models. Whether their training data is politically filtered is not a decision criterion for the vast majority of enterprise use cases such as contract analysis, knowledge search or classification. Every model carries bias. The relevant question is not where it comes from, but whether you can control it in your own application.\u201d<\/p>\n How pertinent this question has become is illustrated by a real-world example from the financial sector. An asset manager wanted to train its own AI models and initially considered a cloud solution. In the highly regulated environment, governance and compliance processes would have taken twelve to twenty-four months. During that period many initiatives are shelved before ever going live.<\/p>\n After analysing the actual use case, the decision went another way: instead of a large generic LLM, the team built a much smaller on-premise solution with its own server and two Nvidia consumer GPUs inside a secured network. According to Hendorf and his team, they brought it into production in four weeks.<\/p>\n The result: the total solution cost less than six months of cloud operation. At the same time, staff could start working with the system immediately instead of waiting for protracted approvals.<\/p>\n For Hendorf, the example highlights a core problem of many AI projects. Companies often aim for maximum scalability even when their requirements are far more specialised: \u201cNot every use case needs a Porsche; sometimes a kick scooter is enough.\u201d Smaller models with only a few billion parameters can be more efficient, cheaper and easier to control for clearly defined tasks than large universal systems. And they run on hardware the company already owns.<\/p>\n This trend is accelerating with the rise of AI agents. Agents access data, leverage APIs, and automate workflows, but they require structured technical environments to do so.<\/p>\n Historically grown silos, poorly documented interfaces, and convoluted tool ecosystems are becoming liabilities. According to Hendorf, AI agents perform far better with standardized APIs and consistent data structures. New open standards such as the Model Context Protocol assume that underlying systems are cleanly documented and accessible. Where this foundation is missing, agents don\u2019t fail because of the model – they fail because of the house architecture.<\/p>\n That forces companies to reassess their infrastructure. Open source can help by making systems more transparent and flexible, yet without sound architecture and solid software engineering, new complexity emerges quickly.<\/p>\n Hendorf also sees widespread misconceptions around security and data privacy. Open source is not automatically secure. Downloading a model from a public hub without vetting its provenance imports the same supply-chain risk as any other software dependency. At the same time, a local infrastructure is not inherently riskier than sprawling cloud landscapes with opaque data flows.<\/p>\n For sensitive data, well-secured internal networks can offer advantages. Ultimately, access control, governance, and technical architecture remain decisive. That includes the ability to trace, if necessary, which inputs a model processed and when.<\/p>\n AI is therefore reshaping the role of IT and security teams. They must enable innovation while ensuring the company retains control over data, models, and processes.<\/p>\n The open-source AI debate is about far more than model choice. Companies must decide how dependent they want to be on external platforms and which technical and organizational knowledge they need to build in-house.<\/p>\n Cloud platforms and proprietary models will continue to play an important role. Yet the growing premium is on operational self-sufficiency. Hendorf sees this as the real foundation for digital sovereignty.<\/p>\n The regulatory backdrop adds urgency. The EU AI Act classifies certain AI applications – such as those in HR, credit scoring, or critical infrastructure – as high-risk and demands traceable data flows, documented model decisions, and auditable operations.<\/p>\n Put simply: to use AI strategically over the long term, you need not only access to models but control over the infrastructure behind them.<\/p>\n Three questions before every new AI initiative<\/p>\n If you can\u2019t answer these questions within a day, you don\u2019t have a model problem – you have an architecture problem. That\u2019s where the real work on sovereign AI begins.<\/p>\n Sovereign AI means a company retains control over models, data flows, quality, and operations. Location alone isn\u2019t the deciding factor. Even a cloud solution can be sovereign if architecture, governance, and migration paths are mastered.<\/p>\n<\/details>\n A harness tests models against your specific use cases and quality criteria. Without this testing environment, every model change carries risk, as deviations often only surface once the system is live.<\/p>\n<\/details>\n Many enterprise use cases don\u2019t require a generic large model. Specialized smaller models can be cheaper, faster, and easier to control for clearly defined tasks.<\/p>\n<\/details>\n More from the MBF Media Network<\/p>\n Process optimization without endless projects: How SMEs stay agile<\/a> <\/div>\n\n
The EU package bets on open source – yet the bottleneck lies elsewhere<\/h2>\n
AI sovereignty starts with the infrastructure<\/h2>\n
Cloud or on-premise is the wrong question<\/h2>\n
Open source is changing the playing field<\/h2>\n
Why smaller solutions are often the smarter choice<\/h2>\n
AI agents intensify pressure on governance<\/h2>\n
Security doesn\u2019t come for free<\/h2>\n
Infrastructure becomes a competitive edge<\/h2>\n
\n
Frequently Asked Questions<\/h2>\n
What distinguishes sovereign AI from traditional cloud-based AI?<\/strong><\/summary>\n
Why is a proprietary harness so important?<\/strong><\/summary>\n
Why do smaller on-premises setups often suffice?<\/strong><\/summary>\n