What you need to know when self-hosting LLMs

Having your own AI in the server room sounds like control, security and science fiction. Meta, Google, DeepSeek and OpenAI are already making language models available for download. However, those who fail to observe the following points, among others, risk significant legal and economic consequences.

Introduction

An LLM (Large Language Model) is a language model that predicts and generates texts. Operating the model on your own hardware is known as self-hosting. There are good reasons for this, such as data and confidentiality protection, independence and cost control. Self-hosting does not come without any obligations.

Obligations arise in part from the General Data Protection Regulation, the licence agreements with the companies providing AI (hereinafter referred to as "language model manufacturers") and soon for everyone from the AI Regulation. The AI Regulation will apply in full from 2 August 2026, but parts of the AI Regulation already apply today.

The licence agreements with the language model manufacturers apply as soon as they are downloaded. Among other things, this can result in contractual obligations during the existence and after termination of the agreement.

"Open source LLM" or just "open weights" after all

The licence agreements clearly specify the rules of the game, e.g. whether commercial use is permitted, whether the publication of your own further development or implementation is mandatory, etc. To determine the specific canon of obligations and the freedoms granted, a distinction must be made between open source and open weights.

In simple terms,open source means: The source is available (e.g. source code) and I may use, understand, modify and pass it on for any purpose. Applied to an LLM, this would mean that

• the entire source code for training, evaluation and operation of the model;
• the training data or meaningful information on its origin, selection and preparation; and
• the parameters of the trained model (i.e. weights and checkpoints of the model)

are openly available. Only then can a skilled person recreate a system that is essentially equivalent.

In contrast to this,open weights means Only the weights (end product of the trained language model - bullet point 3 above) are available. You can work with them and often fine-tune them. However, the training path and the training data in particular are missing, meaning that complete reproducibility is usually not possible.

Clearly illustrated: Open Source is the finished cake with an enclosed recipe, list of ingredients and baking instructions. Open Weights, on the other hand, is the finished cake in a box. It is edible. You might even add your own icing. However, the recipe and what exactly is inside remain hidden.

In practice, language model manufacturers usually publish open weights.

For companies, this means: first classify it properly, then implement the licence conditions consistently. This way, freely accessible does not become a compliance trap.

Pitfall 1: licence violations with open weights

Anyone who fails to comply with licence agreements risks getting a letter from a lawyer. Open weight models are subject to very different licence models. The following licence models exist in extracts:

• There are permissive licences such as Apache 2.0 or MIT, which offer extensive freedom and contain few restrictions.
• There are community or manufacturer licences with usage rules such as naming and attribution obligations as well as Acceptable Use Policies (AUP). For example, compliance with data protection law and export control law or the exclusion of specific uses can be an essential part of the licence agreement via AUP. Meta's licence agreement for Llama contains AUP.
• There are Responsible AI licences with pass-through obligations for derived models. The following almost always applies: Anyone who passes on model files or provides a fine-tuned model must also provide licence texts and notes. Restrictions on use must be observed.

Licence agreements often provide licensors with termination options in the event of a breach of contract. The licence agreements often regulate contractual claims for removal and injunctive relief in the event of termination of the contract (e.g. clause no. 6 of the Meta Llama 3 community licence "[...]you shall delete and cease use").

In addition to contractual claims, copyright claims may also arise in the event of licence infringements. Under German copyright law, an infringement of (copyright) usage rights gives rise to a statutory claim for injunctive relief and damages under Section 97 UrhG. Section 97 (2) UrhG provides the rights holder with a choice of three calculation methods for the assessment of damages: compensation for specific pecuniary loss; surrender of the infringer's profit or payment of a fictitious reasonable licence fee (licence analogy). The last two calculation methods in particular pose serious risks for the infringer under copyright law.

According to the prevailing opinion, published model weights (open weights) are generally not protected by copyright due to the lack of human intellectual creation. The weights are created autonomously during the machine training of the language model and do not represent a human creative achievement. This is to be distinguished from, for example, the architecture of the LLM or programme code (e.g. tokeniser, runtime code), which can enjoy copyright protection.

Pitfall 2: The material scope of application of the AI Regulation

The AI Regulation is risk-based. There are prohibited practices, high-risk AI systems and general low-risk obligations. The specific obligations that apply depend on what is being considered (reference object), who is acting (reference subject) and what risk level exists.

Reference object: AI system or AI model

The AI Regulation provides an important differentiating feature, somewhat hidden in recital 97 p. 7 and 8⁽⁷ In order for AI models to become AI systems, the addition of further components, for example a user interface, is required. ⁸ AI models are usually integrated into and part of AI systems). An AI system is the applicable unit. It processes user input and generates output for the user (user interface). An AI model is the underlying model. It can be integrated into many different systems. In practice, the system is the application with the user interface. The model is the technical centrepiece behind it.

Reference subject: provider or operator

Aprovider is anyone who develops an AI system or model or has one developed and markets it under their own name or puts a system into operation under their own name. An operator is someone who uses an AI system under their own responsibility. The roles trigger different obligations.

Do I become a provider myself by fine-tuning the language model? Anyone who significantly changes an existing system can trigger provider obligations. This also applies if a previously low-risk system becomes a high-risk system due to its new purpose. Do I become a provider if I use RAG (Retrieval Augmented Generation) ? In principle, the inclusion of external knowledge sources for decision-making does not change the weights/parameters of a language model. As long as you do not fulfil the other provider requirements, RAG alone does not lead to the creation of a provider role.

Conclusion

If you want to operate your own AI in the long term, you should think about compliance from the outset. This includes the selection of the right language model for the specific purpose, the specific licence check, compliance with licence agreement and legal regulations and their detailed documentation.