The EU AI Act and on-premise models: what changes for businesses

Regulation (EU) 2024/1689 entered into force on 1 August 2024 with phased application: the bans on unacceptable-risk practices apply from 2 February 2025, obligations for general-purpose AI models (GPAI) from 2 August 2025, while the bulk of the regulation — the requirements for high-risk systems — becomes fully applicable from 2 August 2026. Penalties can reach €35 million or 7% of worldwide turnover for prohibited practices, and €15 million or 3% for most other violations. Exact dates may be subject to regulatory revisions: always check the current status before planning.

The regulation's architecture is risk-based: prohibited practices, high-risk systems (with obligations for risk management, data quality, technical documentation, logging, transparency, and human oversight), limited-risk systems (mostly transparency obligations), and minimal risk. One often-underestimated point: the obligations hit both whoever supplies the system and whoever uses it in a professional context — the so-called deployer. If your company applies AI to people, documents, or material processes, it has its own responsibilities even if the model comes from an external vendor.

In practice, the part of the AI Act that most directly intersects a DPO's daily work is data management: quality, relevance, traceability, and minimization. On top of that sits GDPR, which remains fully applicable. Sending prompts and documents to an external API means transferring personal data to a third-party controller or processor: you need legal bases, assessments of non-EU transfers, processing agreements, and often a DPIA when processing is systematic or sensitive.

Every handoff to an external vendor adds recurring compliance work, not a one-off: the vendor changes terms, processing regions, sub-processors, or retention policies, and the assessment must be redone. This is the hidden cost of cloud inference for sensitive data: not the per-token invoice, but the legal perimeter you must police permanently.

Running the model inside your own perimeter — a corporate server, a dedicated workstation, private infrastructure — eliminates at the root the most problematic category of transfers: data never leaves the environment you control, so no clauses for non-EU transfers are needed, minimization can be applied by design, and data residency is a technical fact, not a contractual promise. Even the model preparation stage can follow the same principle: with a zero-retention service like Distiller Cloud, datasets and training artifacts are automatically destroyed within 48 hours of delivery, with no residual copies held by third parties.

The second advantage is auditability. A model downloaded as a file — in standard formats like GGUF or safetensors — is an artifact: it has a version, it can be tested on your own evaluation set, archived, and re-examined. When the AI Act asks for technical documentation, logging, and the capacity for human oversight, working on an owned artifact is simpler than reconstructing the behavior of an external service that can change without notice.

It is worth saying clearly, because it is the most common mistake: if the use case falls among the high-risk ones — personnel evaluation, recruitment, credit, essential services — the AI Act obligations apply even to a model run entirely on-premises. Deployment does not change the risk classification, which depends on what the system does and to whom.

What changes is the cost of demonstrating compliance. With full control over data, logging, the execution environment, and model versions, the obligations become internal processes instead of negotiations with a vendor. The reverse choice is simpler too: if a use case is too risky or too unclear, you can keep it on hold without depending on anyone else's timeline.

First: build an inventory of AI use cases, existing and planned, recording for each the data processed, the people affected, and the vendors involved. Second: classify every case against the regulation's risk categories — for most internal automations the level will be limited or minimal, but the classification must be documented. Third: for sensitive data, assess which flows can move to local or private models, starting from the repetitive, bounded tasks where a specialized model is enough. Fourth: fix the questions to ask every vendor — retention, use of data for training, sub-processors, processing regions — and apply the same questions to internal processes.

The emerging direction is a two-speed strategy: general-purpose APIs for experimentation and tasks on non-sensitive data, specialized models inside your own perimeter for stable flows on the data that matters. That second scenario is exactly what Distiller Cloud is built for: a small model, cut to your task, that you download and run wherever your policies allow — with the certainty that nothing stays on our servers beyond 48 hours after delivery.