AI and data privacy

This page is an overview of privacy and confidentiality when using AI. It covers what happens to data you send to an AI service, the specific ways AI can expose data, the legal framework in Europe, and practical safeguards. It applies to chat assistants as much as to applications built on model APIs.

Where your data goes

When you use a hosted AI service, everything in the prompt travels to the provider’s servers: your message, attached files, pasted code, and any documents the application adds. From there, several things can happen to it:

Diagram 1

flowchart LR
  prompt["Your prompt and files"] --> provider["Provider's servers"]
  provider --> answer["Answer"]
  provider --> logs["Stored logs and chat history"]
  provider --> review["Abuse monitoring, possibly human review"]
  provider --> training["Training of future models, depending on settings"]

Processing. The model reads the data to produce the answer.
Storage. Chat apps keep conversation history so you can return to it. APIs often keep requests for a limited time, commonly around 30 days, to detect abuse.
Review. Flagged conversations may be read by staff or contractors.
Training. Some services use conversations to train future models.
Sharing. Requests may pass through subprocessors, such as cloud providers, in other countries.

Consumer apps and business services differ

The most important distinction is between consumer products and business products.

Consumer chat apps, free or paid individual plans, often use conversations to improve future models unless the user opts out in the settings. Defaults change over time, so check the settings of the services you use.

Business plans and APIs usually do not use customer data for training by default. They come with contracts that define retention, security, and the provider’s role as a data processor. Some offer zero data retention, where requests are not stored at all, and a choice of data region, such as processing within the EU.

The same model can therefore be fine for confidential work through a company account and unsuitable through a personal free account.

How AI can expose data

AI adds some privacy risks that ordinary software does not have:

Pasting confidential data. The most common leak is simple: employees paste source code, customer lists, or contracts into a personal chatbot account. In 2023, Samsung restricted generative AI tools after engineers entered confidential code and meeting notes into ChatGPT.
Memorization. Models can memorize rare passages from their training data, and researchers have extracted personal details this way. Data that went into training cannot easily be removed from the model.
Shared and stored chats. Shared conversation links have ended up in search engine results. Chat history and memory features keep personal details for a long time.
Retrieval without permissions. A company assistant that searches internal documents can reveal content to employees who should not see it, if the search ignores access rights.
Agents with access. An agent that reads email and files and can send messages can be manipulated through prompt injection into sending data to an attacker.
Inference. Models can deduce sensitive facts, such as location, health, or opinions, from data that looks harmless.

Data protection law in Europe

In the EU, the General Data Protection Regulation (GDPR) applies whenever AI processes personal data, both in training and in use. In short:

There must be a legal basis for the processing, such as consent, a contract, or a legitimate interest that outweighs the person’s rights.
Data must be limited to what is necessary for a stated purpose.
People have rights to information, access, correction, and deletion. Correcting or deleting data inside a trained model is technically hard, which is one of the open legal questions.
A company that uses an AI provider needs a data processing agreement, and transfers outside the EU need a legal mechanism.
Risky processing requires a data protection impact assessment.
People have the right not to be subject to decisions with significant effects based solely on automated processing, with some exceptions.

Data protection authorities enforce these rules on AI. Italy’s authority temporarily blocked ChatGPT in 2023 and fined OpenAI 15 million euros in 2024 over how it handled personal data.

Practical rules for individuals

Do not enter passwords, ID numbers, health records, or other people’s personal data into a chatbot unless you know how it is handled.
Check the privacy settings: training on your conversations, chat history, and memory.
Use your organization’s approved AI tools for work, not a personal account.
Be careful with shared links and with granting assistants access to email, calendar, and files.

Practical rules for organizations

Write an AI usage policy that states which tools are approved and which data may go into them.
Provide approved tools with business contracts, so employees do not turn to personal accounts.
Classify data. Decide which categories, such as public, internal, confidential, and personal, may go to which kind of service.
Minimize and redact. Send only what the task needs, and remove or replace personal data before it reaches the model where possible.
Keep sensitive data in-house with open-weight models on your own infrastructure, or a cloud region and contract that meets your requirements.
Enforce permissions in retrieval, so an assistant shows each user only what that user may see.
Limit agent access and require approval before an agent sends data outside the organization.
Involve the data protection officer early, and document the processing.