When you chat with a cloud AI service, your messages are stored on that company’s servers, usually tied to your email address, and usually kept whether or not you ever open that conversation again. A breach at the company means those stored conversations are the thing at risk. Not a summary of them, not statistics about them. The text you typed.
That’s the short answer. The longer one is worth reading if you’ve used a chatbot for anything you wouldn’t email to a stranger, because the exposure paths are not the dramatic ones people imagine.
What is actually stored when you chat with a cloud AI #
Most consumer AI services keep, at minimum:
- The full text of your prompts and the model’s replies
- Files and images you attached
- The account they belong to, which is an email address and often a phone number
- Timestamps, device and IP metadata
- For paid accounts, a billing record held by the company or its payment processor
Deleted conversations typically move to a deletion queue rather than vanishing, and providers keep copies for a stated period for abuse investigation. That’s normal practice, and it’s also why “I deleted it” is weaker protection than it sounds. How to delete your data from AI chatbots walks through what each provider’s delete actually does.
Three ways chat logs get exposed #
A database left open #
The most common cause of a large leak is not a clever attack. It’s infrastructure exposed to the internet without authentication.
In January 2025, researchers at Wiz found a publicly accessible ClickHouse database belonging to DeepSeek, reachable without any credentials. It held more than a million lines of log streams, including chat history, API secrets and backend details, and anyone who found it could run arbitrary queries against it. Wiz disclosed it and DeepSeek secured it, but the researchers reported finding it within minutes of starting to look. Nothing about that requires a sophisticated attacker. It requires one misconfigured service.
Your account, not their servers #
You don’t need the provider to be breached for your chats to be read. If your email password is reused somewhere that was breached, and the AI account uses that password without two-factor authentication, someone can log in and read the history themselves. This is the most likely way an individual’s chat history gets seen, and it’s entirely within your control to prevent.
The vendors behind the vendor #
AI companies use analytics providers, support desk software, payment processors and cloud hosting. A breach at any of those can expose data the AI company handed over, which is why provider incident notices sometimes name a company you’ve never heard of. You can’t audit this chain. You can only limit what you put into it.
What an attacker gets from a chat log #
This is the part people underestimate. A chat history is a better profile than a stolen password file, because it’s written in your own words and it’s organized by topic.
A year of chats typically contains your employer and job title, the names of people you work with, health questions asked in the first person, relationship details, the city you live in, documents you pasted, your writing style in enough volume to imitate, and often a credential or two that someone pasted into a config file they wanted explained. Keeping personal data out of AI chatbots covers what to hold back.
It’s also usable for targeted phishing. A message referencing the actual project you asked for help with last week gets clicked.
What to do about it today #
- Put a unique password and two-factor authentication on every AI account. This blocks the most likely exposure path.
- Turn off training and chat history where you don’t need it. Most services have both settings. Turning off history usually also shortens retention.
- Delete what you don’t need, and check the retention wording rather than assuming deletion is immediate.
- Stop pasting the categories you’d never email: ID numbers, account numbers, credentials, other people’s medical or legal details, anything covered by a work confidentiality agreement. Using AI at work without leaking confidential data goes through the work side.
- Assume screenshots of chats are as sensitive as the chats. They end up in camera roll backups.
The setup where there is no log to leak #
Encryption in transit doesn’t help here, because the provider decrypts your message to answer it. That’s what end-to-end encryption means for AI chat, and why “it’s encrypted” is not an answer to “can it leak”.
The structural fix is not sending the conversation anywhere. A model that runs on your phone computes the answer locally, so there’s no server-side copy, no account the copy is attached to, and no vendor chain behind it. Personal LLM works this way: you download an open model once, such as Qwen 3.5 or Gemma 4, and after that the chats, the photos you attach and the models themselves stay on the device. There’s no account, and the developer doesn’t operate a server that could hold your conversations. It’s free on iOS and Android.
Be clear about what that does and doesn’t fix. It removes the provider breach, the account takeover and the subprocessor chain, because none of them exist. It does not protect you from:
- Someone with your unlocked phone. Local chats are readable by anyone holding the device, so a passcode and biometric lock are doing real work.
- Your own backups. If the app’s data is included in a phone backup, wherever that backup lives now holds the chats too.
- Malware on the device. An on-device model is only as private as the device.
- You exporting a chat and sending it somewhere else.
Frequently asked questions #
How do I know if my AI chats were in a breach? #
Usually you don’t, unless the provider notifies you or a researcher publishes. Breach notification rules vary by country and by the type of data involved. Watching a credential-monitoring service for your email address catches the account-level risk, but it won’t tell you whether the content of your conversations was exposed.
Does deleting a conversation remove it from the company’s servers? #
Eventually, in most cases, but not instantly. Providers publish a retention window for deleted content, often around 30 days, and separately retain items flagged for abuse review. Deleting is worth doing. Treating it as an undo button is not.
Are AI chats protected by a password reset if I get breached? #
Changing the password stops future access but does nothing about data already copied. If you think an account was accessed, change the password, enable two-factor authentication, then read through the history and treat anything sensitive in it as compromised.
Is an offline AI app really breach-proof? #
There’s no server holding your conversations, which removes the entire category of remote breach. The risk moves to your phone: theft, unlocked screens, backups and malware. That’s a smaller and more controllable attack surface, not a magic one. How to check whether an AI app is really offline shows how to confirm an app behaves the way it claims.
Should I worry about AI companies using my chats for training? #
That’s a separate question from breaches, and a more predictable one, since it’s governed by a setting and a policy rather than an accident. Do AI chatbots use your chats for ads and training covers what each service says it does.