Yes, but most people who want to “train an AI on their own data” don’t need training at all. There are three ways to make an AI know your stuff. You can tell it about yourself in a system prompt, which takes minutes. You can let it look things up in your documents while it answers, which is called retrieval or RAG. Or you can fine-tune the model, which changes the model itself and needs a capable computer, a prepared dataset and some patience.
The first two work privately on a phone today. Fine-tuning doesn’t happen on a phone, but a model you fine-tuned elsewhere runs on one afterwards.
What does “training an AI” actually mean? #
The word covers three very different things:
- Pretraining is how a model is made in the first place, from enormous amounts of text on thousands of chips. Nobody does this with personal data at home.
- Fine-tuning takes an existing model and trains it further on your examples, changing its behavior permanently.
- Giving it context doesn’t change the model at all. You hand it information at the moment you ask, through a system prompt or an attached document.
When people say “train,” they usually want the result of the third: an AI that knows their business, their notes or their writing style.
Which approach fits what you want? #
| What you want | Best tool | Why |
|---|---|---|
| A tone, persona or set of rules | System prompt | Instructions work immediately and cost nothing |
| Answers based on your notes, manuals or files | Attach the documents (retrieval) | The model reads the relevant passages at question time |
| Knowledge of recent events or changing facts | Documents, or a chatbot with web search | Fine-tuning is unreliable for adding facts |
| A strict output format every time | Examples in the prompt first, then fine-tuning | A few examples often do it |
| A specialized style or jargon, at scale | Fine-tuning | Training shapes habits more deeply than a prompt |
| A small model that behaves like a larger one on one narrow task | Fine-tuning | This is where it pays off most |
The common misunderstanding is that fine-tuning “loads your data into the model” like a database. It mostly teaches how to respond: style, format and task habits. For “answer from my data,” retrieval is more accurate and far less work.
Option 1: Tell it about yourself with a system prompt #
A system prompt is standing instructions the model reads before every message in a chat. Put in who you are, what you’re working on and how you like answers written, and it applies every time.
I run a small bakery in Leeds. My customers are mostly families. When I ask for social posts, keep them under 60 words, warm, no hashtags. Prices are in pounds.
It’s the fastest way to make any AI feel personal, and in a local app the prompt never leaves your phone. Personal LLM gives each chat its own system prompt, so a bakery-marketing chat and a study chat can know different things about you. See personalize an AI chatbot without sharing your data for the approach, and system prompt examples for ready-made wording.
Good for: your role, your preferences, a handful of key facts. Not for: a whole handbook, since everything in the prompt eats the model’s limited working memory.
Option 2: Let it read your documents (RAG) #
Retrieval-augmented generation is what “chat with your files” features actually do. When you ask a question, the app searches your document for the most relevant passages and hands those to the model along with the question. The model answers from what it was just given.
In Personal LLM you attach a PDF, text or Markdown file. The text is extracted and searched on the phone, the model answers from the matching passages and tells you which ones it used, so you can check its work. Nothing is uploaded. The step-by-step is in how to chat with a PDF offline, and what is RAG explains the searching.
Good for: manuals, policies, contracts, notes, study material, anything factual you want answers from. Not for: questions that need the whole document at once, like “what’s missing from this report,” since the model only sees the passages the search picked.
Option 3: Fine-tune a model #
Fine-tuning trains an existing open model further on examples you provide, usually pairs of prompts and ideal answers. Most people use LoRA, which trains a small add-on instead of the whole model and needs far less memory. QLoRA does the same on a quantized model to save more.
What you need:
- A computer with a strong GPU, a Mac with plenty of unified memory, or a rented cloud GPU by the hour
- A clean dataset of examples in the style you want
- Open-source tooling, such as Hugging Face’s training libraries
- Time to test, because a first attempt often makes the model worse at everything else
The path from training to your phone #
- Pick a small base model that already runs well on your phone. Starting small keeps training cheap and the result usable.
- Prepare examples. Quality beats quantity; a few hundred careful pairs beat thousands of sloppy ones.
- Train a LoRA adapter on a single consumer or rented GPU.
- Merge the adapter into the model, producing a normal model with your changes built in.
- Convert to GGUF and quantize it. llama.cpp’s conversion tools do this, and Q4_K_M is the usual balance for phones. See GGUF quantization explained.
- Host the file somewhere your phone can download it from, such as a Hugging Face repository, and load it by URL.
Personal LLM adds any GGUF by URL, with an optional vision projector if your model takes images. Once downloaded, your fine-tuned model runs offline with the same settings as the built-in catalog. The steps are in run a custom GGUF model on your phone.
What fine-tuning is and isn’t good at #
| Works well for | Works poorly for |
|---|---|
| A consistent writing style or tone | Teaching lots of new facts reliably |
| A fixed output format | Keeping information up to date |
| A narrow repeated task, like classifying messages | Replacing a search over your documents |
A fine-tuned model absorbs patterns more readily than facts, and it can still invent details about the very material it trained on.
What about privacy when you fine-tune? #
Your training data goes wherever you train. Rent a cloud GPU and you upload your examples to that machine; use a hosted fine-tuning service and the provider holds them under its terms. For personal or confidential material, training on your own computer is the private option. Once the finished model is on your phone, using it is fully local.
Can you train an AI directly on your phone? #
Not a chat model, in any practical sense. Three physical reasons:
- Memory. Running a model needs roughly its file size in RAM. Training needs considerably more on top for the numbers used to adjust the weights, even with memory-saving methods. A phone that can just about run a 4B model has nowhere to put that. See how much RAM you need to run an LLM on a phone.
- Heat and power. Training keeps the chip at full load for hours. Phones throttle within minutes of heavy work, as anyone who has run a long local chat has watched happen: why your phone gets hot running AI.
- Tools. The training software people actually use targets desktop and server GPUs.
Phones do run small amounts of on-device learning for things like keyboard suggestions, sometimes combined with federated learning, which our federated learning explainer covers. That’s a different scale entirely.
It’s also worth knowing that models don’t learn from your chats as you go. The weights stay fixed while you talk. Anything that feels like learning comes from context you gave it, or from a cloud chatbot’s memory feature storing notes about you on its servers.
Which option should you use? #
| Your goal | Best option | Stays private? |
|---|---|---|
| AI that knows your role and preferences | System prompt | Yes, in a local app |
| Answers from your notes, manuals or PDFs | Document search (RAG) | Yes, in a local app |
| A model that always writes in your style | Fine-tuning, plus a system prompt | Yes, if you train on your own computer |
| A narrow repeated task at scale | Fine-tuning | Depends where you train |
| Answers about current events | None of these; use web search | No |
What about custom GPTs, Gems and Projects? #
ChatGPT’s custom GPTs and Projects, Gemini’s Gems and similar features let you add instructions and upload files as “knowledge.” These are the cloud version of options 1 and 2. They don’t retrain the model, and your files sit on the provider’s servers under its privacy policy. For confidential material, build the same setup on your phone instead.
Frequently asked questions #
Can I train ChatGPT on my own data? #
Not in the sense of changing the model, as a regular user. You can give it custom instructions and upload files to a custom GPT or Project, and it will use them when answering. Those files are stored on OpenAI’s servers.
Is RAG the same as training? #
No. RAG searches your documents and shows the model relevant passages at the moment you ask. Training changes the model’s weights permanently. RAG is cheaper, easier to update, and better at showing where an answer came from.
How much data do I need to fine-tune a model? #
Quality matters more than quantity. A few hundred clean, consistent examples can shift a model’s style or format. Teaching broad new knowledge takes far more, and even then retrieval usually works better.
Can a local AI learn from my conversations? #
Not by itself. The model stays the same no matter what you tell it. To make it remember something, put it in the chat’s system prompt or attach a document containing it.
Is it safe to upload my documents to train an AI? #
It depends on the service and the documents. Cloud tools store what you upload under their own policies, and some may use it to improve their models unless you opt out. For confidential files, use a model on your own device, where nothing is uploaded.