Is Local AI on Your Phone a Gimmick? An Honest Look

For some jobs, yes. For others it’s the best tool you have. A 4-billion-parameter model running on a recent phone can draft, rewrite, explain, translate and answer questions about a photo at reading speed, with nothing leaving the device. It can’t match a frontier cloud model on hard reasoning, it doesn’t know this week’s news, and it chokes on very long documents.

The “gimmick” label usually comes from someone who tried a tiny model on an old phone, asked it a trivia question, and got a confident wrong answer. That’s a fair complaint. It just isn’t the whole picture, and this post is about whether it’s worth your time, not what the technology can do in general.

Why do people call local LLMs a gimmick? #

The criticisms are real, so start with them.

The models are much smaller #

Phone models run from under 1 billion to about 9 billion parameters. The big cloud chatbots run models many times that size on server hardware with far more memory. Size isn’t everything, but it shows up in general knowledge, in long multi-step reasoning, and in how often the model makes things up.

They hallucinate more #

Small models are more likely to invent a fact, a quote or a citation when they don’t know something. Ask a 1B model who won an obscure award in 2011 and you’ll often get a plausible name that’s wrong. Our guide to why AI makes things up explains why and how to reduce it.

They can be slow, hot and hungry #

On a phone with 4 GB of RAM and no usable GPU path, a larger model crawls along at a few words a second. Long sessions warm the phone and use battery. The downloads are big too: 0.8 GB to over 6 GB per model.

They don’t know anything recent #

A local model knows what was in its training data and nothing after, with no web search. Ask about yesterday’s game or today’s weather and it either says it doesn’t know or guesses.

The memory window is short #

To stay inside a phone’s RAM, local apps use a smaller context window than cloud chatbots. In a long conversation the model starts forgetting the beginning, and a 200-page PDF can’t be read in one go.

What do the critics get wrong? #

Most of the criticism is about asking a small model to be ChatGPT. Judge it on what it’s for and the picture changes.

• Speed is fine on current phones. A 4B model on a phone from the last few years streams text about as fast as you read it. The 9B models are slower and noticeably smarter.
• The privacy is structural, not a promise. When the model runs on your phone there’s no server to receive your chat, so there’s nothing to leak, subpoena, train on or show a reviewer. You don’t have to trust a privacy policy.
• It works with no signal. On a plane, in a basement, abroad without a data plan. After the one-time download, airplane mode changes nothing.
• Your own text is its strength. Rewriting an email, summarizing notes you paste in, making a packing list, explaining a paragraph, checking a message’s tone. The information is in front of the model, so it doesn’t need to remember facts, and that’s where small models do best.
• Photos work too. Current phone-sized models read a menu, a sign or a form from a photo, offline.

For the full task-by-task breakdown of where a phone model is reliable and where it isn’t, see what on-device AI can and can’t do. The rest of this post is about the verdict rather than the capabilities.

Is the phone hardware good enough yet? #

For small and mid-size models, yes, if you pick a model that fits. RAM is what decides it:

You don’t need a special AI chip for this. Chat apps built on llama.cpp run mostly on the GPU (Metal on iPhone, OpenCL on some Snapdragon phones) or the CPU. How much RAM you need has the full breakdown by model.

Battery use is real but manageable: a long chat costs about as much as a mobile game session, and short questions barely register. We look at the numbers in does local AI drain your phone battery.

Who is local AI worth it for? #

• People who want to ask about health, money, relationships or work without that text sitting on someone’s server.
• Travellers, commuters and anyone often without a signal.
• Anyone who doesn’t want another subscription.
• People curious about open models who want to compare them on their own hardware.
• Owners of older phones that will never get the manufacturer’s built-in AI features.

Who should skip it? #

• If you mainly want current information, like news, prices or opening hours, a local model can’t help. Use a search engine or a cloud assistant.
• If your phone has 3 GB of RAM or less and is several years old, the experience will be slow. It works with the smallest model, and you may find it frustrating.
• If you need the strongest possible reasoning for complex code or research, a frontier cloud model is still better.
• If you’d never use AI for anything but trivia and news, the offline part buys you nothing.

How to test it yourself in ten minutes #

You don’t need to take anyone’s word for it, ours included.

1. Install Personal LLM, a free app for iPhone and Android that runs open models on the phone with no account.
2. On the Models tab, look for the “Fits your device” badge. It’s read from your phone’s actual RAM.
3. Download Qwen 3.5 4B (2.74 GB) on Wi-Fi. It’s the app’s recommended first model. On a phone with 8 GB or more, try Qwen 3.5 9B too.
4. Turn on airplane mode.
5. Try five things you’d really use AI for: rewrite a message, summarize something you paste, explain a term, plan a meal, and ask about a photo (after adding image support).
6. Watch the tokens-per-second readout under each reply, or run the built-in benchmark, to see how fast your phone really is.

If the answers are good enough for those five tasks, it isn’t a gimmick for you. If you mostly asked it for facts and news, it probably is, and that’s a useful thing to learn in ten minutes rather than ten days.

Frequently asked questions #

Is local AI as good as ChatGPT? #

No, not overall. Cloud models are much larger and can search the web. For writing help, summaries of text you provide, and private questions, a 4B to 9B phone model is often good enough, and it keeps everything on your device.

Do local LLMs drain the battery? #

They use more power than most apps while generating, similar to a game. Short questions cost very little. Smaller models, shorter replies and leaving thinking mode off all reduce the drain.

What’s the smallest phone that can run a local LLM? #

Phones with 3 GB of RAM can run models up to about 4B parameters, slowly on older chips. For a smooth experience, 6 GB or more is better, and 8 GB opens up the 9B models.

Is local AI actually private? #

When the model runs on the phone and the app has no server, your chats and photos aren’t sent anywhere. Check each app, though: some “local” apps still collect analytics. Personal LLM uses the network only to download models and to show ads, and ads can’t load at all when you’re offline.

Will local AI get better? #

Each new generation of small open models has improved on the last at the same size, and phones keep getting more RAM. The gap with cloud models on hard tasks remains, and the list of everyday jobs a phone model handles well keeps growing.