A language model running on your phone can write, rewrite, summarize, explain, translate everyday text, help with code and describe photos, all without an internet connection. It can’t look anything up, it doesn’t know about recent events, and it will sometimes state wrong facts with total confidence.
Knowing where that line sits is the difference between finding local AI useful and giving up on it after a day. Below are the most common misconceptions, each with what’s actually true for the phone-sized open models people run today, such as Qwen 3.5 (0.8B to 9B parameters), Gemma 4 E2B and E4B, Ministral 3 3B and GLM 4.6V Flash.
Misconception 1: “A phone can’t run a real AI model” #
It can. Two things made this possible. Phones got more memory (the iPhone 15 Pro and later, and many recent Android flagships, have 8 GB or more). And model makers began releasing small models built for this job, then shrinking them further with quantization, which stores each weight in about 4 bits instead of 16.
A 4-billion-parameter model ends up as a file of roughly 2.5 to 3 GB and runs at reading speed on a recent phone. Even a 9B model fits on phones with 8 GB of RAM. Our guide to how much RAM you need has the numbers for each size.
Misconception 2: “Local AI is as smart as ChatGPT” #
It isn’t, and pretending otherwise sets you up for disappointment. The flagship cloud models are much larger than anything that fits in phone memory. What a local model gives up is mostly:
- Knowledge of rare facts. Small models know the broad strokes of history, science and language well. They’re shaky on specific numbers, niche people, minor events and exact quotes.
- Long reasoning chains. A multi-step math problem or a tricky logic puzzle is where small models slip. Thinking mode helps: the model works through the problem step by step before answering, which catches many mistakes at the cost of speed.
- Very long inputs. Phone memory limits how much text the model can hold at once.
For everyday tasks, a good 4B or 9B model is close enough that you often won’t notice. Whether that trade is worth making for you is a separate question, argued out in is local AI on your phone a gimmick.
Misconception 3: “It knows what’s happening today” #
A local model’s knowledge stops at its training cutoff, and it has no way to check the web. Ask about this week’s news, today’s weather, a live price, or a product released last month, and it will either say it doesn’t know or invent something plausible.
Fix: give it the information. Paste the article, attach the PDF, or photograph the page, then ask your question about that material. Models are far more reliable working from text in front of them than from memory.
Misconception 4: “If it sounds confident, it’s right” #
Language models generate the most likely next words. Fluency isn’t evidence of accuracy. Small models in particular will produce a citation that doesn’t exist, a statistic with a decimal point, or a law that sounds right and isn’t.
Use local AI for tasks where you can judge the output yourself: drafts you’ll edit, explanations you can sanity-check, ideas you’ll filter. For health, legal, tax or financial questions, treat the answer as a starting point for a conversation with a professional or a primary source, never the final word.
Misconception 5: “On-device means it can’t be hacked or misused” #
Local processing removes one big risk: your conversations aren’t sitting on someone else’s server, so they can’t leak from it. It doesn’t make your phone invulnerable. If someone gets into your phone, they can read your chat history the same way they could read your messages. And a model file you download from an unknown source is still a file from an unknown source. We cover this in how on-device AI affects app security.
Misconception 6: “Every local AI app is private” #
“Runs on your device” describes where the model runs, not what else the app does. Some local AI apps still send analytics, crash logs or ads traffic. Some are hybrids that quietly fall back to a cloud model for hard questions. Read the privacy policy and the store’s privacy label, and test it: put the phone in airplane mode and see whether it still answers. How to tell if an AI app is really private has a full checklist.
Misconception 7: “It learns from me over time” #
Most local chat apps don’t train the model on your conversations. The model file stays exactly as you downloaded it. What feels like memory is the conversation history within a chat, which the model rereads each turn, plus any system prompt you set. Start a new chat and it starts fresh. For privacy that’s a feature: nothing you say is being baked into the model.
What on-device AI does well: a quick reference #
| Task | How well a 4B to 9B local model handles it |
|---|---|
| Rewriting, proofreading, changing tone | Very well |
| Summarizing text you provide | Well, within the model’s context limit |
| Explaining concepts, study help | Well; check key facts |
| Brainstorming and outlines | Very well |
| Translating everyday phrases | Well for major languages, weaker for rare ones |
| Short code snippets and explanations | Decent; test the code |
| Describing photos and reading signs or menus | Well, with a vision-capable model |
| Answering questions about a document you attach | Well for specific questions, weaker for “summarize everything” |
| Exact facts, figures and citations | Unreliable |
| Current events, prices, weather | Can’t do it |
| Long multi-step math | Hit and miss; thinking mode helps |
How to get the most out of a local model #
- Pick the biggest model your phone runs comfortably. The jump from a 1B to a 4B model is dramatic. The jump from 4B to 9B is smaller and still noticeable.
- Give it the source material. Paste, attach or photograph rather than asking it to remember.
- Turn on thinking mode for logic and math. Leave it off for chat; it’s slower.
- Use a system prompt. “You are a concise editor. Reply in bullet points.” changes the output more than you’d expect. See how to get better answers from a local AI.
Personal LLM is set up around these habits. Its catalogue runs from Qwen 3.5 0.8B for older phones up to Qwen 3.5 9B and GLM 4.6V Flash for phones with 6 to 8 GB of RAM or more, and each card shows whether it fits your device before you download. You can attach a PDF, text or Markdown file and it answers from the matching passages, telling you which ones it used. Thinking mode shows its reasoning in a collapsible panel so you can check the working, and a live tokens-per-second readout shows how hard your phone is working.
Frequently asked questions #
Can on-device AI access the internet? #
No. A local model has no browsing ability. It works only from what it learned in training and what you give it in the chat. That’s what makes it private, and it’s also why it can’t tell you about current events.
Why does my local AI make things up? #
Small models store less knowledge, so when they reach the edge of what they know they fill the gap with plausible-sounding text. Giving the model the source text, using a larger model, and asking it to say when it isn’t sure all reduce this.
Can a phone AI understand pictures? #
Yes, if the model supports vision. Many current small models do, including Qwen 3.5, Gemma 4, Ministral 3 and GLM 4.6V Flash. They describe a scene, read a sign or a menu, and answer questions about a screenshot. Fine details and small text are harder for them.
Can on-device AI control my phone or apps? #
Not from a chat app. A local model reads what you type and attach and writes an answer back. Setting alarms, sending messages and opening apps are things the built-in assistants do, and they need the permissions that come with it.
Is on-device AI good enough to replace ChatGPT? #
For private, offline and everyday writing tasks, many people find it is. For deep research, current information or the hardest reasoning problems, cloud models are still ahead. Plenty of people use both.