Many of the most useful accessibility features on iPhone and Android now use AI that runs on the phone itself. VoiceOver and TalkBack read your screen aloud, Live Captions turn speech into text in real time, and camera tools can describe what’s in front of you. Because the processing happens on the device, much of this works offline and keeps what the phone hears and sees private. A local AI model can add to that: describing a photo in detail, explaining a confusing letter in plain language, or reading an answer aloud, all without an upload.
Features vary by phone, OS version and language, so check what your device supports.
Why on-device AI matters for accessibility #
Assistive technology is used all day, in every setting, and often on very personal content: medical letters, bank statements, private conversations. Three things make on-device processing a better fit:
- It works without signal. Captions and screen reading shouldn’t stop in a basement or on a flight.
- It’s faster. No round trip to a server means less delay for captions and descriptions.
- It’s private. What your camera sees and your microphone hears stays on the phone.
Built-in features on iPhone #
- VoiceOver: Apple’s screen reader, which speaks what’s on screen and lets you control the phone with gestures. It can also describe images that have no text labels.
- Live Captions: real-time captions for audio on the phone, including calls and videos, as well as conversations around you. Apple processes these on the device.
- Magnifier: uses the camera to zoom in, and on supported iPhones can detect people, doors and text nearby.
- Live Text: recognizes text in photos and the camera view so it can be copied, spoken or translated.
- Spoken Content: reads selected text or the whole screen aloud.
- Voice Control: operate the phone entirely by voice.
- Apple Intelligence Writing Tools on iPhone 15 Pro and later can summarize or simplify long text, many requests on the device.
Built-in features on Android #
- TalkBack: Google’s screen reader, with spoken feedback and gesture control.
- Live Caption: captions any audio playing on the phone, processed on the device.
- Live Transcribe: real-time transcription of speech around you; offline use depends on the phone and language.
- Lookout: a Google app that uses the camera to read text, identify currency and describe surroundings, with some modes working offline.
- Voice Access: control the phone by voice.
- Gemini Nano features on supported phones, including image descriptions and summaries, run on the device, according to Google’s developer documentation.
What local AI adds #
Built-in tools are great at their specific jobs. A local language model is a flexible helper you can ask anything, which fills some useful gaps. Personal LLM runs models on your phone, with no account and no uploads, and a few of its features are relevant here:
- Detailed image descriptions. Add a model’s image-support file (195 MB to 1 GB) and ask about any photo: “Describe this in detail,” “What does this form ask me to fill in?”, “Read the text on this label.” It works offline. See asking AI about a photo offline.
- Read aloud. Any answer can be spoken with on-device text-to-speech.
- Plain-language rewrites. Paste a confusing letter and ask: “Explain this in simple words,” “What do I need to do, and by when?”
- Document questions. Attach a PDF, text or Markdown file and ask specific questions, instead of reading it end to end. The app answers from the matching passages and tells you which ones it used.
- Adjustable output. A per-chat system prompt can ask for short sentences, bullet points or a set reading level every time.
It works with your phone’s screen reader like any other app, and it runs on phones that don’t qualify for Apple Intelligence or Gemini Nano.
Useful prompts for accessibility #
- “Describe this photo for someone who can’t see it. Start with the most important thing.”
- “Read all the text in this image, in order.”
- “Rewrite this in plain English at a middle-school reading level.”
- “Summarize this letter in three sentences, then list what I need to do.”
- “Break these instructions into numbered steps, one action per step.”
- “Explain this bill: what’s the total, and when is it due?”
A system prompt can lock in the format:
Use short sentences and simple words. Put the most important information first. Use numbered steps for any instructions.
More ideas are in our system prompt examples.
Limits to keep in mind #
- AI descriptions can be wrong. Vision models can miss details or describe things that aren’t there. Don’t rely on them for safety decisions like medication identification or reading traffic signals.
- Small models simplify imperfectly. A plain-language rewrite can drop an important detail. For legal and medical letters, check key points with a person you trust.
- Features vary by device. Some built-in tools need specific hardware or languages.
- Speed. On older phones, a local model may take several seconds per answer. Smaller models are faster; see running AI on an old phone.
Cloud tools vs on-device #
Cloud-based assistive services, such as AI description apps that upload photos, can use larger models and are often more detailed. They also need a connection and send your images to a server. Which is better depends on the moment: a detailed description of a public scene may be worth the upload; a photo of a prescription or a bank letter may not.
| On-device tools | Cloud tools | |
|---|---|---|
| Works offline | Yes | No |
| Images and audio stay private | Yes | Uploaded |
| Detail and accuracy | Good, smaller models | Often higher |
| Speed | No network delay | Depends on connection |
Frequently asked questions #
Do iPhone accessibility features work offline? #
Many do. VoiceOver, Magnifier, Live Text and Live Captions are processed on the device on supported iPhones. Features that rely on web services or cloud AI need a connection.
Can AI describe images for blind users offline? #
Yes. Screen readers can describe some images on the device, and local AI apps with vision models can give detailed descriptions of any photo with no internet. Accuracy isn’t perfect, so treat descriptions as helpful, not definitive.
Is Live Caption private? #
On Android, Google says Live Caption processes audio on the device, and Apple processes iPhone Live Captions on the device too. The audio isn’t sent to a server for captioning.
Can a local AI read text aloud? #
Yes, in apps that include text-to-speech. Personal LLM reads any answer aloud with on-device speech, and your phone’s built-in Spoken Content or Select to Speak can read any text on screen.
Which is better for accessibility, cloud or on-device AI? #
Each has a place. On-device AI is private, works offline and responds quickly. Cloud tools can be more detailed but need a connection and upload your content. Many people use on-device tools by default and cloud tools when they need more detail.