Usually not into a consumer chatbot, at least not without some work first. The American Bar Association says lawyers need a client’s informed consent before putting confidential information into a self-learning AI tool. Healthcare providers need a business associate agreement with any vendor that receives patient data, and tax preparers need client consent before most uses of return information beyond preparing the return.
The details depend on your profession and where you practice, so treat this as a map of the questions to ask, not legal advice. Your bar, licensing board or compliance officer has the final word.
Why is client data in a chatbot a confidentiality problem? #
Typing client information into a cloud AI tool is a disclosure to a third party. The provider receives it, stores it on its servers, and depending on the product and settings may:
- Use it to train future models.
- Keep it for months or years. Google, for example, retains Gemini conversations selected for human review for up to three years, according to its Gemini Apps Privacy Hub.
- Let staff or contractors read it for quality and safety checks.
- Hand it over in response to a subpoena, or preserve it under a court order.
Your professional duty doesn’t disappear because the recipient is software. The question regulators ask is whether you took reasonable steps to prevent unauthorized access to information you were trusted with.
What does ABA Formal Opinion 512 say lawyers must do? #
The ABA’s Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512 on July 29, 2024. It was the ABA’s first formal guidance on generative AI, and it applies the existing Model Rules rather than creating new ones:
- Competence (Rule 1.1). You need a reasonable understanding of what the tool can and can’t do, including its habit of making things up. Lawyers have been sanctioned for filing briefs with fake AI-generated citations, most famously in Mata v. Avianca in 2023.
- Confidentiality (Rule 1.6). Before inputting information about a representation, evaluate the risk that it will be disclosed to or accessed by others. For tools that learn from what you type (“self-learning” tools), the opinion says you need the client’s informed consent first.
- Consent has to be informed. Boilerplate buried in an engagement letter isn’t enough. The client needs to understand the specific risks and benefits.
- Communication (Rule 1.4). In some situations you must tell clients you’re using AI, such as when they ask or when it affects decisions about the representation.
- Supervision (Rules 5.1 and 5.3). Firms need policies, and supervising lawyers remain responsible for how staff and vendors use AI.
- Fees (Rule 1.5). If you bill hourly, you bill the time actually spent, not the time a task used to take. You generally can’t bill clients for time spent learning a tool you’ll use across your practice.
The Model Rules are adopted state by state, and several state bars, including California and Florida, have issued their own AI guidance. Check yours.
What about HIPAA and health information? #
If you’re a HIPAA covered entity (most healthcare providers, health plans and clearinghouses), any vendor that creates, receives, maintains or transmits protected health information on your behalf is a business associate. You need a business associate agreement with that vendor before sharing PHI. Consumer chatbot accounts don’t come with one. Some AI companies sign BAAs for specific business or API products, but pasting a patient note into a personal ChatGPT or Gemini account isn’t covered by any of them.
De-identified data falls outside HIPAA if it meets one of HHS’s standards, such as removing the 18 identifiers listed under the Safe Harbor method. In practice, clinical notes are hard to de-identify fully. Dates, rare conditions and small-town locations can point to a person without a name.
Therapists and counselors have a second layer: licensing boards and professional ethics codes set confidentiality rules that apply whether or not HIPAA does.
Rules for accountants, HR and other professionals #
| Profession | Main rule to check | What it means for AI chatbots |
|---|---|---|
| Lawyers | ABA Model Rules 1.1 and 1.6, Formal Opinion 512, your state bar | Informed consent before using self-learning tools with client information |
| Doctors, clinics, health plans | HIPAA Privacy and Security Rules | A BAA before any vendor receives PHI |
| Therapists and counselors | HIPAA if covered, plus licensing board ethics codes | Keep session content out of consumer tools |
| Tax preparers | Internal Revenue Code Section 7216 | Specific client consent before most uses or disclosures of return information |
| HR teams | Employment and privacy laws (GDPR in the EU, state privacy laws in the U.S.) and company policy | Treat employee records, complaints and medical leave details like client data |
If your field isn’t listed, the pattern is the same: find the rule that governs disclosing client information to vendors, then ask whether a chatbot provider counts as one. It almost always does.
What are your options for using AI with client data? #
You have four realistic options:
- Don’t. Use AI only for work with no client information in it: a generic template, an explanation of a concept, a rewrite of your own website copy.
- De-identify first. Remove names, numbers, dates and distinctive facts. This works for “help me structure this letter” tasks and fails when the specifics are the substance.
- Use an enterprise tool with a contract. Firm-licensed AI can come with no-training commitments, retention controls, audit logs and, where needed, a BAA. This is the standard route for firms and practices.
- Use a model that runs on your device. If the data never leaves hardware you control, no third party receives it.
| Consumer chatbot | Enterprise AI with contract | On-device model | |
|---|---|---|---|
| Where client data goes | Provider’s servers | Provider’s servers, under contract | Stays on your phone |
| Training on your data | Often by default | Usually excluded | None |
| Contract available (BAA, DPA) | No | Yes | Not needed for the model itself |
| Answer quality | High | High | Lower, since the models are small |
| Cost | Free or about $20 a month | Per-seat pricing | Free apps exist |
What an on-device model can and can’t do for confidential work #
An on-device model is the one option above where the disclosure question mostly goes away, because there’s no recipient. Personal LLM runs open models such as Qwen 3.5, Gemma 4 and Ministral 3 on your phone’s own chip. It has no account and no server, and after the one-time model download it works in airplane mode. You can attach a PDF, text or Markdown file and ask about it: the text is extracted and searched on the phone, and the model answers from the matching passages and shows which ones it used. That helps with pulling dates out of a lease, summarizing a long intake form, or drafting a plain-English explanation of a document for a client.
Know the limits before you rely on it:
- Smaller models make more mistakes. The catalog runs from 0.8 billion to 9 billion parameters, far smaller than the big cloud models. They misread documents and invent details more often, so check every output against the source.
- It isn’t a research tool. A local model has no internet access and no legal or medical database. Never cite a case, statute or dosage it gives you without verifying it yourself.
- Your device security becomes the safeguard. Use a strong passcode, keep the operating system updated, and be careful with exports. Personal LLM can back up every chat to a single JSON file, and that file is as sensitive as the chats in it.
- The free version shows ads. They come from Google AdMob, which may collect device information to serve them. Your conversations aren’t part of that, but a firm policy may still ask about any network activity. A one-time purchase removes the ads.
- Ask your regulator. Whether processing on your own device changes your consent obligations is a question for your bar’s ethics hotline or your compliance officer.
For more on what stays on the phone with local AI, see how personal data stays protected with on-device AI. If the question is about your own email rather than client files, start with is it safe to paste emails into ChatGPT.
Frequently asked questions #
Can lawyers use ChatGPT at all? #
Yes. ABA Formal Opinion 512 doesn’t ban generative AI. It requires competence, confidentiality safeguards, appropriate communication with clients and reasonable fees. Using ChatGPT for work with no client information raises few issues, while putting client information into a self-learning tool requires informed consent under the opinion.
Is ChatGPT HIPAA compliant? #
The regular consumer ChatGPT app doesn’t come with a business associate agreement, so a covered entity can’t use it with protected health information. HIPAA compliance depends on contracts and safeguards rather than the product name, so check what your organization has actually signed.
Does an on-device AI app need a BAA? #
A BAA is required when a vendor creates, receives, maintains or transmits PHI for you. If an app processes data only on your device and never sends it to the developer, the developer doesn’t receive PHI. Your device still has to meet your own security obligations, so confirm the setup with your compliance officer.
Is anonymized client data safe to paste into AI? #
Properly de-identified data carries much less risk, but data “anonymized” by hand often isn’t. Unusual facts, dates and places can identify a client without a name. When those details are what the AI needs, de-identification rarely works.
Should I tell clients I use AI? #
Opinion 512 says lawyers must disclose AI use in some circumstances, such as when the client asks or when it affects the representation. Many firms now address AI in engagement letters, but for self-learning tools used with client information, a general clause isn’t enough.