What Is Local AI? A Plain Explainer for Lawyers

The distinction that actually matters: where the model lives

Every AI tool has two separate things going on: an interface (the app you're typing into) and a model (the thing that actually reads text and generates a response). With a cloud tool - ChatGPT, Claude, most AI products you've used - the interface can be on your laptop, but the model lives on the vendor's servers. Every question you ask, and every document you attach, travels over the network to get there.

With local AI, the model itself is downloaded and installed on your device. When you ask a question or point it at a document, the entire process - reading the file, running it through the model, generating the answer - happens on hardware you own, with no network request involved in the AI step at all. That's the whole distinction, and it's a bigger deal than it sounds: it's not a privacy setting, it's a difference in where the computation physically happens.

Why this isn't just "offline mode"

It's worth being precise here, because a lot of software already claims to work "offline" without doing anything like this. A note-taking app might cache your notes locally and sync them later - useful, but that's not what local AI means. Some AI-adjacent tools let you view previously-fetched results without a connection, while the AI processing itself still requires a live connection to a cloud model when you actually use it.

True local AI has a simple test: disconnect from the internet entirely and try to use the feature. If the AI still reads your document and answers your question, the model is running on your device. If it stops working, or falls back to a degraded mode, the "AI" part was never local - only the interface was.

What makes this possible now, and what it costs

Local AI runs on models that are smaller than the largest cloud models - a practical requirement, since your laptop doesn't have the server farm of specialized hardware that a frontier cloud model runs on. Getting a capable model small enough to run well on a Mac or PC is a real engineering trade-off, which is why local AI tools typically offer a few model tiers, sized for different hardware and different needs: a lighter model for older machines or simple tasks, a larger one for more demanding work if your hardware supports it.

The trade-off is capability at the margins, not capability across the board. For narrow, well-defined tasks - read this document, answer questions about it, summarize this section - a modern local model does the job well. It's less the right tool for open-ended requests that benefit from a frontier model's broader general knowledge. The practical question for a solo attorney isn't "which is smarter in the abstract," it's "is this model good enough at the specific, scoped task I'm using it for" - and for document review and Q&A, increasingly, yes.

Why this matters specifically for legal work

The core problem cloud AI creates for confidential material is structural, not a matter of vendor trustworthiness: to get an answer, the document has to be sent somewhere. That transmission is what turns "I asked an AI a question" into "I disclosed a client document to a third party" - and it happens regardless of how careful or well-intentioned the vendor is, because it's how the architecture works, not a policy choice either side controls after the fact.

Local AI removes that step entirely rather than managing it more carefully. If the document never leaves your device, there's no transmission to create a disclosure question, no vendor data-retention policy to evaluate, and no need to trust anyone else's infrastructure with privileged material. That's a different kind of answer than "we have strong security" - it's "there's nothing to secure in transit because nothing was ever sent."

What this looks like in practice

Privileged is built on exactly this architecture: a local-first document analysis and Q&A tool for solo attorneys, running entirely on your own Mac via Ollama. Documents are organized by matter, and the two model tiers - Standard and Heavy - are sized for different hardware so the tool works whether you're on an older machine or a recent one. There's no cloud call anywhere in the document-reading path, no data transmission, and nothing to verify beyond disconnecting the network and confirming it still works - because it will.

If you're weighing whether this is a durable approach or a stopgap until cloud tools get comfortable enough with confidentiality, that's a fair question, and worth walking through on its own.