Perfect for: Lawyers, doctors, security researchers, or anyone working with sensitive data who doesn’t want it leaving their computer.
The Concern
You’re working with:- Client confidential information
- Medical records
- Proprietary business data
- Personal financial documents
- Security research
How Local Models Work
Instead of sending your data to the internet, you download an AI model to your Mac (like downloading a large app) and run it locally. Your data never leaves your computer.100% Private
Nothing sent to the cloud. Your data stays on your Mac.
Works Offline
Use AI on airplanes, in secure facilities, or anywhere without internet.
No API Costs
Once downloaded, no per-use fees. Analyze thousands of documents for free.
Faster for Big Files
No upload/download time. Great for large documents.
Setup: Two easy options
Option 1: Ollama (Recommended for beginners)
Ollama makes running local models as simple as installing an app.Install Ollama
- Download from ollama.com
- Install it like any Mac app
- It runs in your menu bar
Download a model
Open Terminal and run:This downloads Meta’s Llama 3.1 model (~4.7GB). Other good options:
ollama pull mistral(faster, smaller)ollama pull codellama(great for code)ollama pull mixtral(more capable, larger)
Connect to Alter
- Open Alter → Settings (
Cmd + ,) - Go to API Keys tab
- Under Custom Provider, select Ollama
- Make sure Ollama is running (check your menu bar)
- Toggle Enable Custom Provider ON
Option 2: LM Studio (More control)
LM Studio gives you a graphical interface to manage models.Install LM Studio
Download from lmstudio.ai and install it.
Download a model
- Open LM Studio
- Browse the model catalog
- Download one that fits your needs and Mac’s specs
- Start the local server (big “Start Server” button)
Real-world example
Dr. Chen, physician:“I need to analyze patient notes for research, but I can’t use cloud AI due to HIPAA. With a local model, I can ask ‘What patterns do you see in these symptoms?’ and get AI assistance while keeping everything on my secure laptop.”Alex, security researcher:
“I analyze malware reports and can’t upload them anywhere. Running a local model means I can ask ‘What indicators of compromise are mentioned?’ without risking data exposure.”
Trade-offs to know about
Speed vs Privacy
Speed vs Privacy
Cloud models: Faster, more capable
Local models: Slower, but completely privateA MacBook Pro with 16GB RAM can run small models smoothly. For larger models, you’ll want 32GB+ RAM.
Capability differences
Capability differences
Local models are getting better every month, but cloud models (GPT-4, Claude) are still more capable for complex reasoning.Best approach: Use local models for sensitive data, cloud models for less sensitive complex tasks.
Storage requirements
Storage requirements
Models range from 4GB to 70GB+. Make sure you have enough disk space.Good starter models:
- Llama 3.1 8B (~4.7GB) – Fast, decent quality
- Mistral 7B (~4.1GB) – Good balance
- Mixtral 8x7B (~26GB) – Higher quality, needs more RAM
Best practices for private workflows
When to use what
| Scenario | Recommendation |
|---|---|
| Medical records | Local model (Ollama/LM Studio) |
| Legal documents | Local model |
| Security research | Local model |
| Financial analysis | Local model or Alter Cloud with Pro plan |
| Creative writing | Cloud models (better creativity) |
| General questions | Cloud models (faster) |
| Code assistance | Either works well |
Related resources
- How to use your own API key – Step-by-step setup guide
- Choosing AI models – Compare local vs cloud options
- API Keys settings – Detailed configuration
Ready to go private? Start with Ollama and the Llama 3.1 model. It’s free, easy to set up, and you’ll have AI assistance that never leaves your Mac!