AI on Your PC
If you prefer to run AI on your own computer – and why that's becoming increasingly important.
Cloud AI is convenient – but every question, every photo, every document goes to the provider. For sensitive data or simply as a matter of principle: AI also runs locally on your PC. Here we show how it works, what you need, and which models get the most out of it.
📋 Page Contents
Why Local AI?
Cloud AI has advantages: no setup, always up-to-date, best models. But local AI has five good reasons for itself:
- Data Protection: Your data never leaves your PC. Family photos, business documents, personal thoughts — everything stays with you.
- No Recurring Costs: Once set up, it costs nothing more. No subscriptions, no token limits.
- Offline Usable: Train, plane, remote locations — AI even without internet.
- No Censorship, No Limits: Local models don't refuse to answer creative writing prompts or avoid sensitive topics.
- Learning: Those who set up local AI themselves gain a much better understanding of what's happening under the hood.
When is it not worth it?
- If you rarely use AI — the setup effort isn't worth it.
- If you have old hardware without a powerful GPU — see "AI on Older PCs".
- If you want maximum quality — cloud models are often (still) better.
- If you have multimodal tasks (images, audio, video) — cloud tools are more advanced here.
Setting up AI Locally on Windows
The setup is easier than you think. Three tools dominate the field:
The favorite for pros. Console-based, but very stable. One line in PowerShell, and you have an AI model running. Huge model library (Llama, Mistral, Phi, Qwen).
Graphical interface. More comfortable for beginners. You click models, download them, and start chatting directly. Can also be used from other apps with a web server function.
Very beginner-friendly. Open source. Models and chat interface in one app. Works surprisingly well even on older computers.
The newcomer. A ChatGPT-like interface that runs completely locally. Open source, very well-designed.
Setup with LM Studio (easiest entry)
- Download LM Studio Go to lmstudio.ai → download Windows installer → install. 5 minutes.
- Select Model In LM Studio, search for "Llama 3" or "Mistral". The app will show you which models fit your hardware.
- Load Model Click "Download". Size 2–40 GB depending on the model. Wait 10–30 minutes.
- Start Chat Interface Select the model, click "Load model", and start chatting in the chat tab.
- Optional: Web server for other apps If you want to use the model from other programs, start the local server in LM Studio — compatible with the OpenAI API.
Hardware Requirements (Rule of Thumb)
| Model Size | Recommended | Example Models |
|---|---|---|
| Small (≤7 billion parameters) | 16 GB RAM, integrated GPU or weak dedicated GPU | Llama 3 8B, Phi 3 Mini, Mistral 7B |
| Medium (13–34 billion parameters) | 32 GB RAM, RTX 3060+ with 12 GB VRAM | Llama 3 70B (quantized), Mixtral |
| Large (70 billion parameters) | 64 GB RAM, RTX 4090 or multiple GPUs | Llama 3 70B (full precision) |
| Huge (200+ billion parameters) | Pro hardware with lots of VRAM, possibly cloud | DeepSeek-V3, Llama 3 405B |
💡 Quantization — the trick for older hardware: Most models are available "quantized" — i.e., with reduced precision. A 70B model quantized to 4-bit only needs 35 GB instead of 140 GB. Works with minimal loss of quality and makes models runnable on normal hardware.
Local AI Models Compared
The most important open-source models in 2026:
The most popular open-source model. From 1B (tiny) to 405B (huge). Very good for general tasks, excellently documented. First choice for most.
From France. Efficient: With 7B parameters, often better than others with 13B. Mixtral uses "Mixture of Experts" — very powerful, very fast.
From China. Caused a stir in 2025/26: top performance at a fraction of the training costs. Very strong in programming and reasoning.
Very small (3–14B), but surprisingly capable. Specifically designed for local setup on standard laptops. Typically good for Office tasks, as expected from Microsoft.
Very strong in multilingual tasks (Chinese, English, German). Multimodal variants available (text + image).
Google's open-source family. Small and efficient. Good if you're looking for an established, well-documented solution.
Which model for what?
- All-rounder, beginner: Llama 3 8B (fits on any reasonably modern hardware)
- Programming: DeepSeek Coder or Qwen Coder
- Long texts / analysis: Mixtral or Llama 3 70B (quantized)
- Weak hardware: Phi 3 Mini or Llama 3 1B
- Maximum quality, lots of VRAM: Llama 3 70B unquantized or DeepSeek-V3
📊 Open Source is catching up fast
Two years ago, proprietary models (ChatGPT, Claude) were clearly superior. Now, the gap with Llama and DeepSeek is only a few percentage points. For most private users, open-source models are perfectly sufficient — and unbeatable when it comes to data privacy.
Getting AI to run on older PCs
You don't have to buy an RTX 4090 workstation to use AI locally. Older computers can do it too — with the right models and settings.
What works on older hardware?
- 4 GB RAM, old CPU, no GPU: Small models like Phi 3 Mini or Llama 3.2 1B. Response time 5–30 seconds per answer.
- 8 GB RAM, 4-core CPU: Llama 3 8B with 4-bit quantization. Usable for everyday tasks.
- 16 GB RAM, old GPU with 4 GB VRAM: Even medium models run — with patience.
Tips for more performance
- Latest GPU drivers: Outdated drivers leave a lot of performance on the table — especially with NVIDIA cards, the difference can be 30%+
- Use quantized models (4-bit or 5-bit) — more speed, hardly any quality loss
- Use small models selectively — Phi 3 for Office, Llama for general tasks
- Close background apps while using AI — free up RAM and CPU
- SSD instead of HDD — models load 10–20× faster
🟡 Engelmann Help for AI Performance
Local AI relies heavily on up-to-date GPU drivers. With NVIDIA cards, an outdated driver can cost up to 30% performance – in AI inferencing, that makes the difference between "smooth" and "stuttering."
Our Driver Updater automatically finds outdated GPU drivers and safely updates them with original manufacturer drivers. If you use AI locally, this is the easiest performance optimization.
AI and Data Privacy
Anyone who uses AI discloses data – whether they want to or not. If you use cloud AI, you should at least know what goes where.
What happens to your data with cloud AI?
- OpenAI (ChatGPT): By default, conversations are used for model improvement — can be disabled in settings. For Pro/Team/Enterprise plans, data is not used for training.
- Anthropic (Claude): Data is not used for training by default — Claude is one of the most privacy-friendly here.
- Google (Gemini): Data can be used for model training and advertising. Very similar to other Google services.
- Microsoft (Copilot): Business and Enterprise data are handled separately — private customer data goes into training.
What you can do
If data privacy is absolutely paramount, there's no "better" than local models. Ollama, LM Studio, GPT4All — see above.
Anonymize sensitive data before prompting. Replace real names and identifiers with placeholders. No medical records, salaries, passwords.
With OpenAI, Anthropic, etc., Pro/Team/Enterprise data is not used for training. Small surcharge, much better data privacy.
In the settings of all major AI providers, you can (often deeply hidden) deactivate training with your own data. It's worth it.
Windows Spying alongside AI
Even without AI, Windows 10/11 collects a lot of telemetry data – this issue gains new urgency in the age of AI, because Microsoft also trains AI models with user data.
🟡 Engelmann Tip: AntiSpy for Windows
Anyone who wants to use AI locally often also has an interest in preventing Windows from sending data to Microsoft in the background. Our AntiSpy deactivates telemetry and unnecessary data transfers — without breaking Windows.
Recommendation: If you're setting up local AI, it's the perfect opportunity to make your Windows setup more privacy-friendly.
Golden Rules for AI & Data Privacy
- Local AI for sensitive data Family photos, personal letters, tax documents, business documents — never enter into cloud AI.
- Cloud AI for general tasks Brainstorming, research, general texts — Cloud is OK. But consciously.
- Anonymize before prompting Replace real names, addresses, amounts in prompts with placeholders.
- Check settings Check the privacy settings of every AI tool. Activate training opt-out.
- Cost-benefit analysis Pro accounts (often €20/month) are significantly better for data privacy than free versions. If AI becomes important, it's worth it.
Engelmann Tools for your AI Setup
Driver Updater keeps GPU drivers up-to-date – important for local AI performance. AntiSpy disables Windows telemetry.