Local & Private AI
Run AI on your own machine: the privacy thesis and the hardware math.
-
What AI model can my computer actually run?
Rough rule: a local model needs about half its parameter count in gigabytes of memory once it's quantized to 4-bit. So 8GB of VRAM comfortably runs a 4–8B model, 16GB runs a 14B, 24GB runs a 32B, and 48GB+ gets you into 70B territory. On a Mac, the number that matters is your unified memory.
-
How do I run a private, local AI assistant on my own device?
To run a private AI assistant locally, you need two layers: a runtime that loads the model (like Ollama) and a friendly app on top (like LM Studio or Jan). Pick a model that fits your machine's RAM and VRAM. Start with one small model and one chat app, and only connect it to your own documents once you trust it. You get full privacy, no per-use fees, and it works offline.