Articles tagged "local-ai"
5 min readIntermediate
Run a 70B LLM on a 4GB GPU: Layer-by-Layer Inference with AirLLM (2026)
Run a 70B language model on a 4GB GPU with AirLLM. Layer-by-layer inference needs no quantization, distillation or pruning. Covers install, code, the VRAM table for 70B/405B/671B models, disk and speed trade-offs, and when llama.cpp offloading is the better choice.
6 min readBeginner
Ollama on Linux: Run Local LLMs, Manage Models and Use the API (2026)
Complete Ollama guide for Linux: install, run LLMs locally, manage models, use the REST API, Python integration, and GPU acceleration with NVIDIA or AMD.