Cover of Running LLMs Locally
From Keyword to Code

Running LLMs Locally

Ollama, Quantisation and Private AI on Your Own Machine

Explore running language models on your own machine with Ollama. Understand the roles of quantisation, local hardware, and privacy when deciding how a local model fits your workflow.

View on Amazon ↗

Opens Amazon in a new tab. Check the listing for formats and availability.

A closer look

About this book.

Can you run a language model on your own computer, and explain every layer of what you built?

That question now appears in job ads for developers, and most tutorials answer it with a single command and no understanding. Running LLMs Locally takes the opposite approach. It teaches you the ideas first, then has you build a complete, fully offline document assistant, and then shows you how to measure it, share it and talk about it.

You will learn how to

  • Work out, with simple arithmetic, whether a model will fit in your memory before you download it
  • Install and run Ollama, call it from Python, and understand each part of the reply
  • Choose a quantisation level by testing it on your own questions, and read a GGUF file
  • Use llama.cpp and the OpenAI-style API so your code works with local or hosted models
  • Build a private assistant that searches your own documents and answers with citations
  • Prove that it works offline, and know exactly what that proof does not cover
  • Measure speed and quality, debug a local stack, and compare local cost with an API
  • Serve it safely to other people, and describe the work clearly in an interview

Every chapter opens with a hook, builds one idea at a time with an analogy, and closes with an AI Sidekick prompt, a practice task, a "break it on purpose" mini-challenge with a model solution, and a recap. The book also includes a capstone brief, a troubleshooting appendix and a glossary.

You need to be able to write some code and use a terminal. You do not need a data-science background or an expensive graphics card. All the models and tools used are free to download.

Written for developers, students and career-changers who want a real, demonstrable local AI project, and who want to understand it well enough to speak about it without bluffing.

Beyond the book

Reader resources.

Download chapter code (ZIP)

Includes a README with setup notes and a chapter-by-chapter file index.

Official documentation and guides for topics covered in this book.

← Back to the books