What we are actually building
A local LLM is the reasoning and language component. The “assistant” is the software around it: the controller decides what context to send, memory stores useful facts, speech converts audio to text and back, and tools expose specific capabilities.
The LLM does not automatically have memory, microphone access, file access, or permission to control your computer. We add each capability deliberately.
The parts
Ollama + local model
Runs the language model on your CPU, GPU, or unified memory.
Python controller
Builds prompts, stores history, calls the model, and mediates tools.
SQLite
A small local database for facts and conversation metadata.
whisper.cpp
Turns microphone audio into text without a cloud speech service.
Offline TTS
Turns the answer back into audio on the machine.
Approved tools
Explicit functions for system info, files, notes, and other safe actions.
What the finished assistant can do
- Chat with you with no cloud LLM connection.
- Keep selected information in a local SQLite database.
- Read and search files you place inside an approved workspace.
- Hear speech and speak responses using offline components.
- Run a small set of tools that you explicitly allow.
- Run as a normal app, background service, container, VM, or dedicated home server.
What it cannot magically do
An offline model does not automatically know what happened five minutes ago. It cannot know live weather, stock prices, current sports scores, or breaking news unless you deliberately provide a network-connected tool. Likewise, a model's confident wording is not proof that a fact is correct.
You should be able to explain the system as: input → controller → local model → optional memory/tools → response. If that makes sense, you are ready to choose a model your hardware can actually run.