Use whisper.cpp for local transcription
whisper.cpp is a lightweight C/C++ implementation of Whisper that can run locally across Windows, macOS, Linux, Raspberry Pi, and other environments. It works on CPU and supports several acceleration options.
Build the speech engine
The exact package manager commands vary by operating system, so use the upstream project instructions for your platform. The basic source-build flow is:
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
cmake -B build
cmake --build build --config ReleaseDownload a speech model and test it
Start with the small English base.en model. The upstream project includes download helpers for both Unix-like systems and Windows.
Linux / macOS
sh ./models/download-ggml-model.sh base.en
./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wavWindows PowerShell
.\models\download-ggml-model.cmd base.en
.\build\bin\Release\whisper-cli.exe -m .\models\ggml-base.en.bin -f .\samples\jfk.wavIf your CMake generator places the executable directly under build/bin instead of build/bin/Release, use that location. A successful run should print a transcription of the included sample audio.
Your LLM and your speech recognizer are separate components. If a different offline speech engine fits your hardware better, you can swap it without changing the rest of the assistant architecture.
Start with push-to-talk
Do not make wake-word detection your first audio feature. A push-to-talk flow is easier to test:
Hold key → record WAV → whisper.cpp → text → Python controller → local LLM
Once that path is reliable, a later improvement can continuously listen for a wake phrase and only begin full transcription after the phrase is detected.
Record a short sentence and produce a usable text transcription without calling a cloud speech API.