XNews

Local-first realtime speech and offline AI tools

Local-first realtime speech and offline AI tools

Open-source projects deliver local-first realtime speech and on-device LLM tools.

Key insights

WhisperLiveKit provides optimized local realtime speech recognition

WhisperLiveKit is an open-source solution that implements local realtime speech-to-text and translation with specific optimizations for streaming latency. The project includes speaker diarization and voice activity detection (VAD). Its backend supports multiple engines including faster-whisper and an Apple Silicon–optimized mlx-whisper. WhisperLiveKit integrates an NLLW model to enable real-time translation across about 200 languages, and ships with Python server and Web front-end examples plus Docker deployment artifacts for production setup.

Hyprnote and GELab-Zero enable local meeting workflows and on-device LLMs

Hyprnote is an open-source, local-first AI meeting-notes app that captures system audio and microphone input for realtime transcription while keeping all data on-device. It supports running local LLMs via Ollama or LM Studio to perform offline end-to-end tasks from speech-to-text to summary generation, and offers custom note templates and deferred editing workflows. Separately, GELab-Zero is a 4B local model reported to reach 73.4% accuracy on an AndroidDaily test set and supports mobile and PC GUI scenarios, suggesting feasible on-device LLM GUIs for meeting or assistant applications.

Source posts