Welcome to Voice Chapter 11 🎉, our long-running collection the place we share all the important thing developments in Open Voice. On this chapter, we are going to inform you how our assistant can now management extra issues within the residence, in a number of languages on the identical time, all whereas not speaking your ear off. What’s extra, our record of supported languages has grown once more with a number of languages that large tech’s voice assistants received’t assist. Be a part of us for a deeper have a look at this voice chapter in our livestream
Multilingual assistants
Our unique objective for the 12 months of Voice again in 2023 was to “let customers management Residence Assistant in their very own language”. We’ve come a great distance in direction of that objective, and actually broadened our language assist. We’ve additionally supplied choices that permit customers to customise voice assistant pipelines with the companies that greatest assist their language, whether or not run domestically or within the cloud of their alternative. However what in case you communicate two languages inside your private home?
For a while, customers have been in a position to create Help voice assistant pipelines for various languages in Residence Assistant, however interacting with the completely different pipelines has both required a number of voice satellite tv for pc units (one per language) or some form of automation set off to modify languages
Since even the tiniest voice satellite tv for pc {hardware} we assist is able to operating a number of wake phrases now, we’ve added assist in 2025.10 for configuring as much as two wake phrases and voice assistant pipelines on every Help satellite tv for pc! This makes it easy to assist twin language households by assigning completely different wake phrases to completely different languages. For instance, “Okay Nabu” might run an English voice assistant pipeline whereas “Hey Jarvis” is used for French.
A number of wake phrases and pipelines can be utilized for different functions as properly. Wish to hold your native and cloud-based voice assistants separate? Simple! Assign a wake phrase like “Okay Nabu” to a completely native pipeline utilizing our personal Speech-to-Phrase and Piper
We’d love to listen to suggestions on how you propose to make use of a number of wake phrases and voice assistants in your house!
Voice with out AI
The entire world is engulfed in hype about AI and including it to all of the issues — we’re not precisely quiet concerning the cool stuff we’re doing with AI. Whereas powering your voice assistants with AI/LLMs makes them rather more versatile and highly effective, it comes at a price: paying to make use of cloud-based companies like OpenAI and Google, or expensive {hardware} and power to run native fashions by way of methods like Ollama. We began constructing our voice assistant earlier than AI was a factor, and thus it was designed with out requiring it. We proceed to make nice progress in direction of delivering a stable voice expertise to customers who need to hold their residence AI free — conserving AI opt-in solely and never required
Help, our built-in voice assistant, can do lots of cool issues with out the necessity for AI! This contains a ton of voice instructions in dozens of languages for:
- Turning lights and different units on/off
- Opening/closing and locking/unlocking doorways, home windows, shades, and many others
- Adjusting the brightness and coloration of lights
- Working scripts and activating scenes
- Controlling media gamers and adjusting their quantity
- Enjoying music on supported media gamers by way of Music Assistant
- Beginning/stopping/pausing a number of timers, optionally with names
- Including/finishing objects on to-do lists
- Delaying a command for later (“flip off lights in 5 minutes”)…
- …and extra!
Wish to embrace your individual voice instructions? You’ll be able to shortly add customized sentences to an automation, permitting you to take any motion and tailor the response.
The best strategy to get began is with Residence Assistant Voice Preview Version, our small and easy-to-start with Voice Assistant {hardware}. This, mixed with a Residence Assistant Cloud subscription, permits any Residence Assistant system to shortly deal with voice instructions, as our privacy-focused cloud processes the speech-to-text (turning your voice into textual content for Residence Assistant) and text-to-speech (turning Residence Assistant’s response again into voice). That is all with out the usage of LLMs, and helps the event of Residence Assistant 😎.
For customers wanting to maintain all voice processing native, we provide add-ons for each speech-to-text and text-to-speech:
All of this collectively reveals simply how a lot might be carried out with no need to incorporate AI, though it will possibly do some fairly superb issues
Extra intents
Intents are what join a voice command to the suitable actions in Residence Assistant to get one thing carried out. Whereas the top result’s usually easy, similar to turning on a lightweight, intents are designed as a “do what I imply” layer above the extent of fundamental actions. Within the earlier part, we listed the kinds of voice instructions that intents allow, from turning on lights to including objects to your to-do record. Over the past three years, we’ve been progressively including new and extra complicated intents.
Not too long ago, we’ve added three new intents to make Help even higher. To manage media gamers, now you can set the relative quantity with voice instructions like “flip up the amount” or “lower TV quantity by 25%”. This provides to the present quantity intent, which lets you set absolutely the quantity stage like “set TV quantity to 50%”.
Subsequent, it’s now attainable to set the pace of a fan by proportion. For instance, “set desk fan pace to 50%” and even “set followers to 50%” to focus on all followers within the present space. Be sure you expose the followers you need Help to have the ability to management.
Lastly, now you can inform the children to “get off your garden” as a result of your robotic goes to mow it! Making use of the lawn_mower integration, your voice assistant can now perceive instructions like “mow the garden” and “cease the mower”. Paired with the present good vacuum instructions, chances are you’ll by no means must raise a finger once more to maintain issues clear and tidy.
Ask query
Image this: you come residence from work and, as you enter the lounge, your voice assistant asks what sort of music you’d like to listen to whereas making ready dinner. Because the music begins to play, it mentions you left the storage door open and needs to know in case you’d prefer it closed. After dinner, as you’re hanging out on the sofa, your voice assistant informs you that the temperature exterior is decrease than your AC setting and asks for affirmation to show it off and open the home windows.
Certainly you’d want a robust LLM to carry out such wizardry, proper? With the Ask Query motion, this could all be carried out domestically utilizing Help and some automations!
Inside an automation, the Ask Query motion means that you can announce a message on a voice satellite tv for pc, match the response in opposition to an inventory of attainable solutions, and take an motion relying on the person’s reply. Whereas solutions might be open-ended, similar to a musical artist or style, limiting the attainable solutions means that you can use the absolutely native Speech-to-Phrase for recognizing speech with out an web connection.
Improved sentence matching
Help was designed to run quick and absolutely offline on {hardware} just like the Raspberry Pi 4 for a lot of completely different languages. It really works by matching the textual content of your voice instructions in opposition to sentence templates, similar to “activate the {identify}” or “flip off lights within the {space}”. Whereas that is very quick and easy to translate to many languages
Beginning in Residence Assistant 2025.9, we’ve included an improved “fuzzy matcher” that’s significantly better at dealing with further phrases or various phrasings of our supported voice instructions.
The fuzzy matcher is pre-trained on the present sentence templates, so we will use it for all of our supported languages. Nonetheless, that is initially solely obtainable for the English language and we’re working to find out the easiest way to allow this for different languages.
Non-verbal confirmations
After a voice command, Help responds with a brief affirmation like “Turned on the lights” or “Brightness set”. This allows you to realize it understood your command and took the suitable actions. Nonetheless, in case you’re in the identical room because the voice assistant, this affirmation is redundant; you’ll be able to see or hear that acceptable actions have been taken.
Beginning with Residence Assistant 2025.10, Help will detect if the voice command’s actions all came about inside the identical space because the satellite tv for pc gadget. If that’s the case, a brief affirmation “beep” might be performed as an alternative of the total verbal response. In addition to being much less verbose, this additionally serves as a reminder that your voice command solely affected the present space.
Non-verbal confirmations won’t be utilized in voice assistant pipelines with LLMs, because the person could have particular directions of their immediate, similar to “reply like a pirate”, and we wouldn’t need to deprive you of a enjoyable response, me mateys 🏴☠️.
Textual content-to-speech streaming
Giant language fashions (LLMs) might be particularly verbose of their responses, and we shortly realized that this uncovered a weak point in Residence Assistant’s text-to-speech (TTS) implementation. For many of its life, TTS in Residence Assistant has required the total response to be generated earlier than any audio might be performed. This meant lots of ready for multi-paragraph LLM responses, particularly with native TTS methods like Piper.
Fixing this required an overhaul of the TTS structure to permit for streaming. As a substitute of ready for your entire audio message to be synthesized earlier than enjoying, we enabled TTS companies inside Residence Assistant to work with chunks of textual content (enter) and audio (output). As chunks of textual content are streamed in from an LLM, the TTS service can synthesize audio chunks and ship them out to be performed instantly.
To show the advantage of streaming, we requested an LLM to “inform me an extended story a few frog” and timed how lengthy it took to begin talking the (multi-paragraph) response. With out streaming, each Residence Assistant Cloud and Piper took greater than 5 seconds to reply! That is lengthy sufficient to make you surprise in case your voice assistant heard you 😄 With streaming enabled, each TTS companies took about half a second to begin speaking again. A 10x enchancment in latency!
New Piper voices
Piper, our homegrown text-to-speech software, continues to develop with assist for a number of new languages! These new voices have been skilled from publicly obtainable voice datasets, and can be found now within the Piper add-on:
- Daniela (Argentinian Spanish)
- Pratham, Priyamvada, Rohan (Hindi)
- Information TTS (Indonesian)
- Maya, Padmavathi, Venkatesh (Telugu)
Wish to know what the brand new voices sound like? You’ll be able to take heed to samples
In case your language is lacking from Piper, otherwise you don’t like the present voices on your language, we’re all the time searching for volunteers to contribute their voices! Please contact us at [email protected]
Conclusion
Previously three years, we’ve made nice strides with Residence Assistant Voice on each the {hardware} and software program fronts. Customers in the present day have all kinds of selections on the subject of voice: from absolutely native to utilizing the most recent and biggest AI to energy their good properties. The wonderful thing about our experimentation with AI is that there are not any buyers searching for returns, faux cash, or “rug-pulls”. We do all the things for you, our group. We’re on this for the lengthy haul, and wish this all to be your alternative, conserving you in full management of whether or not you need to use this expertise or keep away from the hype fully.
A lot of the superior work carried out on voice is just attainable with the assist of our group, particularly those that subscribe to Residence Assistant Cloud or anybody who has bought our Residence Assistant Voice Preview Version (each nice methods to get began with voice).
