I wanted to talk to my house like Tony Stark does: “turn the office lights off”, “what’s on my calendar?”, “play some music in the kitchen”. I already ran Home Assistant, so I didn’t need to buy anything. I needed to stitch a few pieces together.
It works. The first voice sounded stiff, so I moved to ElevenLabs and its Daniel voice, which is far more fluent.
The chain
Everything goes through Home Assistant’s built-in voice system, Assist. A spoken request takes four steps:
- Speech to text turns my voice into words. This is Whisper, which runs on my own server.
- The brain is Google Gemini. It gets the words and works out what I want and which devices to use.
- Actions happen in Home Assistant: lights, speakers, reminders.
- Text to speech speaks the reply. I started with Piper, another local add-on, and now use ElevenLabs with the Daniel voice.
The microphone and speaker are an old Samsung tablet in the office running the Home Assistant app in push-to-talk mode. I skipped a wake word on purpose, because a button is more reliable than a box that’s always listening. Typed commands go through the same pipeline, which made testing easy.
Keeping it private
Gemini only sees the text of what I ask and the names of the devices I’ve let it control. It never gets recordings of my voice. I keep speech recognition on my own server for that reason.
The voice is the one trade-off. ElevenLabs is a cloud service, so the text of each reply goes to it to be spoken. My voice still stays in the house.
I also use the paid Gemini tier. Google’s free tier can use your prompts for training and have humans review them, and the paid tier doesn’t.
What it costs
I set up a prepaid billing account with £10 of credit and auto-reload switched off, so it can’t run away with my money. The model is one of the cheap fast ones, at roughly $0.25 per million tokens in and $1.50 out.
My estimate for normal use is between £0.75 and £3.80 a month. I’ll update this post once I have a few weeks of real usage in the Google console.
The voice is a separate cost. ElevenLabs’ Starter plan is $6 a month (about £4.50) and gives 30,000 credits a month. I’ve used 9,738 in my first two weeks. That brings the total to roughly £5 to £8.50 a month.
One thing surprised me: subscriptions don’t count. Home Assistant can’t use my Gemini subscription. It needs an API key, which is billed separately.
Things that went wrong
- The microphone wouldn’t work. Browsers only allow microphone access on secure (https) pages, and my Home Assistant was on plain http. I had to put it behind my reverse proxy with a proper certificate first.
- The tablet said “name not resolved”. I’d set a second DNS server (1.1.1.1) on its Wi-Fi as a safety net, so it sometimes skipped my Pi-hole and couldn’t find my internal address. Removing the extra DNS fixed it.
- Gemini refused to answer questions about my calendar. Its safety filter blocked everything once I exposed the full calendar. A small helper that holds only the next event’s title and time fixed it.
- It tried to play music on the wrong speaker. It kept picking an Echo Dot, which can’t play it. I hid the Echos and TVs from the assistant so it only sees speakers that work. Nothing was removed from Home Assistant.
What’s still not done
- Daniel is far more fluent than where I started, but it still isn’t Jarvis. Next I’ll try a butler-style personality in Gemini’s instructions, which is free and should change a lot.
- Speaker output goes through the tablet for now. I want it coming out of the proper speakers.
- There’s no wake word.
Would I recommend it?
If you already run Home Assistant, yes. It took an afternoon to set up and costs under £10 a month to run. It’s the first time voice control in my house has felt useful instead of a party trick. The next post covers the screen I built to go with it.