550W: AI Assistant At Home
18 Oct 2026 AI AIAgentI developed an AI assistant to be used at home: 550W. Based on my voice commands, 550W can turn on/off the light, taking notes (more capabilities to add).
Background
I have a smart bulb at the corner of the living room, right beside the cat litter box. The bulb gets connected to Alexa, so I would ask Alexa echo dot to turn on the light every another night when I scoop the litter box. Due to my poor memory, I often cannot recall if I did scooping last night. When I received a free trial of Alexa+, I discovered that I can ask it to take notes, helping me to track the scooping. The dictation is not perfect (partially due to my accent), so Alexa+ sometimes get my point wrong, and we had quite some fun with it. Another issue is Alexa+ never learnt the pattern: if the note shows last scooping was 2 days ago, it is usually followed by turning on the light and updating the note with today’s date. After Alexa+ trail ends, it goes back to the dumb Alexa and don’t even know how to take notes any more.
Last year around this time, I bought a M4 Max Mac Studio with 128GB memory, with the hope to run local LLM service. However, the Mac Studio is barelly used in the year past. Finally, I decide it is time to build an AI assistant by my own. I would call it 550W, naming after the supercomputer in Wondering Earth.
Architecture
Wake Up
It turns out how to wake up 550W becomes the very first challenge. The Mac Studio does not have a microphone in the first place. My first idea is using a spare iPhone as the input device. There is a feature called Continuity Camera which allows the iPhone be connected to Mac Studio like a webcam. After quite a bit struggling, I realized that to establish Continuity Camera connection, the iPhone must be close to Mac Studio and stay stationary on a stand not laid down. And if the Mac Studio goes into sleep after being inactive for a while, the connection is also dropped, which means I would need a deamon process running on the Mac Studio to read and process audio input from Continuity Camera all the time to prevent it from sleep.
Waking up with magic words “Alexa” or “Siri” has the low-power advantage other solutions cannot compete with, so I gave up the Continuity Camera idea and have to consider a “Siri”-based solution.
Siri + Shortcuts
It takes a developer subscription to run self-developed app on iPhone, and I am not sure how many APIs are open for developers to use Siri. Nevertheless, the built-in Shortcuts app provides some programming capability to expand Siri. As long as the shortcut name is unique and starts with a verb, Siri seems to be able to invoke the shortcut pretty reliably. The only inconvenience is I have to first wake up Siri, then say “Connect 550W” the shortcut name to start talking to 550W.
Comes to Shortcuts, we have to build the shortcut flows from a collection of pre-defined actions.
Luckily, there is an action called “Get Content of URL”, which can do HTTP requests (GET or POST).
The first “working” example uses “Ask for Input” to listen to the prompt,
then send the prompt in the POST request body to the OpenAI endpoint API v1/chat/completions,
and speak out the response extracted from the received JSON by indexing choices.1.message.content.
We can put this in a for loop to repeat several times, but it won’t form a conversation,
because every prompt only send the latest prompt, without the context.
It is single-round chat only, and the agent can do nothing beyond generating text.
OpenCode V2
For the OpenAI endpoint APIs, I don’t want to pay subscription and am tired searching free but restricted LLM services, that was why I got the Mac Studio last year. So now I host small LLM models on Mac Studio, initially with lmstudio. But the MLX framework lmstudio uses seems to have issue with its kv-cache, so I later switched to omlx, which runs faster. I use QWen3.6-27B model for coding, while using it for voice assistant the lag becomes noticeable, so I changed to gpt-oss-20b, 8-bit quantization for 550W.
To make use the local LLM for coding, I uses OpenCode as the coding agent. After using OpenCode for a while, I realize OpenCode adapts server-client architecture, and the recent upgrade to V2 cleans up the server APIs to make it fit agentic use scenarios better. The LLM inference endpoint requests, tool calling are all happening on the server side. With the server APIs well-defined, the client could be in various forms: TUI, web, app as OpenCode officially provides. Shortcuts could just be another client. One big enhancement in OpenCode V2 is a unified server, getting the sessions shared among all the clients, so I can open up a web page to monitor the session that shortcut is carrying on.
I developed several plugins (this is how OpenCode defines customized tools) to control the smart bulb (via python-kasa package),
and take notes.
In testing I figured out that gpt-oss-20b does the job better with more details instructions
(apparently it was not trained with OpenCode tool call patterns),
so I developed a skill cat-box to specify the flow:
- Check date and find the “CatLitterBoxTracking” note to read in one JavaScript execution
- Report the last scooping date and offer turning on the light when it was more than 2 days ago
- Turn on the light when get confirmation, offer updating the note
- Update the note when get confirmation
With some tweaks, I successfully triggered it to finish the flow. I am more excited about how many possibilities this design opens up. Basically, I can just develop any tools for anything I could accomplish on the Mac Studio, then I could ask the agent to do that for me via voice commands.
Known Issue
One more issue yet I need to find a solution is how to have Siri wait for long execution patiently.
It turns out if the agent does multiple steps or just think longer before giving the response, Siri would timeout and end the shortcut.
Due to the complexity, I have designed the shortcut being a 3-layer shortcut calls already:
connect 550W -> prompt 550W -> wait 550W.
In the wait 550W I have a 20 iteration for loop, calling wait endpoint and check the idle, updated timestamp until idle timestamp is no longer older than updated timestamp.
But this wait shortcut sometimes is ignored or triggers error then Siri would just say “something went wrong, please try again”, then the whole conversation is forced to end.