Andrej Karpathy introduces Dobby, an AI agent that promises to kill the need for apps
Andrej Karpathy is one of the most respected names in the world of artificial intelligence. Former AI director at Tesla and one of the most influential researchers at OpenAI, he has a gift for turning personal experiments into trends the entire world starts following. When Karpathy builds something on his own, at home, in his spare time, the market stops and pays attention. It’s no exaggeration to say he’s the kind of person whose side projects become entire product categories within a few years.
And his latest one is already making waves.
In a recent episode of the No Priors podcast, Karpathy introduced the world to Dobby, an artificial intelligence agent he built himself and named after the house-elf from Harry Potter. The name choice is no coincidence: just like the elf from the saga, the digital Dobby was created to take care of the house, solve everyday problems, and do it all invisibly, without anyone needing to keep giving it orders.
Its goal is simple but powerful: take control of all the connected devices in your home without you ever needing to open a single app. No screens to navigate, no menus to dig through, no notifications to ignore. Just you saying what you want, as naturally as possible, and the house responding.
Sounds like science fiction, but it’s not.
According to Karpathy’s own description, Dobby runs on an AI agent from OpenClaw that replaced the entire fragmented software stack he was using at home. Before the project, he was bouncing between six different apps to manage things like Sonos speakers, lighting, and security cameras. Now, Dobby controls all of that using natural language.
With minimal initial setup, the agent scanned Karpathy’s local network, identified connected devices, located undocumented APIs, and started executing commands like playing music and turning on lights. All without needing detailed instructions or manually configured integrations for each device.
And the inevitable question comes up: what if you never had to open an app again?
Receive the best innovation content in your email.
All the news, tips, trends, and resources you're looking for, delivered to your inbox.
By subscribing to the newsletter, you agree to receive communications from Método Viral. We are committed to always protecting and respecting your privacy.
This experiment goes way beyond a technical curiosity. It touches on something that could completely change how we interact with technology on a daily basis, and it poses a real threat to the app economy as we know it. 🤖
What Dobby does that voice assistants can’t
The vast majority of voice assistants on the market today still rely on pre-programmed skills, specific integrations, and fairly exact commands to work properly. Anyone who’s ever tried asking Alexa to do something slightly outside the box knows what we’re talking about: the response is usually a list of options that doesn’t solve anything, or a plain old comprehension error. The problem isn’t the voice itself — it’s that these systems still run on fixed rules, and fixed rules can’t keep up with the complexity of human language.
Dobby was built with a completely different approach. Because it’s based on a large language model, it doesn’t depend on exact commands or pre-configured integrations for each device. It understands the context of the conversation, interprets the intent behind what you say, and makes decisions based on what makes the most sense for that situation. If you say you’re cold, it can raise the room temperature without you ever mentioning the thermostat. That’s contextual reasoning, and that’s where the real difference lives.
Another point worth highlighting is how Dobby can reverse-engineer devices and APIs without official documentation. Unlike a conventional voice assistant that needs a published skill or an official manufacturer integration, Karpathy’s agent scans the network, discovers available devices, and figures out on its own how to communicate with them. This ability for autonomous discovery is what makes the project so impressive from a technical standpoint.
The result is an agent that feels like it’s actually present in the environment, rather than just responding to isolated triggers the way conventional assistants do. Karpathy summed up the experience well by saying he no longer needs to use any of those six apps that used to be part of his daily routine. Dobby handles everything through natural language, and that radically changes his relationship with technology at home.
Home automation has never been this close to what it should be
Home automation has existed as a concept for decades, but it’s always had an accessibility problem that was never fully solved. Setting up a truly smart home still requires technical knowledge, patience to deal with integrations that break out of nowhere, and an almost fanatical willingness to keep everything running. Apps helped democratize part of that process, but they created another problem: fragmentation. Every manufacturer has its own app, every app has its own logic, and the user ends up stuck in the middle of an ecosystem that’s supposed to simplify life but in practice makes it more complicated.
And Karpathy knows exactly what he’s talking about. He admitted he nearly threw his Sonos speakers out the window out of sheer frustration with the app. Anyone who’s ever tried to adjust something in the Sonos system can relate to that feeling. The apps, which are supposed to make life easier, are often ridiculously hard to use. It’s an irony worth reflecting on: the technology meant to solve problems ends up creating new ones.
What Dobby proposes is an intelligence layer that sits above all of that. Instead of you needing to know which app controls which device, you simply say what you want, and the agent figures out how to make it happen. This abstraction is powerful because it removes the friction that still keeps a lot of people away from home automation. You no longer need to learn the logic of each platform. The interface becomes natural language, something every person has mastered since childhood.
When you think about this at scale, the impact is huge. A family that currently uses four or five different apps to control their home could replace all of it with a single conversation with an agent like Dobby. Lights, music, temperature, cameras, locks, automation routines — all accessible through the same channel, with the same ease as texting a friend. Karpathy didn’t invent smart home automation, but he may have found the missing piece to finally make it work the way it always promised. 🏠
The silent threat to the app economy
This is where the conversation gets more interesting — and a bit more tense for the tech industry. The app economy is one of the most solid pillars of the modern digital industry. The Apple App Store and Google Play combined move hundreds of billions of dollars every year. This entire ecosystem was built on one basic premise: that users need apps to access services and features. If that premise starts being seriously challenged, the consequences are hard to predict.
Artificial intelligence agents like Dobby point to a future where the traditional graphical interface is no longer the main point of contact between the user and technology. Instead of opening a music app to play a playlist, you say you want to listen to something relaxing while you have dinner. Instead of opening the camera app to check who’s at the door, you ask if there’s someone waiting outside. The software layer that exists today to mediate that relationship starts to lose its purpose when an agent can understand and execute the request directly.
The original article from Business Insider puts this question bluntly: generative AI and agents represent a concrete threat to the app ecosystem and the companies that depend on it. The interface is evolving away from the model of tapping buttons on an iPhone screen. Instead, the trend is to use voice much more, through chatbots and AI agents.
This doesn’t mean apps will disappear overnight, but the warning signal is on. Major tech companies have already noticed this shift and are racing to position their own AI agents as central interfaces. Apple is integrating AI deeper into iOS, Google is transforming the Assistant into something far more capable, and Microsoft bet big on Copilot to take on exactly this role of intelligent intermediary.
Karpathy’s experiment with Dobby is small in scale but enormous in symbolism. It shows, in a practical and real-world working way, that the app-by-app model may not be the right answer for the future. ⚡
Karpathy and vibe coding: an important connection
It’s worth remembering that Karpathy is also considered the father of the vibe coding concept — an approach to programming where the developer uses AI to create software in an intuitive and conversational way, almost as if describing what they want to a coworker. This concept was born precisely from the philosophy that technology should adapt to humans, not the other way around.
Dobby is a natural extension of that philosophy. If vibe coding proposes that anyone can create software by talking to an AI, Dobby proposes that anyone can control their home by talking to an agent. In both cases, the technical barrier drops dramatically, and access becomes democratized. It’s a worldview where the complexity of technology stays hidden behind a simple, human interface.
This connection between the two projects also helps us understand the moment we’re living in. The same advances in language models that made vibe coding possible are what allow Dobby to work so well. They’re pieces of the same puzzle, and Karpathy is putting that puzzle together in real time, right in front of everyone.
The challenges that still exist
It’s important not to lose sight of the fact that Karpathy himself acknowledges this kind of project still requires considerable technical skill. Building an agent like Dobby isn’t something just anyone can do today. You need to understand networking, communication protocols between devices, language model configuration, and a whole range of other technical aspects that are still far from trivial.
On top of that, security and privacy concerns need to be carefully evaluated. An agent that scans the local network, discovers devices, and accesses APIs without official documentation raises legitimate questions about what happens if the system gets compromised. Data protection and the integrity of the home environment are points that any solution in this space needs to take seriously.
There’s also the question of reliability. Traditional voice assistants can be frustrating, but they’re predictable. An agent based on a large language model might interpret an instruction in an unexpected way, make a decision the user didn’t intend, or simply get it wrong. The margin of error in a home environment, where devices control locks, cameras, and heating systems, needs to be extremely low. That’s a real technical challenge that still needs to be addressed before solutions like Dobby can be adopted at scale.
What comes next
Karpathy made it clear that Dobby is still a personal project — an experiment he works on out of genuine interest in the problem. But his track record shows that this kind of project has a way of growing and influencing what comes next. Micrograd, nanoGPT, the neural network tutorials he published over the years — they all started as personal projects and became references for developers worldwide. Dobby follows the same pattern.
What makes this moment particularly relevant is that the technology needed to build something like Dobby only became accessible very recently. Large language models have reached a level of contextual understanding that makes it viable to use natural language as a real interface, not just a demo trick. Home automation protocols have matured enough to allow more open integrations. And the hardware to run models locally has become cheaper and more efficient. All of this converged at the same time, and Karpathy was one of the first to put the pieces together.
The question hanging in the air isn’t whether agents like Dobby will become common, but how soon and in what form it will happen. The interaction model based on natural language — no apps, no menus, no mandatory screens — is advancing faster than most people realize. And when it fully arrives, it’ll feel so natural that we’ll wonder why it took us so long to get there. 🚀
