AI agents have become the darlings of the corporate world, and big tech companies are loving every minute of it.
Nvidia, Microsoft, OpenAI, Anthropic, and Google are all selling the idea that these tools are true digital collaborators, with flexibility and cognitive power comparable to a real human being.
But there is a big problem with that whole narrative. 😬
A recent study by Professor Emma Wiles at Boston University threw cold water on this story and showed that treating AI agents as if they were employees not only creates unrealistic expectations but also makes the people around them perform worse.
In practice, study participants identified 18% fewer errors when the work was presented as coming from an agent labeled an AI employee rather than a simple chatbot. Small change in name, big change in behavior.
Nearly a third of the 1,261 managers who participated in the research already frame AI agents as employees, and 23% go so far as to list them on official org charts.
Seems harmless, right?
But the numbers show that this shift in terminology has a real and concerning impact on human accountability within organizations, with ripple effects that go well beyond the office.
And that is exactly what we are going to break down here. 👇
What the study actually found
The research conducted by Emma Wiles is not just another academic paper loaded with theory. It draws on concrete data collected from managers who have already integrated AI agents into their day-to-day workflows. What stood out was not just the speed at which organizations adopted these tools, but the way they started to perceive them within the corporate structure. When an AI agent begins showing up on an org chart with a name, a title, and even a defined area of responsibility, something shifts in the minds of the people working alongside it — and that shift is not necessarily a good thing.
The study found that by humanizing AI agents this way, companies end up creating a kind of artificial comfort zone for their teams. Employees start delegating responsibilities less carefully, assuming the agent will cover the gaps, catch the mistakes, and make the right calls at the right time. But that is not what actually happens. AI agents have no moral judgment, carry no emotional context of situations, and take no responsibility for anything they do. They execute tasks based on instructions and data, period. Anything beyond that is human projection.
The most alarming finding from the study is precisely this: when work was presented as the output of an AI employee, participants saw themselves as less responsible for the outcome. On top of that, they were 44% more likely to escalate questionable AI work to a manager for review instead of trusting their own corrections. In other words, the agent that was supposed to save time ends up generating more steps and more bureaucracy. The quality of human work drops, attention decreases, critical review of AI-generated outputs becomes rarer, and accountability starts dissolving in a way that, at the end of the day, lands nowhere.
Why big tech pushes this narrative
It is no coincidence that companies like Microsoft, OpenAI, Anthropic, and Google have been investing heavily in the idea that their AI agents are practically team members. There is a very clear commercial interest behind it. Nvidia CEO Jensen Huang himself has openly talked about offices full of digital humans. Since April, tech giants have launched tools designed to manage entire teams of agents, often announced explicitly as digital colleagues with the flexibility and cognitive power of real people.
The more organizations view these products as digital collaborators capable of replacing or complementing human functions, the more they invest in licenses, integrations, and expanded usage. Positioning agents as digital employees is an extremely effective sales strategy. The problem is that this narrative carries a cost that goes well beyond the financial. When a company buys into the idea that its AI agent is a legitimate team member, it starts making structural decisions based on that premise. It redefines workflows, redistributes human roles, adjusts performance expectations, and in some cases even reduces hiring based on the expected capabilities of the agent.
All of this is built on unrealistic expectations about what these tools can consistently and reliably deliver. And when reality does not match the expectation, the fallout does not land on the agent. It lands on the people. There is also a subtler issue at play: the language we use to describe technology shapes how we interact with it. Calling an AI agent a colleague, giving it a name, a profile, and a spot on the org chart, creates a perception of agency that it simply does not have. This is not just an internal communications problem. It is an organizational design problem, because when a company’s structure is built around a false premise, the decisions made from it tend to be equally flawed.
Human accountability cannot be outsourced to AI
To be fair, the technical progress of agents is not just hype. These systems, which can be understood as AI tools programmed to work in a loop until they reach a goal, have gotten measurably better at complex tasks. But making the leap to calling them colleagues or employees is a different story entirely, and that choice has serious consequences for the irreplaceable role of human accountability within organizations.
No AI agent, no matter how sophisticated, can be held accountable for a wrong decision, for incorrect data that caused financial loss, or for misguided advice that affected someone’s life. This is not a technical limitation that will be fixed in the next model update. It is a structural question about what it means to be responsible for something — and that condition requires awareness, intent, and the possibility of facing consequences, characteristics that are exclusively human.
This risk goes far beyond office culture. As AI agents are integrated into critical areas like healthcare, education, government, and even military contexts, the danger grows that they become a convenient place to shift blame for failures that are actually the result of poor human decisions, misaligned incentives, and inadequate oversight. A striking example of this was when an attack on a girls’ school in Iran was popularly attributed to an AI, when all the evidence pointed to a cascade of errors made by people. The machine became the scapegoat for a failure that was deeply human.
Keeping human accountability at the center of decision-making does not mean ignoring the potential of AI agents or stopping their use altogether. It means integrating them honestly, with clarity about what they do well, where they fall short, and who is in charge at every stage of the process. Organizations that manage to do this tend to extract far more value from these tools precisely because they are not operating under illusions about what they are.
What this means for human capabilities at work
There is a cruel irony in all of this. The promise of AI agents is to expand human capabilities, freeing people from repetitive tasks so they can focus on things that demand more creativity, judgment, and empathy. But when these agents are treated as coworkers, the effect can be the exact opposite. People begin delegating not just the mechanical tasks but also the ones that require critical attention — and over time, that dulls the very human capabilities that should have been preserved.
Economist Daron Acemoglu from MIT, the 2024 Nobel laureate who studies AI’s impact on the economy, sums up the issue well. In his view, selling AI agents as things capable of replacing humans is a proposition doomed to fail. The ideal approach would be to optimize these tools so they enhance human capabilities, not take people’s place. And according to him, that is not what we are seeing right now.
An interesting effort in that direction comes from Stanford University, where researchers presented 1,500 workers across 104 occupations with information about what AI could, in theory, do in their jobs. Then they asked what would actually be most useful and productive. The results were eye-opening. Workers did want automation in certain areas — like legal assistants who saw value in AI for ensuring cases were progressing properly. But in many cases, the tasks that tech experts considered most suitable for AI were exactly the ones that professionals said they neither wanted nor needed to hand over to an agent.
The good news is that this path can be reversed, and the first step is changing the language and the way AI agents are presented and integrated into organizations. Treating them as powerful tools rather than digital collaborators does not diminish the value they deliver. On the contrary, it creates a context where people maintain active responsibility over processes, continue developing their human capabilities, and are able to use AI in a more strategic and safer way. Calling an agent an employee is easy and convenient, especially when something goes wrong. But that is just a marketing exercise. It does not make the tool more capable, and as Wiles’ research shows, it makes the humans around it worse at what they do. And they are the ones carrying the true agency that AI is trying to imitate. They deserve better. 🧠
