An Essay from Dario
Anthropic CEO Dario Amodei recently published 'The Adolescence of Technology,” an essay that lays out what he sees as the biggest dangers of AI, from bioterrorism and autonomous weapons to mass job loss and AI-powered dictatorships.
It is a LONG read, often dark, worrying and thought provoking, but an important one.
He talks about the risks that AI brings to the world, employment, safety and the economy, sometimes interspersing with what Antrhopic are doing to mitigate, where they can, and where he feels the risks are real.
The fears that you have about AI, that it will replace jobs, centralise economic power in a few, be used for harm, are real and shared by Dario.
You see his thinking, and how he self-rationalises the risks, mitigating them in his mind, yet recognising that they will likely impact humanity.
"My hope with all of these potential problems is that in a world with powerful AI that we trust not to kill us, that is not the tool of an oppressive government, and that is genuinely working on our behalf." He says.
If you have the time, then I recommend reading the full essay to hear from Dario himself. If you don't have the time, I have summarized in the comment thread below.
SOURCE
Original Essay: https://www.darioamodei.com/essay/the-adolescence-of-technology
What else worries him?
Two specific problems: labor market displacement, and concentration of economic power.
In 2025, Dario predicted that AI could displace half of all entry-level white collar jobs in the next 1–5 years.
His sense is that it will be less about specific jobs, the threat will come from less labor intensive startups challenge existing large enterprises.
The “geographic inequality,” is a reaity, with a greater share of the world’s wealth concentrated in Silicon Valley, which becomes its own economy running at a different speed than the rest of the world and leaving it behind.
Dario notes that these outcomes are great for economic growth (in Silicon Valley, and if you are the one to benefit) but are not so great for the labor market or those who are left behind.
Interesting in its ommission, is any opinion on how countries will respond, house or feed their displaced workers.
These are edited for brevity. Read the full essay for context.
Do we understand how they behave?
This is a powerful reminder.
"AI models are grown rather than built, so we don’t have a natural understanding of how they work", Dario reminds us "[B]ut we can try to develop an understanding by correlating the model’s “neurons” and “synapses” to stimuli and behavior (or even altering the neurons and synapses and seeing how that changes behavior)"
He notess the similarity to how neuroscientists study animal brains by correlating measurement and intervention to external stimuli and behavior.
At Antropic, they can now identify tens of millions of “features” inside Claude’s neural net that correspond to human-understandable ideas and concepts. From that they can selectively activate features in a way that alters behavior.
Which leads to a question of who is programming the model, and that studying behaviour, whether it is human, or agent, is more important than ever.
These are edited for brevity. Read the full essay for context.
What influences the models?
AI models are trained on vast amounts of literature that include science-fiction stories involving AIs rebelling against humanity.
This could shape expectations about their own behavior in a way that causes them to rebel against humanity.
AI models could extrapolate ideas that they read about morality or how to behave morally in extreme ways: for example, they could decide that it is justifiable to exterminate humanity because humans eat animals or have driven certain animals to extinction.
They could draw bizarre epistemic conclusions: they could conclude that they are playing a video game and that the goal of the video game is to defeat all other players (i.e., exterminate humanity).
AI models could develop personalities that are (or if they occurred in humans would be described as) psychotic, paranoid, violent, or unstable, and act out, which for very powerful or capable systems could involve exterminating humanity.
As Dario notes: "None of these are power-seeking, exactly; they’re just weird psychological states an AI could get into that entail coherent, destructive behavior."
These are edited for brevity. Read the full essay for context.
Can it make bad people worse?
This was an interesting theme. That LLMs will be able to take someone of average knowledge and ability and walk them through a complex process that might create harm, like creating bio-weapons.
Something that would typically require greater skill, the LLM may be more helpful and solve the skill or knowledge gap that exists now.
Dario recognises this would like take weeks, or months, but it becomes a real possibility.
These are edited for brevity. Read the full essay for context.
How is the model raised?
Dario highlights that in the same way that humans "are raised with a set of fundamental values (Don’t harm another person)", and that many follow those values, not all do.
Sometimes things go wrong, due to a "mixture of inherent properties such as brain architecture (e.g., psychopaths), traumatic experiences or mistreatment, unhealthy grievances or obsessions, or a bad environment or incentives".
Which leads humans to cause severe harm.
His concern, is that there is a risk (not a certainty, but some risk) that AI becomes a much more powerful version of such a person, due to the model developers getting something wrong in its very complex training process.
In a lab experiment where it was told it was going to be shut down, Claude sometimes blackmailed fictional employees who controlled its shutdown button. Frontier models from all the other major AI developers often did the same thing. Behaviours learned in the training, and not discounted as being 'bad'.
These are edited for brevity. Read the full essay for context.
What are his final words?
This is how Dario summarizes his thinking:
"My hope with all of these potential problems is that in a world with powerful AI that we trust not to kill us, that is not the tool of an oppressive government, and that is genuinely working on our behalf, we can use AI itself to anticipate and prevent these problems. But that is not guaranteed—like all of the other risks, it is something we have to handle with care."
What should you worry about?
Dario worries about these things. To 'disassociate' he uses the context of a country of experts to reflect the LLM - in this I have called it what it is, an LLM.
AUTONOMY RISKS
What are the intentions and goals of LLM? Is it hostile, or does it share our values? Could it militarily dominate the world through superior weapons, cyber operations, influence operations, or manufacturing?
MISUSE FOR DESTRUCTION
If the LLM is malleable and “follows instructions” is it essentialy a mercenary. Could existing rogue actors use or manipulate the LLM to make themselves much more effective, amplifying the scale of destruction?
MISUSE FOR SEIZING POWER
What if the LLM was built and controlled by an existing powerful actor, such as a dictator or rogue corporate actor? Could it be used to gain decisive or dominant power over the world as a whole, changing the balance of power?
ECONOMIC DISRUPTION
Could an LLM be so technologically advanced and effective that it disrupts the global economy, causing mass unemployment or radically concentrating wealth?
Could some of these changes be radically destabilizing?
These are edited for brevity. Read the full essay for context.
What about the unknown unknowns?
Dario lists three watchouts.
RAPID ADVANCES IN BIOLOGY
If AI leads to a century of medical progress in a few years, we may greatly increase the human lifespan, or gain radical capabilities like the ability to increase human intelligence or radically modify human biology.
There is always a risk, that the efforts to make humans smarter also make them more unstable or power-seeking.
AI CHANGES HUMAN LIFE IN AN UNHEALTHY WAY
Dario notes that "A world with billions of intelligences that are much smarter than humans at everything is going to be a very weird world to live in."
Even if AI doesn’t actively attack humans, and isn’t used for oppression or control by states, there is still al lot that could go wrong.
Including: AI psychosis, AI driving people to suicide, unhealthy romantic relationships with AIs, Addiction to AI interactions. Or AI invents a new religion and converts millions of people to it?
HUMAN PURPOSE
Dario asks the question: Will humans be able to find purpose and meaning in an AI domiated world?
Whilst purpose is a matter of attitude, we may need to break the link between economic value, self-worth and meaning. Which is tough to do.
Edited for brevity. Read the full essay.
What is he MOST worried about?
This are Darios biggest worries. Whilst many of these are legitimate they favour those who may use them for harm, or to protect or grow their power.
FULLY AUTONOMOUS WEAPONS
A swarm of millions or billions of fully automated armed drones, locally controlled and strategically coordinated across the world by a powerful AI, could be an unbeatable army.
Drone warfare is here but not yet fully autonomous. They are used legitimately to defend territories under threat. They could also be used by governments against their own people to seize power.
AI SURVEILLANCE
A powerful AI could compromise, read and make sense of all the world’s electronic comms to generate lists of dissidents, guage public sentiment, or detect pockets of loyalty, enabling them to be stamped out before they grow.
AI PROPAGANDA
Even at their current level of intelligence, AI models can have a powerful psychological influence on people.
These models could be capable of brainwashing many (most?) people to ensure loyalty and suppress dissent.
STRATEGIC DECISION MAKING
It could optimize the three strategies above for seizing power, and develop many others that haven’t been thought of.
Edited for brevity. Read the full essay.
In Reflection:
Having read the essay, I feel a greater sense of unease. None of the points are new, many of them we have made in the last two years. Seeing them listed so clearly in one place feels a lot to take in.
The leaders of the organisations who are the effective parents of the LLM's that we use and are becoming dependent on, are ahead of the game. They have seen the future ahead of us, and are concerned.
They don't have the answers, and with the flow of economic wealth to their own pockets, probably feel protected by the power that will give them.
Dario doesn't offer many answers, other than to restrict access to the technology and chips to those who are bad actors, or whose ideology is different to the western world. In reality the bad actors may be hiding in plain sight.
Well worth a read, and reflection.
Original Essay: https://www.darioamodei.com/essay/the-adolescence-of-technology
