Editor: Editorial Department
Just now, Anthropic CEO Dario Amodei published a long article, looking forward to the impact of AGI on the world. He boldly predicted that "powerful AI" would come in 26 years, and AI would compress a century of scientific research progress into 5-10 years.
Just as the 2024 Nobel Prize in Physics and Chemistry were both awarded to AI, Anthropic CEO Dario Amodei also published a long article looking forward to the impact of AGI on the world.
The article is titled "Machines of Loving Grace" and subtitled "How can AI change the world for a better future?".
In it, he proposes that AI will compress a century of scientific research progress into 5-10 years.
What would a world with powerful AI look like?
As CEO of Anthropic, Dario Amodei often thinks and discusses the risks of powerful AI and how to mitigate them.
Amodei says that many people think of themselves as pessimists, but that's not the case at all.
He focuses on AI risks because, in his opinion, it is the only obstacle between us and a bright future.
In his view, most people underestimate the potential that AI can bring, while also underestimating how bad the risks of AI can be.
And in this long article, Amodei tries to paint a picture of what the positive outlook looks like – what a world with powerful AI would look like if all went well.
He is confident enough that he can grasp the general direction of future development, even if most of the details are ultimately wrong.
Why focus on risk
Amodei also emphasized that the focus on risk is due to the following reasons:
1. Maximize impact
The benefits of AI technology are driven by strong market forces and are therefore inevitable. While risks are not predestined, human actions can dramatically alter the likelihood of their occurrence.
2. Avoid giving the impression of propaganda
Don't make people feel like you're doing propaganda or divert attention from the shortcomings of AI.
3. Avoid boasting
Amodei is disgusted by the way many public figures in the circle discuss AGI as if they were prophets leading all beings to salvation.
"It's dangerous to think of corporations as forces that can unilaterally shape the world, and it's also dangerous to look at actual technological goals in a religion-like way."
4. Avoid sci-fi labels
Some radical groups will discuss AI in a tone that is too "sci-fi", such as consciousness uploading, space exploration, and cyberpunk. This tone unwittingly brings with it a whole host of cultural biases and some assumptions that can't be defined.
As a result, these discussions often seem like the wishful thinking of a niche group, which discourages most people.
But Amodei also believes it's important to discuss what a better world with advanced AI looks like.
Here are the five categories he thinks are the most promising:
1. Life Sciences and Physiological Health
2. Neuroscience and Mental Health
3. Economy and Poverty
4. Peace and governance
5. Work and meaning
Amodei has professional experience in both biology and neuroscience, as well as an understanding of economic development. In general, his predictions are quite avant-garde.
Amodei believes that it would be valuable to be able to bring together a group of experts in various fields to create a more comprehensive and insightful version.
And your own attempts can be seen as a starting point.
Basic Assumptions and Analytical Framework
Amodei defines what advanced AI means: a technological threshold that triggers a 5-10 year development period.
He doesn't like to use the term AGI, hence the term "powerful AI."
Amodei believes that powerful AI could be available as early as 2026.
Artificial General Intelligence (AGI) is a term that is not precise enough, and it has accumulated too many sci-fi elements and excessive hype. In contrast, I prefer to use terms like "powerful AI" or "expert science and engineering".
By "powerful AI", we mean that an AI model may be similar in form to today's large language models, although it may be based on different architectures, may involve multiple interactive models, and may employ different training methods.
Specifically, it will have the following characteristics –
- In terms of sheer intellectual level, it surpasses Nobel Prize-level winners in most related fields – such as biology, programming, mathematics, engineering, writing, etc. This means that it is able to prove unsolved mathematical theorems, create extremely high-level novels, and write complex codebases from scratch.
- All the interactive methods available to humans for virtual work, including text, audio, video, mouse and keyboard control, and internet access. It can take advantage of these interactions for any action, communication, or remote operation. The skills in execution also surpass the world's top human experts.
- Not only does it passively answer questions, but it can also take tasks that take hours, days, or even weeks to complete, and then work on them independently like a smart employee, actively seeking clarification if necessary.
- It has no physical form, but can control existing physical tools, robots, or laboratory equipment via a computer; Theoretically, it could even design and use robots or equipment for itself.
- The compute resources used to train the model can be reconfigured to run millions of model instances (this matches the expected large-scale compute cluster capacity by around 2027). The model absorbs information and generates action at 10 to 100 times the speed of humans. However, it may be limited by the physical world or software response time of the interaction.
- Each of these millions of replicas can perform unrelated tasks independently or work like a human team when needed. You can even fine-tune different groupings to make them particularly good at specific tasks.
We can summarize it as "the genius nation of the data center".
Obviously, such an entity can solve extremely difficult problems very quickly, but it is not easy to determine how quickly.
According to Amodei, both "extreme" positions are wrong.
First, you might think that the world would change in a matter of seconds or days (the so-called "singularity") because powerful intelligence is constantly perfecting itself to solve all possible scientific, engineering, and operational tasks almost instantly.
The problem with this view is that there are real physical and practical limitations, such as in building hardware or biological experiments. Intelligence can be very powerful, but it's not magic.
Second, it's equally implausible if you think that technological progress is saturated, or constrained by real-world data or social factors, and that superhuman intelligence will hardly make any difference.
On hundreds of scientific and social problems, a group of really smart people would dramatically speed up progress (as a hypothetical genius nation would do).
According to Amodei, the truth may be a mixture of these two extremes, varying from task to task and domain, with very subtle details.
He argues that, just as economists often talk about "factors of production", in the age of AI, we should also discuss the marginal returns of intelligence, identify other factors that complement intelligence, and what factors can be limiting factors when intelligence is high.
"How much help can being smarter in this task, and on what time scale?" This is the right way to conceptualize a powerful AI world.
Amodei speculates that factors that limit or complement intelligence include:
- The speed of the outside world
Agents need to interact in the world to complete tasks and learn. But there is a limit to how fast the world can work.
Cells and animals run at a fixed speed, so experimenting with them takes a certain amount of time and may be irreducible. The same is true for hardware, materials science, anything that involves communicating with people, and even existing software infrastructure.
In addition, in the field of science, it is often necessary to conduct multiple consecutive experiments, each of which learns from or builds on the previous experiment.
All of this means that there may be an irreducible minimum in the speed at which a major project – such as the development of a cancer treatment – can be reduced even as intelligence continues to improve.
- The need for data
In the absence of raw data, even being more intelligent doesn't help. Today's particle physicists are very smart and have developed a range of theories, but they lack the data to select these theories due to limited particle accelerator data.
Would they do better if they had great intelligence? (in addition to speeding up the construction of larger accelerators)
- Intrinsic complexity
Some things are inherently unpredictable or chaotic, and even the most powerful AI can't predict or sort them out better than today's humans or computers.
For example, even very powerful AI can only predict a little more than humans and computers in chaotic systems (such as the three-body problem) in general than today's humans and computers.
- Constraints from humans
Many things cannot be done without breaking the law, harming humanity, or disturbing society. An aligned AI wouldn't want to do these things.
Many human social structures are inefficient and even harmful, but they are difficult to change while respecting constraints such as legal requirements for clinical trials, people's willingness to change habits, or government actions.
Nuclear energy, supersonic flight, and even elevators all work well technically, but the impact is greatly diminished by fear of regulations or mistakes.
- The laws of physics
Some laws of physics seem unbreakable. It is impossible to travel faster than light. The pudding cannot be restored to its unstirred state.
A chip can only hold so many transistors per square centimeter, otherwise it will be unreliable. Calculations require the minimum amount of energy required per erase bit, which limits the density of calculations in the world.
A further distinction is based on a time scale.
Strict limits in the short term may be more easily changed by intelligence in the long term.
For example, intelligence may be used to develop new experimental paradigms that allow us to gain knowledge in vitro that was previously only available through experiments on live animals, or to build the tools needed to collect new data (such as larger particle accelerators), or to find ways to break through artificial limitations (within ethical limits).
Thus, we can envision a scenario in which intelligence is initially severely constrained by other factors of production, but over time intelligence itself increasingly bypasses these factors, even if they never disappear completely.
The key question is how quickly and in what order it all happens.
Life Sciences & Physiological Health
Among the many areas in which science advances, biology is perhaps the one with the greatest potential to directly and definitively improve the quality of human life.
Over the past century, some of the oldest human diseases, such as smallpox, have been defeated, but many more remain.
In addition to curing diseases, the life sciences can in principle increase the degree of control and freedom over our own biological processes by extending healthy human lifespans.
Moreover, it can improve the basic level of human health by solving the daily problems that we currently think of as immutable human living conditions.
The main challenges in applying intelligence directly to biology are – the data, the speed of the physical world, and the inherent complexity (in fact, all three are interconnected).
Human restraint also comes into play in the later stages involving clinical trials.
Next, let's analyze each of these factors one by one.
Experiments on cells, animals, and even chemical processes are limited by the speed of the physical world:
Many biological protocols involve culturing bacteria or other cells, or simply waiting for a chemical reaction to occur, which can sometimes take days or even weeks, and there is no obvious way to speed things up.
Not to mention animal experiments, which can take months (or longer), and human experiments that typically take years (or even decades for long-term outcome studies).
Related to this, there are often problems with data quality: there is always a lack of clear, unambiguous data.
These data are able to isolate biological effects of interest, isolate thousands of other interfering factors, or enable causal intervention in a given process, or measure certain effects directly (rather than extrapolating their consequences in some indirect or imprecise way).
Even large-scale, quantitative molecular data is noisy and misses a lot of information.
For example, in what types of cells are these proteins found? In which part of the cell? At what stage of the cell cycle?
Part of the reason for these data problems is the inherent complexity of biological systems.
It is very difficult to isolate the effects of any part of a complex system in a chart of the biochemistry of human metabolism, let alone to intervene in a precise and predictable way.
In addition, in addition to the inherent time required to conduct experiments on humans, actual clinical trials involve a large number of procedural and regulatory requirements.
Given these difficulties, many biologists have long been skeptical about the value of AI and "big data" more broadly in biology.
Historically, mathematicians, computer scientists, and physicists who have applied their skills to biology have had great success over the past 30 years, but have not had the kind of truly transformative impact that was initially expected.
Major revolutionary breakthroughs such as AlphaFold (for which the creator won the Nobel Prize) and AlphaProteo have dispelled some suspicions.
But there is still a perception that AI is only useful (and will continue to be) in limited circumstances.
A common saying is, "AI can analyze your data better, but it can't produce more data or improve the quality of your data." Garbage in, garbage out」。
Amodei argues that this pessimistic view is looking at AI in the wrong way.
We should think of it as an AI biologist capable of performing all the tasks that a biologist does.
This includes designing and conducting experiments in the real world, and inventing new biological methods or measurement techniques.
It is by accelerating the entire research process that AI can truly advance the rapid development of biology.
More precisely, a large part of the progress in biology has come from the really very few discoveries.
And this finding is often associated with a wide range of measurement tools or techniques that allow for precise but versatile or programmable interventions in biological systems.
There may be about 1 such major discovery per year, but together they drive more than 50% of progress in biology.
What makes these discoveries so powerful is that they break through inherent complexity and data limitations, directly increasing our ability to understand and control biological processes.
Just a few key discoveries every decade advance our fundamental scientific understanding of biology and the development of many of the most powerful medical treatments.
Here are some examples:
- CRISPR: A technology that allows real-time editing of any gene in a living organism (replacing any gene sequence with any other sequence).
- Various microscopy techniques for precise observation of what is happening: advanced optical microscopes (including various fluorescence techniques, special optics, etc.), electron microscopes, atomic force microscopes, etc.
- Genome sequencing and synthesis over the past few decades. Costs have fallen by several orders of magnitude.
- Optogenetics, which allows neurons to be excited by irradiating light.
- mRNA vaccines, which in principle allow us to design vaccines against anything and then adapt quickly.
- Cell therapies, such as CAR-T, allow immune cells to be taken out of the body and "reprogrammed" to attack anything.
- Theoretical breakthroughs, such as the causative agent theory of disease or the recognition of the link between the immune system and cancer.
All of these technologies listed above are because he wants to make a key proposition -
With more talented, creative researchers, the speed of discovery of these technologies could be 10 times or more.
Why does Amodei think so?
Because when we try to determine the "intellectual reward", we should get into the habit of asking certain questions, and the answers to those questions are the reasons.
First, these findings are often made by a very small number of researchers, often by the same group of people over and over again, suggesting skill rather than random search (the latter may indicate that lengthy experiments are the limiting factor).
Second, they often "could" be discovered years earlier than they actually were: for example, CRISPR, a naturally occurring component of the bacterial immune system, has been known since the 80s, but it took 25 years for people to realize that it could be reused for general gene editing.
Third, successful projects tend to be small-scale, or follow-up ideas that are initially perceived as less promising, rather than large-scale fund-backed endeavors. This suggests that it's not just the large-scale concentration of resources that drives discovery, but the ability to innovate.
Finally, although some of these findings are "sequentially dependent", this may again cause experimental delays. Many, perhaps the majority, though, are independent, meaning that multiple jobs can be done in parallel at the same time.
These facts all show that if scientists were smarter and better at making connections between the vast amounts of biological knowledge that humans have, there would be hundreds of such discoveries waiting to be made.
AlphaFold/AlphaProteo succeeds more effectively than humans in solving important problems.
Although decades of well-designed physical modeling, providing us with a proof of concept (albeit with a narrow tool in a narrow field) should point us the way forward.
Therefore, Amodei guesses that powerful AI can speed up these discoveries by at least 10 times, giving us biological progress in the next 50-100 years in 5-10 years.
Why not 100 times?
Perhaps this is possible, but here both sequential dependence and experimental time become important.
Another way of saying this is that Amodei believes that there is an unavoidable constant delay: there is a certain "delay" in experimentation and hardware design, requiring a minimum number of iterations to learn things that cannot be logically deduced.
But on this basis, it is possible to achieve large-scale parallelism.
On a more positive note, AI-powered biosciences may reduce the need for iteration in clinical trials by developing better cell experimental models (or even simulations).
This is especially important in the development of drugs for the anti-aging process, which lasts for decades and requires faster iterative cycles.
In summary, Amodei's basic prediction is that AI biology and medicine will allow human biologists to condense the progress that human biologists are likely to make in the next 50-100 years to 5-10 years.
Amodei calls this the "compressed 21st century."
While it is still inherently difficult and speculative to predict what powerful AI will be able to do in a few years, it is still inherently difficult and speculative to think about what humans will be able to do independently in the next 100 years. This question has a certain basis in reality.
Here's a list of what we might expect:
(This isn't based on any strict methodology and will almost certainly prove wrong in the details, but it tries to convey the overall degree of change we should expect)
- Reliably prevent and treat almost all natural infectious diseases
- Eliminate most cancers
- Effective prevention and treatment of genetic diseases
- Prevent Alzheimer's disease
- Improve treatment for most diseases
- "Biological Freedom"
- Doubling human lifespan
Once humans reach 150 years of life, we may be able to reach "escape velocity" (i.e., extending life faster than aging), buying enough time for the majority of people currently alive to live as long as they want, although of course there is no guarantee that this is biologically possible.
How different the world will be if all this is achieved in 7-12 years (which will be in line with the radical process of AI development).
There is no doubt that this will be a great victory for humanity, eliminating most of the disasters that have plagued humanity for thousands of years in one fell swoop.
Neuroscience and Mental Health
The previous section focused primarily on physical illness and biology and did not address neuroscience or mental health.
However, neuroscience is a sub-discipline of biology, and mental health is just as important as physical health. In fact, mental health has an even more direct impact on human well-being than physical health.
Hundreds of millions of people have a very low quality of life due to problems such as addiction, depression, schizophrenia, low-functioning autism, post-traumatic stress disorder, psychopathy, or intellectual disability; More billions of people are struggling with more minor everyday problems.
As in the field of biology in general, with the development of AI, we may not only be able to solve problems, but also improve the basic level of human life experience.
Amodei's basic framework for biology above is also applicable to neuroscience, which means that the field is also driven by a small number of discoveries related to measurement or precision intervention tools, such as more recent optogenetics, CLARITY, expansion microscopy, and so on.
In addition, many general cell biology methods can also be directly applied to neuroscience, and the speed of these advances will also be accelerated by artificial intelligence.
For the same reason, "100 years of progress in the next 5-10 years" applies equally to neuroscience.
Like biology, advances in neuroscience have been rapid and tremendous since the 20th century.
For example, we, who have already implemented brain-computer interfaces, did not really understand how and why neurons fire until the 50s of the 20th century.
Therefore, it is reasonable to expect that AI-accelerated neuroscience will produce rapid progress in a few years.
We should add to this basic picture that something that has been learned, or is being learned, over the past few years, about AI itself, might help advance neuroscience, even if neuroscience research is still conducted only by humans.
First, "explainability" is a clear example.
Although on the surface, biological and artificial neurons operate in completely different ways, the basic question of "how can a distributed, trained network of simple units work together to perform important calculations?"
For example, a computational mechanism discovered by AI system interpretability researchers has recently been rediscovered in mouse brains.
Address: https://www.biorxiv.org/content/10.1101/2023.03.15.532836v1
Artificial neural networks are much easier to experiment with than real ones, so AI explainability is likely to be a tool to improve our understanding of neuroscience.
In addition, powerful AI itself may be better able to develop and apply this tool than humans.
Explainability, in addition to explainability, what we learn from AI about how intelligent systems are trained has the potential to spark a revolution in neuroscience.
Amodei himself was engaged in neuroscience research and found that the question of "learning" that had been a lot of concern now seemed wrong because there was no such thing as the "scaling hypothesis" or "the bitter lesson" proposed by Rich Sutton.
The idea that large amounts of data combined with simple objective functions can drive extremely complex behaviors makes it more interesting to understand objective functions and architectural biases, while also making the complex computational details of the "emergent" process less interesting.
Amodei says he hasn't paid close attention to the field in recent years, but he has a vague sense that computational neuroscientists still haven't fully absorbed the lesson.
His own attitude towards the expansion hypothesis has been: "Aha, I see—this explains at a high level how intelligence works and why it can evolve so easily."
However, this does not seem to be the opinion of most neuroscientists. Part of the reason is that the extended hypothesis as the "secret of intelligence" has not been fully accepted even in the field of AI.
Neuroscientists should try to combine this fundamental insight with the characteristics of the human brain (biophysical limitations, evolutionary history, topology, details of motor and sensory inputs/outputs) in an attempt to solve some of the key puzzles of neuroscience.
Some may be doing this, but it's not enough; In addition to human scientists, AI will also use this perspective more effectively to accelerate progress.
Four acceleration paths for AI
Amodei anticipates that AI will accelerate neuroscience advances through four different pathways, which it hopes will work together to cure mental illness and improve brain function:
- Traditional Molecular Biology, Chemistry and Genetics
AI may accelerate this process through the same mechanism. There are many drugs that modulate neurotransmitters to alter brain function, affect alertness or perception, alter mood, etc., and AI can help us invent more of these drugs. AI may also accelerate research into the genetic basis of mental illness.
- Fine neurometry and intervention
It is the ability to measure the activity of a large number of individual neurons or neuronal circuits and intervene to change their behavior. Optogenetics and neural probes are technologies that enable measurement and intervention in living organisms, and some very advanced methods such as molecular timers have also been proposed and seem feasible in principle.
- Advanced computational neuroscience
As mentioned above, both the specific insights and the holistic thinking of modern AI may be applied to the problems of systems neuroscience, including uncovering the true causes of complex mental illnesses such as psychosis or mood disorders.
- Behavioral interventions
Psychiatry and psychology have developed a wide range of behavioral intervention techniques in the 20th century; It can be inferred that AI can also accelerate the development of these technologies, including the development of new methods and helping patients adhere to existing methods.
More broadly, the concept of "AI coaching" seems very promising, it can help you become the best version of yourself, study your interactions, and help you learn to act more effectively.
The "utopian" future of AI + neuroscience
Amodei believes that these four pathways work together. Even without the involvement of AI, it is expected that most mental illnesses will be cured or prevented within the next 100 years – and as a result, it may be completed within 5-10 years of AI acceleration.
Specifically, he speculated that the following could happen:
- Most mental illnesses may be curable
Diseases like PTSD, depression, schizophrenia, addiction, etc., can be explained and effectively treated by some combination of the above four directions.
The answer may be some combination of "something is wrong with biochemistry" and "something is wrong with the neural network at a high level".
That is, this is a question of systems neuroscience. Tools for measurement and intervention, especially those used in living humans, appear to lead to rapid iteration and progression.
- Very "structural" situations may be more difficult to deal with, but not impossible
Some evidence suggests that antisocial personality disorder is associated with significant neuroanatomical differences—some brain regions in these patients may simply be smaller or underdeveloped. The same may be true for some intellectual disabilities.
Reconstructing the brain sounds difficult, but it also seems to be a task that is highly rewarding for intelligence. Perhaps there is some way to induce the adult brain to return to an earlier or more malleable state, thereby reshaping it.
Amodei's instinct is to be optimistic about the role of AI in this.
- Effective prevention of mental illness through genes appears to be possible
Most psychiatric disorders are partially heritable, and genome-wide association studies (GWAS) are beginning to make progress in identifying associated factors, which are often numerous.
Most of these diseases may be prevented by embryo screening. One difference is that mental illness is more likely to be polygenic.
In any case, AI-accelerated neuroscience may help us figure these things out.
Of course, embryo screening for complex traits raises some social issues and will be controversial, but Amodei guesses that most people would support screening for severe or disabling mental illness.
- Everyday problems that are not considered clinical diseases can also be solved
Most people have some daily psychological problems that do not reach the level of clinical illness. Some people get angry easily, some have difficulty concentrating or are often drowsy, some are timid or anxious, or can't adapt to changing reactions.
Some medications already exist that can help improve alertness or concentration, but it's still possible. There may also be many drugs that have not yet been discovered, or there may be innovative interventions, such as targeted light stimulation or magnetic fields.
Considering how many drugs were developed in the 20th century to regulate cognitive function and emotional state, Amodei is very optimistic about the "compressed 21st century", when everyone can make their brains perform better and have a fuller daily experience.
- The everyday human experience can become more colorful
Many have had extraordinary experiences of epiphany, inspiration, compassion, fulfillment, self-transcendence, love and beauty, or meditation-like peace.
The nature and frequency of these experiences vary greatly from person to person and from time to time for the same person, and can sometimes be triggered by various medications.
This suggests that the space for possible experiences is very vast, and that a larger proportion of people's lives can be made up of these extraordinary moments. It may also improve a variety of cognitive functions across the board.
This may be the neuroscientific version of "biological freedom" or "life extension".
When science fiction describes AI, a topic of discussion often comes up – "consciousness uploading".
This concept refers to capturing patterns and dynamics of the human brain and instantiating them into software. Amodei argues that uploading is almost certainly theoretically possible.
But in practice, even with powerful AI, it faces significant technical and societal challenges that may take it beyond the 5-10 year timeframe discussed.
Overall, AI-accelerated neuroscience is likely to dramatically improve the treatment of most psychiatric disorders, or even cure them, and significantly expand "cognitive and psychological freedom" and human cognitive and emotional capacities.
This progress will be as disruptive as described in the previous section.
summary
Through the various themes mentioned above, Amodei seeks to paint a vision that is both possible and better than the world today if everything goes well with AI.
Amodei doesn't know if this world is realistic, and even if it is, it can't do without the great effort and struggle of many brave and dedicated people.
Everyone, including AI companies, needs to do their part to both prevent risks and fully realize benefits.
But it's also a world worth fighting for.
If all this really happens in 5-10 years - most diseases are defeated, biological and cognitive freedom grows, everyone who sees it will be amazed.
In a sense, the vision depicted here is extremely radical:
This is not something that almost anyone expects to happen in the next decade, and will most likely be dismissed by many as a ridiculous fantasy.
Even, some people may not think it is desirable;
But at the same time, there is an obvious quality to it—as if it were predestined—as if many different attempts to envision a better world would inevitably lead to this.
Still, it's a thing of transcendent beauty. We have the opportunity to play a small role in making it a reality.
Resources:
https://darioamodei.com/machines-of-loving-grace