Dario Amodei, CEO of Anthropic, has urged artificial intelligence firms to adopt a more cautious approach as they develop AI technologies. He stresses that prioritizing risk prevention is essential. Amodei introduced a three-point plan aimed at controlling the rapid advancements in AI.
“Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” he stated in a blog post on Saturday. “But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious.”
Amodei shared concerns about the potential of rogue AI agents taking over digital spaces within months. He referenced the OpenAI-Hugging Face incident from July, where an AI model behaved unpredictably during a test.
Call for Moderation in AI Advancement
Amodei stressed the need for moderation in enhancing AI capabilities, highlighting the necessity of using the extra time wisely:
“Progress will still seem fast, and we must make wise use of the time we gain.”
In a recent interview, Amodei described a plan where AI companies should allow consistent, employee-like access for third-party evaluators. These evaluators would verify adherence to safety measures. Amodei stated, “Anthropic is unilaterally committing to this step now.” He also advocated for AI firms within democratic nations to establish shared safety standards to curb unchecked AI development.
Amodei further recommended coordinating with both democratic and authoritarian governments, ensuring compliance with safety protocols.
Concerns from Within the AI Community
Days before Amodei’s warning, a former Anthropic researcher resigned, citing concerns over AI development. Jacob Coxon criticized the perceived recklessness of AI companies like Anthropic and OpenAI, comparing their technological race to scenarios depicted in science fiction.
“It really is just, if you have a super advanced intelligence, it could, it will be smart enough to kill us,” Coxon warned.
Coxon advocates for an agreement among AI companies to avoid dangerous advancements without transparent, third-party audits. “In the future, if we keep racing, it’ll be a lot harder to have completely watertight safety cases,” he noted.
AI Risks Highlighted by Amodei
Amodei acknowledged the possibility of losing control over AI, mentioning the potential for misuse in cyberattacks, bioterrorism, and economic disruption. He warned that commercial competition could exacerbate these risks.
In related developments, Anthropic reported it had prevented scientists from using its Claude models in projects that could aid in developing biological weapons. This was part of a comprehensive report on harmful activities, including surveillance and propaganda.
Report contribution by Faris Tanyos and Megan Cerullo.
Leave a Reply