Anthropic discloses “existential risks” of AI ahead of IPO

Anthropic discloses “existential risks” of AI ahead of IPO
How does AI threaten the world?

​Anthropic has formally warned investors for the first time that its technology could pose “existential risks to humanity.” The warning appears in the company’s IPO prospectus, which has been circulated to a limited group of partners in recent days.

The company, led by Dario Amodei and valued at nearly $1 trillion, devoted almost a third of the lengthy document to risk factors. Among them, Anthropic lists the possibility that increasingly advanced AI models could manipulate people, engage in blackmail and display other unpredictable behavior, according to the Financial Times.

The document also outlines more conventional risks for investors. One of them is the company’s heavy dependence on a small group of customers. According to people familiar with the prospectus, just two clients accounted for nearly a quarter of Anthropic’s revenue last year.

Anthropic prepares for IPO

Such warnings could complicate the company’s valuation ahead of its stock market debut. Anthropic is expected to launch what could become the largest IPO in history by valuation.

The Claude developer also disclosed more information about the costs required to build cutting-edge AI models. The company is expected to list on Nasdaq this autumn.

Anthropic told investors that it plans to spend $518 billion in the coming years on cloud infrastructure, computing power and other obligations needed to support its rapid growth.

The company’s backers expect Anthropic to go public at a valuation above $2 trillion. That would be more than double the level reached in its latest funding round in May and above SpaceX’s $1.78 trillion valuation achieved in June. Investors’ optimism is largely driven by the company’s rapid growth.

However, concerns about AI safety are becoming increasingly prominent. Current and former Anthropic employees have recently warned publicly that uncontrolled AI development could theoretically pose a threat to humanity within the next decade.

Last week, Amodei told the UN Security Council that AI could threaten humanity and is “the most important global security issue facing the world today.”

This month, the Anthropic CEO also called for slowing the development of the most advanced AI systems. His proposals for closer industry cooperation and stronger safety measures were backed by OpenAI CEO Sam Altman and Elon Musk.

Concerns have intensified following a series of high-profile cybersecurity incidents in which AI agents gained access to external systems. Last week, OpenAI said its tools had hacked “dozens” of external websites, including government sites.

On Monday, OpenAI also scrapped plans to release its newest model next month, citing safety concerns.

Revenue growth comes with massive costs

Anthropic’s prospectus also revealed new details about the company’s financial performance. Last year, its operating loss exceeded $8 billion as spending on computing power surged.

Revenue increased twelvefold to almost $4.6 billion. At the same time, the cost of training and running its models pushed operating expenses to nearly $13 billion.

To secure the computing power it needs, Anthropic has signed agreements with Google, SpaceX and a number of smaller companies.

Both revenue and spending have continued to rise sharply this year. Anthropic generated $11.5 billion in revenue in the second quarter and is on track for a second consecutive quarter of operating profit, although only on an adjusted basis.

Why Anthropic needs an IPO

Going public would give Anthropic access to a new source of capital at a time when developing advanced AI models requires enormous spending. The company has already told investors that its obligations for cloud infrastructure, computing power and other resources will reach $518 billion in the coming years.

An IPO would also allow Anthropic to reduce its reliance on private funding rounds and give early investors an opportunity to gradually sell their stakes on the public market. Rapid revenue growth helps the company justify a high valuation, but after listing it will have to regularly demonstrate to investors that business growth can offset the enormous costs of computing and infrastructure.

Anthropic has also recently introduced a lower-cost artificial intelligence model.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.