On January 21st, President Trump announced Stargate, a $500 billion project to scale up AI infrastructure in the United States, alongside big tech CEOs Sam Altman (OpenAI), Masayoshi Son (SoftBank), and Larry Ellison (Oracle).

It appears that Trump may have read from a statement prepared for him by the Stargate CEOs:

Together these world-leading technology giants are announcing the formation of Stargate … a new American company that will invest 500 billion dollars at least … creating over 100,000 American jobs, almost immediately. This monumental undertaking is a resounding declaration of confidence in America’s potential, under a new president — let me be, I didn’t say it, they did, so I appreciate that

$100 billion, they say, is to be deployed immediately.

Origins

This isn’t the first time Sam Altman has planned a $100 billion project called Stargate. Last March, there was reporting that Microsoft and OpenAI were planning an unprecedented supercomputer that could cost as much.

The US-based supercomputer, known as "Stargate," would be the centerpiece of a five-phase plan focused on a series of supercomputer installations the companies plan to build in the next six years, the outlet reported. Stargate, which would be phase 5 of the plan, could launch as soon as 2028, people involved in the proposal told The Information.

This hasn’t panned out, or at least some names have been switched, and relations between Microsoft and OpenAI appear to have since weakened, so perhaps we should treat claims made about Stargate 2.0 with some skepticism.

Jobs

In his announcement, President Trump said that Stargate will create over 100,000 jobs in the United States. While big projects like Stargate could create many jobs in the short term, it appears that he was misled on this point. This is not a mechanism to truly create jobs, quite the opposite.

Masayoshi Son himself said during the announcement "I think AGI is coming very, very soon. And then... That's not the goal. After that, artificial superintelligence will come".

The goal of this project is not to cure cancers, like Sam Altman spoke about in his comments, but to build AGI and artificial superintelligence. There are varying definitions of AGI, from “able to outperform most humans economically” to “smarter than the smartest human at everything”. Superintelligence is often defined as an AI system that is more intelligent than all of humanity collectively.

OpenAI defines AGI on their website as “a highly autonomous system that outperforms humans at most economically valuable work”

AGI will be able to replace most human jobs. This is one of several key motivations for investment into AI development. Projects like Stargate are likely to eventually lead to problems of mass unemployment. Fewer jobs, not more.

Sam Altman most surely knows this to be the case — he does not hide that he is trying to build AI more capable and more efficient than every human, the obvious consequence of this being that most jobs will just not be viable for humans to do — yet recently he has been downplaying the impact on jobs, writing in a blog post last September:

most jobs will change more slowly than most people think, and I have no fear that we’ll run out of things to do (even if they don’t look like “real jobs” to us today). People have an innate desire to create and to be useful to each other, and AI will allow us to amplify our own abilities like never before.

During the Stargate announcement, Sam Altman claimed that it would create “hundreds of thousands of jobs”.

Job displacement is something that is already starting to happen, with Salesforce (around 30th largest company by market cap, with over 72,000 employees) CEO Marc Benioff saying that they won’t be hiring any more software engineers in 2025, amid productivity boosts from AI.

In this case, it seems much more likely that Altman is engaging in a form of deception on this topic than the possibility that he actually believes the words he is emitting.

Based on Trump’s statement, and Altman’s own words he provided while standing next to him, this appears to have been mis-sold to Trump as being great for jobs.

Risks

Social disruption is not even the most compelling reason to be skeptical of those aiming to build superintelligence.

Nobel Prize winners, hundreds of top AI scientists, and even the CEOs of the AI companies driving this have warned that AI poses a threat of human extinction.

The current state of the field is that nobody knows how to ensure that AI that is more intelligent, and more powerful than us, will be safe. Nobody even has a particularly good idea for how to attempt this.

Continuing to rush headlong into a world where we are no longer the most powerful species on this planet is something that we should only expect to lead to disaster.

President Trump has spoken about his concern over some of the risks of AI, so he has some awareness. But it seems unlikely he would support a project like this if those pushing him to announce it were frank with him about even their own publicly stated beliefs about the risks.

This isn’t a point I’ve seen discussed elsewhere, I suspect because this form of deception has become normalized. We simply expect that when Sam Altman goes to Trump he tells him AI might cure cancer, and neglects to mention that, oh, by the way, the bad case “is lights out for all of us”.

Subscribe now

Money

So $500 billion has been announced. OpenAI’s statement on their website reads: “We will begin deploying $100 billion immediately.“ Then they must have at least $100 billion, right?

The statement reads: “The initial equity funders in Stargate are SoftBank, OpenAI, Oracle, and MGX.”

Notably, Microsoft is not listed here, but instead as a “technology partner”. Microsoft’s CEO Satya Nadella was asked in an interview in Davos about this, he said

“Microsoft is investing 80 billion dollars in capital each year, I’m not particularly in the details of what they’re investing … all I know is, I’m good for my $80 billion, I’m going to spend 80 billion dollars building out Azure”

Nadella doesn’t say he’s investing in Stargate. The $80 billion is part of an already announced investment by Microsoft, and seems to be for scaling up Azure, rather than a part of Stargate. So it looks like it’s not coming from Microsoft.

Microsoft’s relationship with OpenAI appears to have soured in recent months, with Bloomberg reporting:

Microsoft did pump $750 million into OpenAI’s most recent $6.6 billion funding round. But its refusal to make a larger commitment prompted Altman to look for other partners and to chafe against a contractual exclusivity provision that required OpenAI to get all of its cloud capacity from Microsoft through 2030.

Elon Musk says they don’t have the money. Sam Altman hits back, and not very subtly accuses Musk of lying:

There are two questions here. Do they have the money ($100 billion to deploy now), and does SoftBank have more than $10 billion secured?

There was an article published this week suggesting SoftBank is preparing to invest $25 billion into OpenAI, which would make it OpenAI’s top investor, presumably as part of a separate investment to the Stargate one. This seems like decent evidence that they might have this kind of cash to throw around on AI.

President Trump was asked, in relation to Musk’s comments, whether he thought they had the money. His response was:

I don't know if they do, but you know they're putting up the money. The government's not putting up anything. They're putting up money. They're very rich people, so I hope they do.

I mean Elon doesn't like one of those people, so.

When asked whether it bothered him that Elon Musk criticized the deal, he said: “it doesn’t, he hates one of the people in the deal … one of the people he happens to hate, but I have certain hatreds of people too”

So it seems to be the case that Trump doesn’t actually know, but that he might suspect that Elon Musk is motivated by his feud with Sam Altman. We also got confirmation that Trump doesn’t currently intend to invest public money into the project.

However, The Information reports that OpenAI and Softbank have each committed $19 billion to the project, while Oracle and MGX will each commit $7 billion. That makes $52 billion, far short of the announced $100 billion, and scarcely more than 10% of the $500 billion project. The rest of it is supposed to be sourced from limited partners and debt.

The relationship with MGX, a UAE investment firm, hasn’t received as much attention as the others, but it is interesting, as last February Sam Altman was reportedly running around to UAE investors looking for $7 trillion dollars. That MGX is apparently providing precisely 0.1% of that amount, is quite amusing, but shouldn’t be surprising, given it’s only a $100 billion fund.

Atreides Management’s chief investment officer Gavin Baker wrote the following on Twitter:

Stargate is a great name but the $500b is a ridiculous number and no one should take it seriously unless SoftBank is going to sell all of their BABA and ARM.

SoftBank has $38b in cash, $142b in debt and generates $3ish billion in FCF per BBG. They own $143b in ARM and $18b in BABA. If they start selling ARM, their stake will be worth much less very quickly.

…

If you a professional investor or analyst and you took $500b at face value - or even worse wrote a *note* about this - you should resign in disgrace.

In conclusion, it seems to be the case that Stargate maybe only actually have $52 billion, if that, the rest is sort of hand waved. Sam Altman likely did mislead Trump — putting him in a difficult position of having to defend an announcement he made seemingly without being provided reliable information.

Stargate might get the money in the future, but it doesn’t look like they have it now.

If you enjoyed reading this post, consider sharing it with friends, or on social media, by email, Twitter, LinkedIn, BlueSky, or via Substack’s Notes:

Share

Social links:

• https://x.com/TolgaBilge_/status/1885481485324452089

• https://www.linkedin.com/posts/tolga-bilge_stargate-gate-did-sam-altman-lie-to-president-activity-7291252934300848128-yyVz

• https://bsky.app/profile/tolgabilge.bsky.social/post/3lh3aqenrhc2p

—

In this article I’ll walk you through the major events in AI politics, policy, and governance of the last two years, interspersed with some of my own takes, where we are now, where we’re heading, and what change is needed. A lot has occurred in two years, and I will no doubt have missed things, but here is my attempt.

2023: Awakening

General purpose AI systems approach human-level, AI as a threat to humanity enters the public consciousness, and consensus is built on the risks of AI and the need to address them.

March

We can pinpoint the moment where AI politics really began as around the middle of March 2023. You could probably push it back to 2022 (e.g. with the CHIPS Act), but this is when it entered the public consciousness.

March saw OpenAI’s launch of GPT-4 on the 14th, a significant advance on their GPT-3.5-turbo model that they launched via ChatGPT only 4 months earlier. GPT-4 was multi-modal, able to take both text and images as an input, and scored 86.4% on the MMLU — at the time, a useful benchmark.

Plotting a straight line on state-of-the-art MMLU scores would project us roughly to this level. Indeed, in a never-published forecasting project, I used the same method and predicted GPT-4’s capabilities quite accurately. Even so, the leap from GPT-3.5 shocked and surprised many, including myself, when I first used it.

For many, including myself, GPT-4 did feel approximately human level. Now having used it for longer, its limitations on reasoning are more evident.

This was shortly followed by The Future of Life Institute’s open letter, a week later, which called for all AI labs to immediately pause training AI systems more powerful than GPT-4 for at least 6 months, also specifying that “If such a pause cannot be enacted quickly, governments should step in and institute a moratorium”. The reasoning being that AI systems are rapidly approaching human-level competency, will soon surpass us, and we currently have no way to ensure that we don’t lose control of our civilization, along with a number of other risks we don’t know how to mitigate.

Notably, this call received tens of thousands of signatures, including by top experts in the field such as Yoshua Bengio, Stuart Russell, and John J Hopfield, along with Elon Musk — and made top news headlines in major outlets.

Some have criticized the specific policy asks of this call. For example, one potential flaw is that due to the cost of the compute curve, it arguably wasn’t really feasible to train AI systems significantly more powerful than GPT-4 for a period longer than 6 months anyway. OpenAI has probably only in the last few months of 2024 trained what would be a GPT-5 scale compute AI model (so, ~66x the amount of training compute spent on GPT-4).

On the other hand, in hindsight there was, and very likely still is, vast potential for algorithmic improvements to enable the training of systems significantly more powerful than GPT-4 with a similar amounts of compute.

I think this open letter served as a very important risk awareness-raising and consensus-building moment, I think it’s good that it happened, and I signed it.

April

The UK founded the Frontier AI Taskforce, with an initial budget of £100 million (ca. $125 million USD), which in November evolved into the UK AI Safety Institute, which is tasked with evaluating and ensuring the safety of the most advanced AI systems.

May

The momentum of FLI’s letter continued, with the Center for AI Safety organizing a short joint statement that read: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” In some ways this could be seen as a slight softening of FLI’s message, since there are no concrete policy asks. However this is offset by the strong consensus that was built on mitigating the risk of extinction from AI, with the statement signed by hundreds of top AI experts, and even the CEOs of the leading AI companies themselves.

This statement was then echoed by world leaders such as then-prime minister of the UK Rishi Sunak, and EU Commission President Ursula von der Leyen. Similar acknowledgements of the existential risk posed by AI to us have been made by other world leaders since.

August

At the 2023 BRICS summit, Chinese President Xi Jinping made some interesting comments as part of his speech, which received very little coverage but which I nevertheless think are notable and worth highlighting here:

BRICS countries have agreed to launch the AI Study Group of BRICS Institute of Future Networks at an early date. We need to enable the Study Group to play its full role, further expand cooperation on AI, and step up information exchange and technological cooperation. We need to jointly fend off risks, and develop AI governance frameworks and standards with broad-based consensus, so as to make AI technologies more secure, reliable, controllable and equitable.

Often a narrative is painted that China would never cooperate on AI risks, and would simply go as fast as possible, but this seems to be without much evidence. In fact, as early as 2018, Xi Jinping said “It is necessary to strengthen the analysis and prevention of potential risks in the development of artificial intelligence, safeguard people's interests and national security, and ensure that artificial intelligence is safe, reliable, and controllable.” [Source (Chinese)]

October

Ahead of the first AI Safety Summit in November, there were a couple more open letters published:

• One by IDAIS, in which top western and Chinese AI scientists called for coordinated global action on AI safety research and governance, recommending the defining of clear red lines on AI development.

• Another letter, led by myself, and signed by hundreds of experts, which called for governments to actively respond to the catastrophic risks of AI. It called for an international AI treaty framework to be developed and agreed, proposing several policy measures, including global compute limits and a global CERN for AI Safety, covered in the MailOnline.

Both of these open letters demonstrated the potential for international consensus-building on global AI governance to address the grave risks posed to us by AI.

Since then, and throughout 2024, there has been a large number of open letters pushing in the same direction.

On the 30th, President Biden signed an AI executive order, which among other measures, introduced reporting requirements on models trained above certain compute thresholds (10^26 FLOP, or 10^23 FLOP for bio-sequence data).

November

On the 1st of November, the US established its own AI Safety Intitute.

Early November saw the world’s first AI Safety Summit, hosted in the UK, where world leaders, AI experts, and leaders from industry came together to discuss AI Safety, with countries agreeing the Bletchley Declaration.

In the Bletchley Declaration, countries recognized the potential from AI for catastrophic harm, and committed to work together to address these risks. Notably, it was signed by all major AI powers, including the United States, China, the United Kingdom, and France.

On the 17th, OpenAI’s non-profit board attempted to oust OpenAI’s CEO Sam Altman, after they lost confidence in him. While they had the legal right to do it, and I think they were justified in trying it and we would be in a better place if they had succeeded, it seems that ultimately they probably didn’t really know what they were getting into when they pulled the trigger, and the attempt failed catastrophically. As a result, Ilya Sutskever, Tasha McCauley, and Helen Toner lost their places on the board. It appears that OpenAI is since under full control of a confluence of Sam Altman and Microsoft.

Later in the month, a struggle began over the EU AI Act, which was making its way through the EU legislative process. The EU AI Act eventually did pass, and placed some requirements on the developers of the most powerful AI systems, but big tech companies fought hard to try to exempt themselves from this. They ultimately failed.

Subscribe now

2024: Confusion

One of the biggest stories of 2024 was the constant stream of resignations from OpenAI in the aftermath of November 2023’s board crisis, with key figures such as Ilya Sutskever leaving the company, and half of its AI safety researchers quitting.

The other was the push by some, including Leopold Aschenbrenner, OpenAI CEO Sam Altman, and Anthropic CEO Dario Amodei of a strategy where the US races to superintelligence in a mad dash to secure world domination.

March

The EU AI Act, after much conflict over whether foundation models would be covered, was approved by the EU parliament.

June

Leopold Aschenbrenner published his “Situational Awareness” manifesto. Briefly, it makes the case that artificial superintelligence could be a small number of years away from being developed, and that the US should engage in a Manhattan Project style race to superintelligence, in order for the US to obtain a decisive strategic advantage over rivals, particularly China, and effectively secure full control over the future of the world (and universe) indefinitely.

Aschenbrenner seems not to have seriously considered cooperative solutions to the risks that the development of superintelligence presents, which I consider to be a major flaw of the essay series. The essay series appears to be motivated more by strict ideological faith, showing naivety to the realities of geopolitics, while being overly optimistic on the ability to rapidly build *safe* superintelligence.

That is to say:

• Cooperation between the US and China on managing AI risks seems both from obvious assumptions (neither want to bring about human extinction), and from evidence (Bletchley Declaration, and many other statements and moves by leaders on both sides) to be eminently possible.

• It is far from obvious that if China judged that the US was X number of months from developing artificial superintelligence as part of the suggested strategy that China would not put traditional weapons of mass destruction on the table, though other means of prevention and deterrence are more numerous and I assume would likely be tried first.

• We currently have no way to ensure on a technical level that superintelligence can be built safely (that is to say, that it does not result in our own extinction, or a number of other comparably bad outcomes). It also seems highly unlikely that this problem would be solved in time, and when you are in a racing situation, as proposed by Aschenbrenner, the incentive is always to move as fast as possible, deprioritizing safety in the process.

July

The White House announced that they had secured voluntary commitments to manage AI risks from Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI. These included commitments to internal and external security testing of their AI systems before release, and information sharing on managing AI risks.

Sam Altman wrote an article in the Washington Post echoing Leopold Aschenbrenner’s call for the US, or US-led coalition, to race to superintelligence to dominate the world.

September

Ivanka Trump tweeted out Situational Awareness. This clearly and publicly demonstrates the influence this essay series has had on elite opinion.

A Beijing AI Safety Institute was established.

The first AI treaty was agreed, signed by the US, UK, EU, and others, covering human rights, democracy, and rule of law.

OpenAI launched o1-preview, in my opinion a vast improvement on GPT-4, which is capable of usefully performing reasoning, trading off additional inference compute for increased performance.

In September, and the months leading up, there was an intense battle between AI safety advocates and big tech companies over California’s SB-1047 bill, which would have made leading AI companies liable if their technology caused a catastrophe.

The bill was not perfect, proposing a fixed compute threshold above which AI systems would be covered. This is problematic, as compute is only a proxy for capabilities, and over time algorithmic improvements mean that you can squeeze equivalent amounts of capabilities out of lower and lower amounts of compute. There is likely some fairly fixed critical danger threshold of capabilities, above which systems are inherently potentially dangerous in the hands of e.g. biothreat actors, and so a fixed compute threshold means that in perhaps a small number of years potentially dangerous AI systems would not be covered by the legislation. A better approach, in my opinion, would be to task the regulator with lowering the compute thresholds over time to account for algorithmic improvements.

Nevertheless, the bill seemed to be a good step in the right direction, and would have set an important precedent on legislating to mitigate the catastrophic risks of AI development.

SB-1047 faced significant lobbying efforts from big tech companies and venture capital, including by OpenAI, Meta, Anthropic, Y Combinator, and Andreessen Horowitz, with Anthropic changing their position later on after the bill was watered down. The bill was supported by a broad coalition of AI experts, civil society, artists, and public figures. Ultimately, Governor Gavin Newsom vetoed the bill.

October

ControlAI published a concrete policy plan for humanity to survive AI and flourish “A Narrow Path”. I worked on this as part of the team, mainly on the design of the international AI governance framework that is proposed — I think it’s a good plan.

Anthropic CEO Dario Amodei published his manifesto “Machines of Loving Grace”. Notable was the public shift towards advocating that, as Aschenbrenner and Altman do, a US-led coalition of democracies race to superintelligence to control the future.

Max Tegmark wrote a critical article in response to this, “The Hopium Wars: the AGI Entente Delusion”, where he identified Amodei and Aschenbrenner as proposing a suicide race.

December

OpenAI launched o1, having previously launched o1-preview in September, and announced o3, its next model. o3 shows tremendous gains on the ARC-AGI-1 benchmark, and other benchmarks. I haven’t got to use it yet, and it seems likely to be launched in January, but it looks as though it could be another phase-change in AI capabilities, similar to the launch of GPT-4 back in ‘23.

The benchmarking and communications surrounding the announcement have received criticism from some, including from Gary Marcus. I haven’t spent a long time digging into the details yet, but currently I’m basically persuaded that OpenAI probably did do a bad science. On the other hand, numbers continue to go up, a lot, and this seems like a compelling argument in favor of o3 still being a significantly more powerful AI system than e.g. March 2023 GPT-4.

2025: Racing?

We are in a bad position. Although a broad scientific and political consensus that AI is potentially a very dangerous technology has been obtained, the fact remains: the narrative that the US should race to superintelligence is growing in strength — I expect that to continue.

Furthermore, as the AI industry continues to grow, I expect that ever greater amounts of cash will be dumped into influencing politics towards an accelerationist tendency, and those advocating to slow down and develop AI in a way that doesn’t risk our own extinction will be increasingly out-spent, and uncompetitive. This is a strong reason to push on policy now, or rather yesterday.

Notably, it seems as though Silicon Valley will have an unprecedented influence on the incoming Trump administration. Some of these actors are openly accelerationists, others like Jacob Helberg (a close ally of Sam Altman), have publicly called for a dangerous AGI Manhattan Project — and Helberg will be serving in the administration.

In mitigation, others, such as Elon Musk, are much more reasonable, with Musk having consistently expressed concern about the existential risks of AI over many years. I recently attended a conference where I met dozens of people in this milieu, and my main general learning was that on the whole these kinds of people seem far less accelerationist, and far more concerned about the risks of AI and its intersection with geopolitics than I had anticipated.

Nevertheless, the racing narrative is gaining traction, it has the heads of both OpenAI and Anthropic advocating for it, and will have political actuators inside the Trump administration. Once the US is locked into an AGI Manhattan Project, it could be very difficult to climb down from it, leaving us stuck in a suicide race.

Of course, Trump could be just the man to prevent this disaster. He is a negotiator and deal-maker, and a US-China deal on managing the risks of AI would probably literally be the biggest deal in the history of deals, and of the future too.

What next?

The clock is ticking on our survival. AI development is rapidly advancing, we have no technical way to ensure that superhuman AI systems do not cause catastrophic consequences. Meanwhile, the politics of the issue continue to move in the wrong direction.

While we have acknowledgement from scientists and political leaders of the risks of AI, this isn’t enough. We need to make addressing the risks of AI a top policy priority for every government with relevant AI development in their jurisdictions, but in particular the United States and China.

We need to combat the racing narrative, and avoid locking ourselves into a suicide race. In practice, this means facilitating US-China cooperation on AI risk and associated geopolitical risk.

We should massively grow the field of AI policy and governance. This is not just a problem for tech dudes, there is a vast untapped pool of talent that have spent their lives dedicated to the study of international relations, history, society, politics, and many other fields which may be crucial to getting it right. Everyone has a stake here, and there are plenty that could meaningfully contribute. There are a lot of smart tech dudes, and they are making important contributions, but there is also a notably large contingent that scarcely understand the politics of their own countries, let alone those of others and the interactions between them, and may in some cases as a result be prone to conjuring up nightmarish fever-dreams like Situational Awareness. So, help is needed!

I consider that part of the reason we are in this position now, and at such a late hour, is because pre-2023 those concerned about the risks of AI largely kept quiet and to themselves. It seems that they thought they could avoid politics, and perhaps quietly solve the problem on a technical level. In my judgment, time has run out for technical solutions, and if we are to get them, we need policy to buy more time.

As a goal: I still stand behind the AITreaty.org open letter I led in October 2023, that is to say, we should move on getting an international AI treaty agreed, with broad international support, that addresses the catastrophic risks of AI. I still consider that the policy recommendations it makes are appropriate, most prominently:

• Global Compute Thresholds: Internationally upheld thresholds on the amount of compute used to train any given AI model, with a procedure to lower these over time to account for algorithmic improvements.

• CERN for AI Safety: A collaborative AI safety laboratory akin to CERN for pooling resources, expertise, and knowledge in the service of AI safety, and acting as a cooperative platform for safe AI development and safety research.

• Compliance Commission: An international commission responsible for monitoring treaty compliance.

There are steps that need to be taken along the way to this, and I have a lot more to say about what should be done, but this article is already longer than I had intended, so I will save it for another time.