- I used an LLM to help redraft some arguments for an EA specific audience and it likely contains ca.10% AI-generated text, but I’ve edited/rewritten it extensively and endorse it.
Epistemic status: The empirical claims rest on institutional sources cited at the end. The central argument, that AI safety research as a field does not ask whether its deployment trajectory is compatible with the physical systems it draws on, is visible in what the field's research agenda includes and excludes; I treat this as observable rather than inferred. By "AI safety research" I mean specifically the alignment, x-risk, and long-term safety research agenda as the field currently defines it (most recently: Singapore Consensus on Global AI Safety Research Priorities, June 2025), not compute efficiency or energy research that exists in adjacent technical communities but only barely connect to the safety frame. The structural and sociological reasons for this absence are, I will argue, understandable. Understandable does not make them acceptable. How much the framing would change safety research priorities if adopted is genuinely uncertain. I intend to develop a concrete research agenda in a follow-up post, and am actively seeking engagement with researchers and funders willing to work through what that programme would look like in practice.
I spoke recently with an economist who specialises in AI. Thoughtful, well-read, deeply engaged with the question of how artificial intelligence will reshape economic activity. At some point the conversation turned to growth. I mentioned physical constraints. His answer: “Like physical limits? But they at least allow for multifold increases in GDP, no?”
Well, no. Or only if you look at GDP as it is defined: a function of capital and workforce. You accumulate one, you deploy the other, and the economy expands.
I asked where energy came from in that model. A puzzled pause. Where minerals came from. Another pause. The conversation moved on.
This exchange is not a criticism of one economist. It describes a structural absence in how most educated people, economists, technologists, policymakers, safety researchers, were trained to think about economic systems. The dominant framework treats the economy as a financial system that happens to use some physical inputs. Thermodynamics does not appear in the model. The depletion of ore bodies does not appear. The absorption capacity of forests and oceans does not appear. These are externalities in both the technical and the colloquial sense: things that happen outside the system being studied. This is not a recent oversight. It is baked into the intellectual tradition that most working scientists and researchers inherited, largely without being aware of inheriting it.
The AI safety field has inherited this blind spot. It asks serious questions about model behaviour, misuse, and long-term alignment between machine intelligence and human values. What it does not ask is what world this technology is being built for, whether that world has the physical capacity to support it, and whether the physical systems it depends on are stable enough for current safety assumptions to remain meaningful. This piece tries to open that question.
The economy is a physical system
Nicholas Georgescu-Roegen, working in the 1970s, made an observation whose continued absence from mainstream economics is itself a kind of data point: economic processes transform low-entropy resources into high-entropy waste, and the laws of thermodynamics do not negotiate with financial instruments. The specific entropy framing remains contested in parts of ecological economics. The empirical record makes the same point without requiring any theoretical position on entropy. We are extracting renewable resources faster than they renew themselves, degrading ecosystems faster than they regenerate, and emitting pollution faster than natural systems can absorb it. These are measurable rates, tracked by multiple independent institutional sources, pointing consistently in the same direction across several decades. None of this appears in the models that guide investment decisions, technology deployment, or safety research.
The standard production function (GDP as a function of capital and labour) omits the variable that does the actual physical work. Robert Ayres and Benjamin Warr spent years tracking the relationship between useful work, meaning energy successfully converted into economic output, and GDP growth across the twentieth century. The correlation came out at close to 0.9. Not price, not capital investment: the availability of useful energy. The economy grew because energy was cheap, abundant, and increasingly available. The model that describes that growth does not include the variable that drove it.
Global material extraction makes the same point from a different direction. The UN Environment Programme's International Resource Panel puts annual extraction at 104 billion tonnes in 2023, up from 22 billion in 1970. The economy grew because we found and consumed vastly more physical stuff. Capital and labour organised that consumption. They did not replace it. Experts agree: Eah additional % of GDP requires more mineral extraction than the previous one.
There is a financial dimension that sharpens this. Global debt stood at $348 trillion at the end of 2025, roughly three times global GDP of around $115 trillion according to the Institute of International Finance. Here again, each dollar of new debt generates progressively less GDP than the last. US Federal Reserve data shows this decline running consistently from the early post-war decades, when a dollar of new debt produced more than 70 cents of GDP growth, to well below 10 cents by the time of the 2008 financial crisis. The analysis that popularised these precise figures comes from a non-neutral financial source and the numbers are contested; the direction of the trend in the underlying Federal Reserve data is not. The money being created flows into financial assets and real estate, inflating valuations rather than building productive capacity. The financial system is pricing the future as if the physical constraints described above do not exist.
Three things happening at once
Start with energy. Conventional crude oil production peaked somewhere between 2005 and 2008, according to successive IEA World Energy Outlook editions. What kept total supply figures rising afterward was not new conventional discovery but unconventional sources (fracked tight oil, oil sands, deepwater), each more expensive to extract, each with steeper decline rates than the fields they supplemented. Eighty percent of global primary energy still comes from fossil fuels. Oil, while about a third of the total is still the blood powering all of our economy. The era of cheap, abundant, high-quality energy has been ending for two decades, covered by increasingly costly substitutes.
The ore grades being mined are declining in parallel. In Chile's copper mines, which hold between 40 and 55 percent of known global reserves, the average grade dropped 28.8 percent in a single decade. At today's average of around 0.5 percent copper content, roughly 99.5 percent of everything extracted is waste rock requiring permanent management, generating acid drainage, and consuming water and energy in quantities that scale inversely with ore quality. Energy consumption in those operations rose 46 percent between 2003 and 2013 while copper output rose 30 percent. S&P Global puts gold head grades down 13.4 percent since 2012, copper down 7.5 percent across the industry. The direction has been consistent across decades, metals, and continents.
The scale of what a genuine energy transition would require makes this more acute. Simon Michaux of the Geological Survey of Finland calculated in 2021 what replacing the existing fossil fuel system with electrified alternatives would demand in copper, lithium, cobalt, nickel, and rare earths. His specific figures attracted serious criticism: Carbon Tracker argued, with some justification, that he overstated battery requirements by assuming chemical compositions the market was already moving away from. The rebuttal conceded the underlying point, that resource bottlenecks are real, geopolitically concentrated, and not resolved by efficiency improvements on any timeline relevant to a 2050 target. The critics' own reassuring comparison, that transition minerals weigh 300 times less than the fossil fuels they replace, measures refined metal against fuel stream without accounting for the 200 tonnes of rock that must be mined, crushed, processed, and permanently managed as tailings to produce one tonne of copper at current ore grades. It is a unit comparison, not a physical one. Both Michaux and his critics agree that the mineral extraction implied by the transition is incompatible with a 2050 deadline without fundamental changes to the system being designed. The argument concerned the specific figure, not whether the number is compatible with the deadline.
The third pressure operated quietly for long enough that it barely registered until recently. Forests, soils, and oceans have been absorbing roughly half of human CO₂ emissions for decades, a subsidy provided free by systems we did not build and cannot replace. In 2023, the land carbon sink collapsed to its lowest recorded level since systematic measurement began in 1958, absorbing between 1.5 and 2.6 billion tonnes of CO₂ against a recent average of 9.5 billion. Atmospheric CO₂ grew at 86 percent above the previous year's rate despite fossil fuel emissions rising by less than one percent. A single year's data does not establish a permanent trend; the 10 New Insights in Climate Science 2025, produced by more than 70 scientists across 21 countries, concluded that land and ocean systems are approaching the limits of what they can absorb, and that the 2023 signal is consistent with multiple converging indicators rather than a single anomalous year.
These three pressures are not independent. Lower ore grades require more energy per tonne of useful metal. Higher energy use under a still-fossil-dominated grid adds to the load on absorption systems that are already weakening. A destabilised climate damages the water systems and agricultural soils that mining and food production depend on. The problems compound, they do not queue.
Into this world, we are building AI infrastructure at speed
The International Energy Agency's 2025 report on energy and AI puts data centre electricity consumption growing at 15 percent per year, more than four times faster than all other sectors combined, with AI as the primary driver. Electricity consumed by AI-focused infrastructure alone surged 50 percent in 2025. By 2030, US data centres are projected to consume more electricity than the production of aluminium, steel, cement, and chemicals combined. Natural gas and coal are expected to cover over 40 percent of the additional load through the end of the decade. In material terms, the AI buildout is partly a fossil fuel buildout, landing on a grid that is already struggling to decarbonise.
The hardware side draws on the same mineral base already under pressure from transition demand. Copper, lithium, cobalt, and rare earth elements appear in both the energy transition bill of materials and the AI infrastructure bill of materials. Neither the Michaux calculation nor its rebuttal included AI infrastructure demand. The 2020 Nature Communications study by Sonter et al., which found that without strategic planning, mining threats to biodiversity from renewable energy production alone may surpass those averted by climate change mitigation, was published before the current AI buildout began. Two large-scale industrial demands are now competing for the same geographically concentrated, grade-declining mineral base simultaneously, and no major assessment has modelled them together. This is not a minor gap in the literature. It is a missing calculation that would change the shape of every supply chain and resource governance discussion currently underway.
Computational efficiency has improved substantially and the electricity consumed per AI query has fallen sharply. This does not resolve the system-level picture. The number of tasks, the size of models, and the scale of deployment are growing faster than efficiency gains can offset. The rebound effect, whereby lower cost per unit enables more units, is a well-documented pattern in energy economics and is operating here at scale and speed.
A scenario analysis published in June 2026 by a team of European AI researchers (europe2031.ai) maps the geopolitical and competitive consequences of the AI buildout in considerable detail, projecting global AI compute to 370 gigawatts by 2031. It does not once ask what physical systems that buildout draws on or what it costs the resource base it enters. This is not a criticism of the authors, who are doing exactly what they set out to do. It is an illustration of the inherited frame: rigorous analysis of AI's trajectory conducted without the physical world appearing anywhere in the model. The absence is structural, not deliberate.
Why this question is not being asked
The AI safety field has produced work that matters. Alignment research, interpretability, governance frameworks, threat modelling: these are real contributions to real problems pursued by serious people. The question of whether AI's deployment trajectory is compatible with the physical systems it draws on is nevertheless absent from the research agenda. The Singapore Consensus on Global AI Safety Research Priorities, the field's most authoritative current self-definition, explicitly scopes itself to making AI more trustworthy rather than more powerful, covering general-purpose AI systems and their behaviour. Physical resource constraints, thermodynamic limits, and material supply chains do not appear. This is a data point about where the field currently draws its boundaries, not a criticism of the document itself.
Nobody would design a building without accounting for gravity, not because gravity is exotic specialist knowledge but because it is the physical system the building operates within. The analogous omission in economic modelling, treating the physical world as external to the system being studied, is simply not taught at all. The relationship between economic activity and physical resource systems is a specialist literature sitting at the intersection of ecological economics, industrial ecology, and earth system science. A researcher trained in mathematics, computer science, or analytic philosophy (which describes the majority of the AI safety field) would not encounter it unless they went looking, and there is no obvious reason to go looking for something you have not been told is missing. The blind spot is structural to the education that produced the field.
The field is also young and moving very fast. In any intellectual community, raising an entirely new category of question, one that crosses disciplinary boundaries, resists formalisation in the terms the field currently uses, and does not connect obviously to the model-behaviour problems already on the research agenda, is genuinely difficult. In a community where the core questions are still being established and where capability development creates constant pressure to address immediate problems, it is more difficult still.
Any young, tight-knit intellectual community develops shared assumptions before it has the diversity of background needed to challenge them from within. The EA Forum's own norms explicitly name the scout mindset, the drive to see what is actually there rather than to defend a position, as a core value, precisely because such communities recognise this risk. Physical resource constraints are exactly the kind of question where the scout mindset, applied consistently and across disciplinary lines, would likely produce different conclusions than the current research agenda reflects.
There is a structural question about research funding worth naming carefully. Open Philanthropy is the dominant funder of AI safety research and simultaneously holds a significant equity stake in Anthropic. This is public information, and I raise it not to impute bad faith, as the people involved are thoughtful and their commitment to addressing AI risk is genuine, but to name a structural condition that exists independent of anyone's intentions. Questions that would require interrogating the pace and scale of deployment are structurally harder to fund from inside that arrangement than questions about how to make the technology safer once deployed. This is a risk worth naming as a risk, and I would suggest it is also a reason for funders to want this conversation rather than to avoid it. I would welcome a direct engagement with Open Philanthropy and other major funders about whether a research programme in this space is something they would consider supporting.
Three bodies of EA-adjacent work a reader might cite as closing this gap do not quite reach it. The "Great Energy Descent" series (EA Forum, 2022) covers broadly the same physical-limits terrain but predates the current AI buildout and asks whether civilisation survives resource constraints, not whether safety guarantees engineered for a stable world hold in an unstable one. ALLFED and civilisational resilience research asks whether re-industrialisation is possible after a discrete catastrophe; the concern here is gradual pre-catastrophe degradation, not recovery after a triggering event. The accumulative x-risk literature, discussed below, models AI-induced societal instability, which runs in the opposite causal direction. The gap between the 2022 series and the Singapore Consensus of 2025, which still does not include these questions, is itself evidence of what this piece describes.
The questions that are not being asked
There is a literature on accumulative AI x-risk, the idea that AI risks can manifest gradually through compounding societal erosion rather than a single catastrophic event. That work is valuable and closer to this paper's terrain than most safety research. The critical difference is the direction of causality. The accumulative literature models AI as the cause of instability: incremental AI-induced disruptions erode societal resilience until a triggering event produces irreversible collapse. The question here runs in the opposite direction: what does pre-existing physical instability — instability that precedes and is independent of AI — do to AI safety guarantees that were designed for a stable world? An AI system operating in a world where multiple physical tipping points have already been crossed faces a different risk landscape than one operating in a world broadly continuous with the recent past. That is not a question the accumulative x-risk literature asks, because it treats societal stability as the outcome to be explained rather than the input to be modelled. These are complementary concerns with different research implications. The second has not been addressed.
The AI safety field’s own self-definition, as reflected in the Singapore Consensus and the broader alignment and x-risk research agenda, proceeds from a shared background assumption: that the world into which AI is being deployed is broadly continuous with the world of the recent past. Institutions remain functional. Energy systems remain available. Supply chains remain intact. The physical systems underpinning social and economic stability, while stressed, do not break in ways that fundamentally alter the operating environment for AI systems or the humans overseeing them. This assumption is not stated because it does not need to be stated. It is the water the field swims in, and it is becoming an empirical claim rather than a neutral methodological baseline.
The evidence assembled in this paper, covering ore grade trajectories, energy transition constraints, carbon sink weakening, and the compounding of AI infrastructure demand on top of transition demand, points consistently in the same direction. The question is not whether these trends exist. It is whether safety research that does not account for them is modelling the actual risk landscape or a simplified version of it that may not survive contact with the next decade.
The research this gap calls for has two faces that cannot usefully be separated, because they are the same question approached from opposite ends. First: as the physical systems underpinning economic and social stability weaken, what happens to the risk profile of AI systems operating in that environment? Safety research models AI risk against a stable background. A field serious about long-term safety would also model it against an unstable one. What does misalignment look like when institutions are under severe resource stress? What does concentration of AI capability mean when the physical infrastructure it depends on is geopolitically fragile and supply-constrained? What are the specific failure modes of AI systems deployed into a world where multiple ecological or societal tipping points have already been crossed, not as an exotic scenario, but as a plausible trajectory given current data?
Second: what are the physical requirements of full-scale AI deployment, and are those requirements compatible with the resource base available to meet them alongside the energy transition already underway? This is not a question about marginal efficiency improvements. It is a question about aggregate demand. Nobody has calculated the combined mineral and energy requirements of simultaneous full-scale energy transition and AI infrastructure buildout drawing on the same depleting, geographically concentrated resource base. That calculation does not exist because the field boundaries that would generate it do not exist: energy transition planning and AI deployment planning proceed in separate institutional silos with no shared physical accounting framework. The absence of the calculation is not a weakness in this argument. It is an illustration of it. Producing that calculation is one of the research questions this paper is calling for, not a precondition for calling for it.
I am conscious that a full research programme in this space is beyond the scope of a single post and requires collaboration with researchers who hold technical depth in physical systems modelling, resource economics, and safety research simultaneously. I intend to develop the agenda in a follow-up post. If you are working on adjacent questions, or think the agenda I am pointing toward is misguided, I would welcome the conversation, and am particularly interested in engaging with teams and funders willing to work through what this looks like in practice.
Where the conflict between AI deployment trajectories and physical system capacity actually sits is a precondition for the kind of governance a finite world will eventually require. That question is developed elsewhere. The point here is narrower: the conflict needs to be mapped before it can be governed.
Safe for what world?
An AI system that does not say harmful things, does not pursue misaligned goals, and resists weaponisation is not safe in any meaningful sense if its deployment is accelerating the depletion of the systems that economies and societies depend on, or if the safety guarantees engineered for one world do not hold in a significantly different one.
The world AI is being built for is not stable in the ways the safety research assumes. Its energy supply is transitioning under physical constraint. The mineral base that transition depends on is simultaneously being asked to support an AI infrastructure buildout that was not in any transition model. The carbon absorbers that cushioned decades of emissions are weakening faster than the forecasts predicted. None of this is speculative. It is in the published data of the IEA, the UNEP, the FAO, and the scientific bodies tracking atmospheric carbon.
None of it makes AI development wrong. It makes the absence of these questions from the safety agenda a choice, even if that choice was arrived at through structural inheritance rather than deliberate exclusion. A technology deployed without reference to the physical world it runs on and draws from is not a solution to the problems of that world. It is one more claim on systems that are already overdrawn.
The safety field is serious and the people in it are asking real questions. They have been handed a frame that excludes the physical world, and they are working inside it without knowing it is a frame. At the pace and scale AI is now moving, that gap has stopped being academic.
Safe for what world? The field has not yet asked. The answer will shape everything that follows.
Sources and data
Energy-GDP correlation: Ayres, R. & Warr, B. (2003, 2009), INSEAD / Chalmers University; IEA World Energy Balances Database (1960–2023). Global material extraction: UNEP International Resource Panel, Global Material Flows Database (1970–2023). Global debt: Institute of International Finance, Global Debt Monitor (February 2026). Marginal productivity of debt: Federal Reserve Bank of St. Louis, FRED Database; precise figures from Weiner, K., Monetary Metals (2017–2024), a non-neutral source, treated as illustrative of a trend documented in the underlying data. Conventional oil peak: IEA World Energy Outlook 2008; Aleklett et al. (2010), Energy Policy vol. 38. Ore grade decline: S&P Global Market Intelligence (2024); Calvo, Mudd et al. (2016); Escondida operational data, Earth Resource Investments (2025). Energy transition mineral demand: Michaux, S. (2021), Geological Survey of Finland; rebuttal: Hoekstra, A. & Carbon Tracker (2022). Carbon sink: Ke, Ciais et al. (2024), National Science Review vol. 11; 10 New Insights in Climate Science 2025, Future Earth / The Earth League / WCRP. AI energy demand: IEA, Energy and AI (2025); IEA, Key Questions on Energy and AI (April 2026). Mining and biodiversity: Sonter, L.J. et al. (2020), Nature Communications 11, 4174. AI safety research scope: Singapore Consensus on Global AI Safety Research Priorities (June 2025). Accumulative x-risk: Olejniczak, M. (2024), ‘Two Types of AI Existential Risk: Decisive and Accumulative’, arXiv:2401.07836. European AI compute: Juijn et al., Europe 2031 (June 2026), europe2031.ai.