PolicyPolicy
Posts tagged Policy
Quick takes
The policy topic is very broad, covering any post about improving government policy (in developing and developed countries alike).
Governments are typically committed to the notion that their policies should be effective. This means that members of the effective altruist community can be in a good position to help governments reach their aims. Moreover, the fact that governments are very powerful, and control significant proportions of world GDP, suggests that helping policy-makers can be a high-value strategy. This strategy can be pursued either from the outside—by effective altruist organizations which advise policy-makers—or from the inside—by policy-makers who try to do the most good possible.
Some of the highest-impact reforms affect people who are less able to advocate for their own interests, such as prisoners or migrants. Other policies, like macroeconomic policy and land use reform, have effects that are somewhat diffuse and non-obvious, which makes it difficult to assemble groups to lobby for change. The more mainstream focus areas of global poverty and health, animal welfare and existential risk could also be addressed using political advocacy.
Tentative thesis: China is unlikely to accept a subordinate position in AI capabilities to the US, just as the US is unlikely to accept a subordinate position in AI capabilities to China or anyone else.
This thesis suggests that there are two likely paths for the future international AI (non)regulation:
1) a continuing race between the US and China, perhaps joined by some late entrants, possibly with some rules established (e.g. mutual ban on autonomous weapons or something like that), or
2) a “pause or stop deal” that will establish some sort of ceiling on capabilities, which will be equal for China and the US.
What I think is far less likely is a deal in which China would accept lower AI capabilities than the US.
I simply don’t see a good reason why Chinese leaders would accept subordinate position. They know that China has the ability to catch-up to the US in various technological domains, as evidenced by the fact that in, like, 1990, China was behind in more or less everything, and now they are pushing the technological frontier in many areas.
Some estimates floating around the web suggest that, if the US would stop AI development now, China could reach current US capabilities within months. That is maybe overly optimistic/pessimistic depending on where you stand, but I very much doubt that catch-up would take more than a decade. And Chinese government, being patriotic about the abilities of the Chinese nation, probably will not have an absurdly pessimistic estimate.
Moreover, the Chinese regime is in many ways oriented around this idea of catching up (this is a big difference between China and EU). It is a central plank of the official historical narrative of the People’s Republic of China (see for example preamble to their constitution: https://english.www.gov.cn/archive/lawsregulations/201911/20/content_WS5ed8856ec6d0b3f0e9499913.html) that the century between First Opium War and the Communist victory in the Chinese civil war in 1949 was the century of humiliation, during which various imperialist powers exploited China due to the fact that it was technologically backward country unable to stand up to them. I happen to think this narrative is largely correct, but even if it isn’t, what matters is that Chinese government believes it and is determined to never again allow China to be put in the position where a technologically superior power could walk over it.
This has some implications:
A) Since China is currently behind, any sort of “pause or stop deal” between the US and China would likely have to involve either the US stopping further development and allowing China to catch up, or the US somehow crippling its own capabilities to a current Chinese level. Of course, for either option, the US would have to place a lot of trust in the Chinese government.
B) As long as China is behind, export controls on hardware and other attempts to limit Chinese capabilities are at best of doubtful utility as a way to get to a "pause or stop" agreement. In fact, imho there are making a deal less likely. They antagonize Chinese leaders, make them even more paranoid and less inclined to a constructive agreement. But also, to the extent they are successful in limiting Chinese capabilities, they make it less likely to that the US government would agree to an equal deal, since that would mean sacrificing larger lead, which is harder to politically accept for the US. Perhaps this is a reason why nuclear arms treaties in the Cold war were agreed only after the USSR more or less caught up to US nuclear arsenal?
Now, attempts to limit Chinese capabilities can still be seen as useful if you think it is important for the US to “win” the competition, for example because there are reasons to believe US AI will be safer. Or because you think the world would be a better place if the US “wins”, even though both countries would develop similarly dangerous systems. Or of course if you think that the US victory would be preferable even if American AI would be more dangerous than Chinese AI.
Btw. for a well written snapshot of Chinese opinion/vibes, I recommend this: https://www.chinatalk.media/p/how-anthropic-became-chinas-goliath
Rather than total installed GW’s, it is much more useful to look at data centre installed capacity as a share of regional average power demand to reveal where political strains of the AI build-out are likely to appear first. Helpfully, the IAE provides these statistics: https://www.iea.org/data-and-statistics/data-tools/energy-and-ai-observatory?tab=Energy+for+AI 
I've heard several enterprise leaders describe the Hugging Face attacks as an “accident.” I think that framing is worth challenging.
An accident suggests something unforeseen happened without meaningful human agency. But security failures don't happen in a vacuum. They emerge from systems, incentives, architectures, controls, decisions, and human behavior. Calling an attack an “accident” can inadvertently turn a failure of those systems into an event that simply happened.
There is a useful precedent in road safety. We moved away from calling vehicle-related injuries “accidents” toward terms like “crash” and “collision” because the language better reflects causality. A collision can be predictable and preventable based on road design, infrastructure, vehicle characteristics, and human behavior. That framing creates an obligation to ask: what conditions produced the outcome, who had agency over them, and what needs to change?
AI security deserves the same discipline.
If we are serious about humans remaining in charge, our language should not erase human agency when something goes wrong. “Attack,” “failure,” “vulnerability,” and “incident” may be uncomfortable words—but they preserve the causal chain that allows us to investigate what happened, assign responsibility appropriately, and reduce the probability of recurrence.
One of my biggest regrets in EA is not having spoken out more boldly about how one of the most pressing outreach concerns by a lot was political diversity and how major EA figures/orgs openly supporting Democratic politics as an EA intervention was a major error.
Many people might of course be sympathetic to such causes and might well support them privately, but I think it is critically important for EA to be a big tent that is mostly politically neutral, and in recent years we've lost a lot of ground there.
I have believed this and spoke out about it in minor ways and in private since seeing talk of phone banking for Hillary as an EA intervention -- which struck me as surprising misconduct at the time -- but I now think I probably should have made a bigger deal of it.
I'm not sure my speaking up would have solved things, but I regret not being more open about it since things seem to have gone really wrong on this front recently. Unfortunately, this error now seems very relevant to the current AI safety debates...
edit: update:
According to people I know who have personally seen the flyers, there are flyers offering to pay people to protest a recent Sam Altman appearance (unclear why they're mad at Sam Altman, there are many reasons why they might be mad at him). However, the protest was cancelled because of ???. https://www.wunc.org/education/2026-09-02/protest-chapel-hill-unc-g20-innovation-technology-sam-altman-elon-musk-artificial-intelligence gives a bit of coverage of the story.
If this comes from the EA side of things, people should know that the optics of "paid protests" are widely regarded as absolutely horrible! It's a good way to a) make it look like you have no popular support, b) alienate potential supporters by affiliating your movement with random people who need $30 badly, and c) show that you are not taking this politics thing seriously.
If this doesn't come from the EA side of things, maybe some people should look into where it's coming from?
We know that unhatched chick embryos can suffer late in development, but current evidence is mixed on mid-term development around day-13. The in-ovo sexing technology used by the NestFresh brand in the US works on day-13 (which is banned in Germany). Claude's overview is here.
Given that public policy and industry tech adoption is still in early stages outside Europe, it seems to me that it is great time to fund further research on chick embryo sentience. Before day-13 becomes standard.