2192 karmaJoined Jan 2017


Granted, in principle you could also have a situation where they're less cautious than management but more cautious than policymakers and it winds up being net positive, though I think that situation is pretty unlikely. Agree the consideration you raised is worth paying attention to.

I think there are a few reasons why this is very unlikely to work in practice, at least in society today (maybe it would work if most people were longtermists of some sort):

  • Focusing on such far-off issues would be seen as "silly," and would thereby hurt the political careers of those who focused on them
  • Political inertia is generally strong, meaning that, by default, most things in politics don't happen, even if there's slightly more support than opposition. Here, you'd be shifting the policy proposal to one that wouldn't affect the lives of constituents, and taking away any argument for urgency, making it even harder for the policy to gather enough momentum to overcome the political inertia
  • Due to cognitive dissonance/motivated reasoning/reciprocity/etc, most people who benefit from policies that are on net harmful come to believe that the policy is actually good (e.g., rich homeowners conveniently becoming ideologically NIMBY), and so approaching this from a transactional perspective of "okay, we'll let you personally benefit but let's end the policy after that so it stop hurting everyone else" will likely fail to gain any headway with the current beneficiaries of the policy
  • Etcetera

As an aside, regarding this point:

For instance, voting researchers widely consider the presidential voting system in America to be inferior to many alternatives. But if you want to change it, you require consent from Democrats and Republicans—i.e. the very people who benefit from the status quo.

I decently strongly suspect that the main opposition to changing the presidential system in the US (say, to a parliamentary system) is not that the Democratic and Republican politicians would lose power (I think they'd wind up relatively similarly powerful under that system, though possibly parties would fragment), but instead some combination of status quo bias/Chesterton’s Fence conservatism/sentimentality towards the current system/political inertia/etc. 

A union for AI workers such as data scientists, hardware and software engineers could organise labor to counterbalance the influence of shareholders or political masters.


It's not obvious to me that AI workers would want a more cautious approach than AI shareholders, AI bosses, and so on. Whether or not this would be the case seems to me to be the main crux behind whether this would be net positive or net harmful.

Do people actually think that Google+OpenAI+Anthropic (for sake of argument) are the US? Do they think the US government/military can/will appropriate those staff/artefacts/resources at some point?


I'm pretty sure what most (educated) people think is they are part of the US (in the sense that they are "US entities", among other things), that they will pay taxes in the US, will hire more people in the US than China (at least relative to if they were Chinese entities), will create other economic and technological spillover effects in greater amount in the US than in China (similar to how the US's early lead on the internet did), will enhance the US's national glory and morale, will provide strategically valuable assets to the US and deny these assets to China (at least in a time of conflict), will more likely embody US culture and norms than Chinese culture and norms, and will be subject to US regulation much more than Chinese regulation.

Most people don't expect these companies will be nationalized (though that does remain a possibility, and presumably more so if they were Chinese companies than US companies, due to the differing economic and political systems), but there are plenty of other ways that people expect the companies to advantage their host country['s government, population, economy, etc].

In my mind there are 2 main differences:

  1. Economic degrowth is undesirable, vs pausing AI is at least arguably desirable – climate change is very unlikely to lead to a literal existential catastrophe, "business as usual" tech improvements and policy changes (i.e., without overthrowing capitalism) will likely lead to a clean energy transition as is, economic degrowth would probably kill many more people than it would save, etc. Meanwhile, AI presents large existential risk in my mind, and I think a pause would probably lower this risk by a non-negligible amount.
  2. Economic degrowth is much less politically feasible than an AI pause – first, because people are loss averse, so degrowth would be taking something away from them vs a pause would just be asking them to forgo future progress; second, because the fear of actual existential risk (from AI) may motivate more extreme policies.

I will say, if I thought p(doom | climate) > 10%, with climate timelines of 12 years, then I would be in favor of degrowth policies that seemed likely to reduce this risk. I just think that in reality, the situation is very different than this.

So I notice Fox ranks pretty low on that list, but if you click through to the link, they rank very high among Republicans (second to only the weather channel). Fox definitely uses rhetoric like that. After Fox (among Republicans) are Newsman and OAN, which similarly both use rhetoric like that. (And FWIW, I also wouldn't be super surprised to see somewhat similar rhetoric from WSJ or Forbes, though probably said less bluntly.)

I'd also note that the left-leaning media uses somewhat similar rhetoric for conservative issues that are supported by large groups (e.g., Trumpism in general, climate denialism, etc), so it's not just a one-directional phenomena.

I don't recall seeing many reputable publications label large-scale progressive movements (e.g., BLM, Extinction Rebellion, or #MeToo) as "uninformed mobs"


So progressive causes will generally be portrayed positively by progressive-leaning media, but conservative-leaning media, meanwhile, has definitely portrayed all those movements as ~mobs (especially for BLM and Extinction Rebellion), and predecessor movements, such as for Civli Rights, were likewise often portrayed as mobs by detractors. Now, maybe you don't personally find conservative media to be "reputable," but (at least in the US, perhaps less so in the UK) around half the power will generally be held by conservatives (and perhaps more than half going forward).

the piece has an underlying narrative of a covert group exercising undue influence over the government

My honest perspective is if you're an lone individual affecting policy, detractors will call you a wannabe-tyrant, if you're a small group, they'll call you a conspiracy, and if you're a large group, they'll call you an uninformed mob. Regardless, your political opponents will attempt to paint your efforts as illegitimate, and while certain lines of criticism may be more effective than others, I wouldn't expect scrutiny to simply focus on the substance either way.

I agree that we should have more of an outside game in addition to an inside game, but I'd also note that efforts at developing an outside game could similarly face harsh criticism (e.g., "appealing to the base instincts of random individuals, taking advantage of these individuals' confusion on the topic, to make up for their own lack of support from actual experts").

either because they see the specific interventions as a risk to their work, or because they feel policy is being influenced in a major way by people who are misguided

or because they feel it as a threat to their identity or self-image (I expect these to be even larger pain points than the two you identified)

"how well resourced scientific research institutes are and thus able to get organs that they need for research."
Hmm, are they allowed to buy organs, though? Otherwise, the fact that they're well resourced might not matter much for their access to organs.

Load more