This is from the same guy who wrote about "the shadowy influence" of EA on AI safety regulation. Obviously he's covering money in politics, but it's interesting how he frames industry lobbying as business as usual but philanthropy as weird and possibly nefarious.

Something that crystallized for me after listening to the A16Z podcast a bit is there are at least 3 distinct factions in the AI debate: the open-source faction, the closed-source faction, and the Pause faction.

• The open-source faction accuses the closed-source faction of seeking regulatory capture.

• The Pause and closed-source factions accuse the open-source faction of enabling bioterrorism.

• The Pause faction accuses the closed-source faction of hypocrisy.

• The open-source faction accuses the Pause faction of being inspired by science fiction.

• The closed-source faction accuses the Pause faction of being too theoretical, and insufficiently empirical, in their approach to AI alignment.

If you're part of the open-source faction or the pause faction, the multi-faction nature of the debate might not be as obvious. From your perspective, everyone you disagree with looks either too cautious or too reckless. But the big AI companies like OpenAI, Deepmind, and Anthropic actually find themselves in the middle of the debate, pushing in two separate directions.

Up until now, the Pause faction has been more allied with the closed-source faction. But with so many safety people quitting OpenAI, that alliance is looking less tenable.

I wonder if it is worth spending a few minutes brainstorming a steelman for why Pause should ally with the open-source faction, or at least try to play the other two factions against each other.

Some interesting points from the podcast (starting around the 48-minute mark):

• Marc thinks the closed-source faction fears erosion of profits due to commoditization of models.

• Dislike of big tech is one of the few bipartisan areas of agreement in Washington.

• Meta's strategy in releasing their models for free is similar to Google's strategy in releasing Android for free: Prevent a rival company (OpenAI for LLMs, Apple for smartphones) from monopolizing an important technology.

That suggests Pause may actually have a few objectives in common with Meta. If Meta is mostly motivated by not letting other companies get too far ahead, slapping a heavy tax on the frontier could satisfy both Pause and Meta. And the more LLMs get commoditized, the less profitable they become to operate, and the less investors will be willing to fund large training runs.

It seems like most Pause people are far more concerned about general AI than narrow AI, and I agree with them. Conceivably if you discipline Big AI, that satisfies Washington's urge to punish big tech and pursue antitrust, while simultaneously pushing the industry towards a lot of smaller companies pursuing narrower applications. (edit: this comment I wrote advocates taxing basic AI research to encourage applications research)

This analysis is quite likely wrong. For example, Marc supports open-source in part because he thinks it will cause AI innovation to flourish, and that sounds bad for Pause. But it feels like someone ought to be considering it anyways. If nothing else, having a BATNA could give Pause leverage with their closed-source allies.

It is always appalling to see tech lobbying power shut down all the careful work done by safety people.

Yet the article highlights a very fair point: that safety people have not succeeded at being clear and convincing enough about the existential risks posed by AI. Yes, it's hard, yes it's a lot about speculations. But that's exactly where impact lies : trying to have a consistent and pragmatic discourse about AI risks, that is not uselessly alarmist or needlessly vague.

The state of the EA community is a good example of that. I often hear that yes, risks are high, but what risks exactly, and how can they be quantified? Impact measurement is awfully vague when it comes to AI safety (and a minor measure, AI governance).

[brainstorming]

It may be useful to consider the % of [worldwide net private wealth] that is lost if the US government commits to certain extremely strict AI regulation. We can call that % the "wealth impact factor of potential AI regulation" (WIFPAIR). We can expect that, other things being equal, in worlds where WIFPAIR is higher more resources are being used for anti-AI-regulation lobbying efforts (and thus EA-aligned people probably have less influence over what the US government does w.r.t. AI regulation).

The WIFPAIR can become much higher in the future, and therefore convincing the US government to establish effective AI regulation can become much harder (if it's not already virtually impossible today).

If at some future point WIFPAIR gets sufficiently high, the anti-AI-regulation efforts may become at least as intense as the anti-communist efforts in the US during the 1950s.