"The new influence web is pushing the argument that AI is less an existential danger than a crucial business opportunity, and arguing that strict safety rules would hand America’s AI edge to China. It has already caused key lawmakers to back off some of their more worried rhetoric about the technology.

... The effort, a loosely coordinated campaign led by tech giants IBM and Meta, includes wealthy new players in the AI lobbying space such as top chipmaker Nvidia, as well as smaller AI startups, the influential venture capital firm Andreessen Horowitz and libertarian billionaire Charles Koch.

... Last year, Rep. Ted Lieu (D-Calif.) declared himself “freaked out” by cutting-edge AI systems, also known as frontier models, and called for regulation to ward off several scary scenarios. Today, Lieu co-chairs the House AI Task Force and says he’s unconvinced by claims that Congress must crack down on advanced AI.

“If you just say, ‘We’re scared of frontier models’ — okay, maybe we should be scared,” Lieu told POLITICO. “But I would need something beyond that to do legislation. I would need to know what is the threat or the harm that we’re trying to stop.”

... After months of conversations with IBM and its allies, Rep. Jay Obernolte (R-Calif.), chair of the House AI Task Force, says more lawmakers are now openly questioning whether advanced AI models are really that dangerous.

In an April interview, Obernolte called it “the wrong path” for Washington to require licenses for frontier AI. And he said skepticism of that approach seems to be spreading.

“I think the people I serve with are much more realistic now about the fact that AI — I mean, it has very consequential negative impacts, potentially, but those do not include an army of evil robots rising up to take over the world,” said Obernolte."

97

2
0

Reactions

2
0
Comments7


Sorted by Click to highlight new comments since:

This is from the same guy who wrote about "the shadowy influence" of EA on AI safety regulation. Obviously he's covering money in politics, but it's interesting how he frames industry lobbying as business as usual but philanthropy as weird and possibly nefarious.

It is always appalling to see tech lobbying power shut down all the careful work done by safety people.

Yet the article highlights a very fair point: that safety people have not succeeded at being clear and convincing enough about the existential risks posed by AI. Yes, it's hard, yes it's a lot about speculations. But that's exactly where impact lies : trying to have a consistent and pragmatic discourse about AI risks, that is not uselessly alarmist or needlessly vague.

The state of the EA community is a good example of that. I often hear that yes, risks are high, but what risks exactly, and how can they be quantified? Impact measurement is awfully vague when it comes to AI safety (and a minor measure, AI governance).

It seems like the pivot towards AI Pause advocacy has happened relatively recently and hastily. I wonder if now would be a good time to step back and reflect on strategy.

Since Eliezer's Bankless podcast, it seems like Pause folks have fallen into a strategy of advocating to the general public. This quote may reveal a pitfall of that strategy:

“I think the more people learn about some of these [AI] models, the more comfortable they are that the steps our government has already taken are by-and-large appropriate steps,” Young told POLITICO.

I hypothesize a "midwit curve" for AI risk concern:

  • At a low level of AI knowledge, members of the general public are apt to anthropomorphize AI models and fear them.

  • As a person acquires AI expertise, they anthropomorphize AI models less, and become less afraid.

  • Past that point, some folks become persuaded by specific technical arguments for AI risk.

It puzzles me that Pause folks aren't more eager to engage with informed skeptics like Nora Belrose, Rohin Shah, Alex Turner, Katja Grace, Matthew Barnett, etc. Seems like an ideal way to workshop arguments that are more robust, and won't fall apart when the listener becomes more informed about the topic -- or simply identify the intersection of what many experts find credible. Why not more adversarial collaborations? Why relatively little data on the arguments and framings which persuade domain experts? Was the decision to target the general public a deliberate and considered one, or just something we fell into?

My sense is that some Pause arguments hold up well to scrutiny, some don't, and you might risk undermining your credibility by making the ones which don't hold up. I get the sense that people are amplifying messaging which hasn't been very thoroughly workshopped. Even though I'm quite concerned about AI risk, I often find myself turned off by Pause advocacy. That makes me wonder if there's room for improvement.

Something that crystallized for me after listening to the A16Z podcast a bit is there are at least 3 distinct factions in the AI debate: the open-source faction, the closed-source faction, and the Pause faction.

  • The open-source faction accuses the closed-source faction of seeking regulatory capture.

  • The Pause and closed-source factions accuse the open-source faction of enabling bioterrorism.

  • The Pause faction accuses the closed-source faction of hypocrisy.

  • The open-source faction accuses the Pause faction of being inspired by science fiction.

  • The closed-source faction accuses the Pause faction of being too theoretical, and insufficiently empirical, in their approach to AI alignment.

If you're part of the open-source faction or the pause faction, the multi-faction nature of the debate might not be as obvious. From your perspective, everyone you disagree with looks either too cautious or too reckless. But the big AI companies like OpenAI, Deepmind, and Anthropic actually find themselves in the middle of the debate, pushing in two separate directions.

Up until now, the Pause faction has been more allied with the closed-source faction. But with so many safety people quitting OpenAI, that alliance is looking less tenable.

I wonder if it is worth spending a few minutes brainstorming a steelman for why Pause should ally with the open-source faction, or at least try to play the other two factions against each other.

Some interesting points from the podcast (starting around the 48-minute mark):

  • Marc thinks the closed-source faction fears erosion of profits due to commoditization of models.

  • Dislike of big tech is one of the few bipartisan areas of agreement in Washington.

  • Meta's strategy in releasing their models for free is similar to Google's strategy in releasing Android for free: Prevent a rival company (OpenAI for LLMs, Apple for smartphones) from monopolizing an important technology.

That suggests Pause may actually have a few objectives in common with Meta. If Meta is mostly motivated by not letting other companies get too far ahead, slapping a heavy tax on the frontier could satisfy both Pause and Meta. And the more LLMs get commoditized, the less profitable they become to operate, and the less investors will be willing to fund large training runs.

It seems like most Pause people are far more concerned about general AI than narrow AI, and I agree with them. Conceivably if you discipline Big AI, that satisfies Washington's urge to punish big tech and pursue antitrust, while simultaneously pushing the industry towards a lot of smaller companies pursuing narrower applications. (edit: this comment I wrote advocates taxing basic AI research to encourage applications research)

This analysis is quite likely wrong. For example, Marc supports open-source in part because he thinks it will cause AI innovation to flourish, and that sounds bad for Pause. But it feels like someone ought to be considering it anyways. If nothing else, having a BATNA could give Pause leverage with their closed-source allies.

Thanks for the link to Open Asteroid Impact. That's some really funny satire. :-D

Thank you! You might like the 3 minute youtube version as well.

Fwiw I think the website played well with at least some people in the open-source faction (in OP's categorization). Eg see here on the LocalLlama reddit. 

[brainstorming]

It may be useful to consider the % of [worldwide net private wealth] that is lost if the US government commits to certain extremely strict AI regulation. We can call that % the "wealth impact factor of potential AI regulation" (WIFPAIR). We can expect that, other things being equal, in worlds where WIFPAIR is higher more resources are being used for anti-AI-regulation lobbying efforts (and thus EA-aligned people probably have less influence over what the US government does w.r.t. AI regulation).

The WIFPAIR can become much higher in the future, and therefore convincing the US government to establish effective AI regulation can become much harder (if it's not already virtually impossible today).

If at some future point WIFPAIR gets sufficiently high, the anti-AI-regulation efforts may become at least as intense as the anti-communist efforts in the US during the 1950s.

Curated and popular this week
Garrison
 ·  · 7m read
 · 
This is the full text of a post from "The Obsolete Newsletter," a Substack that I write about the intersection of capitalism, geopolitics, and artificial intelligence. I’m a freelance journalist and the author of a forthcoming book called Obsolete: Power, Profit, and the Race to build Machine Superintelligence. Consider subscribing to stay up to date with my work. Wow. The Wall Street Journal just reported that, "a consortium of investors led by Elon Musk is offering $97.4 billion to buy the nonprofit that controls OpenAI." Technically, they can't actually do that, so I'm going to assume that Musk is trying to buy all of the nonprofit's assets, which include governing control over OpenAI's for-profit, as well as all the profits above the company's profit caps. OpenAI CEO Sam Altman already tweeted, "no thank you but we will buy twitter for $9.74 billion if you want." (Musk, for his part, replied with just the word: "Swindler.") Even if Altman were willing, it's not clear if this bid could even go through. It can probably best be understood as an attempt to throw a wrench in OpenAI's ongoing plan to restructure fully into a for-profit company. To complete the transition, OpenAI needs to compensate its nonprofit for the fair market value of what it is giving up. In October, The Information reported that OpenAI was planning to give the nonprofit at least 25 percent of the new company, at the time, worth $37.5 billion. But in late January, the Financial Times reported that the nonprofit might only receive around $30 billion, "but a final price is yet to be determined." That's still a lot of money, but many experts I've spoken with think it drastically undervalues what the nonprofit is giving up. Musk has sued to block OpenAI's conversion, arguing that he would be irreparably harmed if it went through. But while Musk's suit seems unlikely to succeed, his latest gambit might significantly drive up the price OpenAI has to pay. (My guess is that Altman will still ma
 ·  · 5m read
 · 
When we built a calculator to help meat-eaters offset the animal welfare impact of their diet through donations (like carbon offsets), we didn't expect it to become one of our most effective tools for engaging new donors. In this post we explain how it works, why it seems particularly promising for increasing support for farmed animal charities, and what you can do to support this work if you think it’s worthwhile. In the comments I’ll also share our answers to some frequently asked questions and concerns some people have when thinking about the idea of an ‘animal welfare offset’. Background FarmKind is a donation platform whose mission is to support the animal movement by raising funds from the general public for some of the most effective charities working to fix factory farming. When we built our platform, we directionally estimated how much a donation to each of our recommended charities helps animals, to show users.  This also made it possible for us to calculate how much someone would need to donate to do as much good for farmed animals as their diet harms them – like carbon offsetting, but for animal welfare. So we built it. What we didn’t expect was how much something we built as a side project would capture peoples’ imaginations!  What it is and what it isn’t What it is:  * An engaging tool for bringing to life the idea that there are still ways to help farmed animals even if you’re unable/unwilling to go vegetarian/vegan. * A way to help people get a rough sense of how much they might want to give to do an amount of good that’s commensurate with the harm to farmed animals caused by their diet What it isn’t:  * A perfectly accurate crystal ball to determine how much a given individual would need to donate to exactly offset their diet. See the caveats here to understand why you shouldn’t take this (or any other charity impact estimate) literally. All models are wrong but some are useful. * A flashy piece of software (yet!). It was built as
 ·  · 16m read
 · 
Over the years, I have learned many things that are rarely taught about doing cost-benefit or welfare analysis. Here are a few things that I often end up repeating when I mentor individuals or teams working on these kinds of projects: A Point Estimate is Always Wrong For any purpose other than an example calculation, never use a point estimate. Always do all math in terms of confidence intervals. All inputs should be ranges or probability distributions, and all outputs should be presented as confidence intervals. Do not start with a point estimate and add the uncertainty later. From day one, do everything in ranges. Think in terms of foggy clouds of uncertainty. Imagine yourself shrinking the range of uncertainty as you gather more data. This Google Sheets Template allows you to easily set up Monte Carlo estimations that turn probabilistic inputs into confidence-interval outputs. Use Google Sheets I have experience programming in half a dozen languages, including R. Sometimes they are useful or necessary for certain kinds of data analysis. But I have learned that for almost all cost-benefit analyses, it is best to use Google Sheets, for several reasons. The main one is transparency. A cost-benefit or welfare analysis is a public-facing document, not an academic one. You should not use esoteric tools unless absolutely necessary. Anyone in your society with basic literacy and numeracy should be able to read over and double-check your work. When you are done and ready to publish, you make your Sheet visible to everyone, and add a link to it in your report. Then anyone can see what you did, and effortlessly copy your code to refine and extend it, or just play around with different priors and assumptions. This transparency also helps improve results and correct mistakes as you are doing the work. The more people review your math, the better it will be. The number of people who are willing and able to look over a spreadsheet is orders of magnitude higher than the