The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message.

Smaller models sometimes handle critical decision-making involving sensitive data, such as electrical grids and medical records, while bigger models at times handle low-risk activities such as customer service.

39

0
0

Reactions

0
0
Comments14


Sorted by Click to highlight new comments since:
huw
13
1
0

For those that have been following this: Is he serious, or is this just lip service and he's blocking it because he was lobbied by people in the tech industry?

(Not super well-informed) My guess is that it's 95%+ lip service. He doesn't seem like someone with scruples or object-level opinions about most things other than what gives him power. 

One new thing to me in that thread was that the California Legislature apparently never overrides the governor's vetoes. I wonder why this is the case there and not elsewhere.

Newsom's press release and veto message include much more detail and suggest "it's too weak" is not the actual reason.

Reasons mentioned:

  1. Discrimination by model size
    1. "SB 1047 only applies to large models, giving us a "false sense of security about controlling this fast-moving technology. Smaller, specialized models may emerge as equally or even more dangerous"
  2. "Real risks" are limited to critical decision-making, critical infrastructure etc.
    1. "While well-intentioned, SB 1047 does not take into account whether an Al system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data. Instead, the bill applies stringent standards to even the most basic functions - so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology."
    2. Newsom wants to focus on "specific, known" "demonstrable risks to public safety" "rooted in science and fact", like the deepfake laws he signed.

Suggests Newsom is going to be very hostile to any legislation that is designed to deal with X-risk concerns, and that he, frankly thinks they are bullshit. (I personally am also pretty skeptical of X-risk from AI, but I don't want nothing done given how bad the risk would be if it did manifest.) 

This seems pretty bad news from an AI safety perspective :/ 

Any chance to override his veto, or get a similar bill passed soon? 

Some people have talked about trying to get something like that as a ballot initiative, since initial polling suggests that something like this might be very popular with the general public.

Yep, numbers ranged from 60% to 80% support for approving SB 1047, and it was impressively bipartisan, too.

To do a [citizen-initiated] ballot initiative, you stand on the street and ask passerby to sign your petition. Perhaps it would be possible to simultaneously build a mailing list of interested passerby to attend protests and such. That could translate the poll numbers into a stronger street presence.

I talked to people who seem to know about this and ~0 modern ballot initiatives are done this way. You need >500,000 signatures, which is a massive logistical undertaking that's not going to be enacted by a couple of volunteers. 

About a decade ago, I worked collecting signatures for ballot initiatives in California. I worked with a company which contracted with organizations that financially sponsored the ballot initiatives. At the time I was doing it, the sponsor would usually pay from between $1 to $4 per signature. I would stand in an area with lots of foot traffic and try to persuade passerby to sign my petitions. To maximize profits, the typical strategy is to order petitions from highest-paying to lowest-paying, and try to get any given passerby to sign as many petitions as possible. People who sign your petitions need to be registered voters, so if you're on e.g. a college campus with lots of people who aren't registered voters, you can carry voter registration forms in order to register them. But that strategy is more time-consuming and therefore less profitable. There is a process to randomly verify that petition signatories are registered voters to prevent fraud.

I got into this business because a friend of mine said it could be a good way to make extra cash if you find a good place to stand, and a good way to practice charisma. I got out of it because I wasn't making all that much money, and I was worried that bothering people who were just trying to go about their day was making me callous.

California's current petition system seems like a pretty clear perversion of whatever the designers had in mind. I barely remember having any sort of substantial discussion regarding the policy merits of the petitions I was collecting signatures for. The guy who ran the petition company freely admitted to collecting petitions for initiatives he didn't believe in, and said his most effective tactic for collecting a signatures was to emphasize that "this just puts it on the ballot". There was only one time I ever remember a lady who said "This sounds like bad policy so I'm not signing". I respected the heck out of that, even though her reasoning didn't persuade me. (I believe I was trying to collect her signature for the current top-paying petition related to minutiae of car insurance law.)

So overall, I agree that if you just want to put a petition on the ballot as efficiently as possible, and you have the money needed to hire contractors, then that's a good way to go. But I am also not terminally cynicism-pilled. Based on my insider knowledge of the system, I don't see any reason in principle why a group couldn't use the law more like it was intended.

Yes, collecting half a million signatures is a big project. Imagine 1000 volunteers, each committed enough to collect an average of 500 signatures each. But starting a mass movement is also a big project. So if you want to start a mass movement anyways, you might consider combining those objectives, and using contractors to make up any signature shortfall.

The AI Pause protests I've seen haven't struck me as very effective. I remember in the early days, some EAs were claiming that attending early Pause protests would be high impact, if the protests grew over time. Despite the poll numbers, the Pause protests don't seem to be growing much beyond the core EA/LW audience. If growing those protests is a goal, and a mass movement is considered desirable (a big "if", obviously), maybe it's time to embrace the grind and put in the same sort of leg work you see with e.g. vegan activism.

So -- I'm not suggesting volunteer signature collection as a substitute for professional signature collection, so much as I am suggesting volunteer signature collection as a substitute for doing protests. The Pause movement might see more growth if volunteers split off into groups of two and tried to talk to passerby about AI on an individual basis. AI is a hot topic, much hotter than car insurance law, and the hypothesis that passerby are interested in having 1-on-1 conversations about it may be worth cheap testing. Petitioning could serve as an excuse to start conversations which would ideally end in a signature, a mailing list signup, or a new committed volunteer.

I can share more strategy thoughts if people are interested.

I don't think this is the type of thing that armchair theorizing is good for. If you believe it's good to experiment with signature collection and talking to passerby about AI and/or getting more people on PauseAI mailing lists, I encourage you to do so and report back. 

Good analysis of this from PauseAI:

I don't want to presume to paraphrase their analysis into one phrase, but if I were forced to, it would seem to be that there was a lot of pressure on Governor Newsom from powerful AI companies and interests, who also threatened to ruin the bill's sponsor Scott Wiener. 

Still a pity that he couldn't resist the pressure. 

It's kind of pathetic, but this is the reality of politics today. With their money, they really can either make or break a politician, and we voters are not smart enough to avoid being taken in by their negative advertising and dirt-digging. 

It's clear that we need a much stronger movement on this. The other reason he was able to veto this bill is that the vast majority of people do not agree that AI poses a major / existential risk, and so they do not insist on the urgent action we need. 

More context: 
TechCrunch article (not paywalled like the WSJ article)
Response from Senator Wiener on Twitter + discussion

Thanks for sharing! 

 

Curated and popular this week
Garrison
 ·  · 7m read
 · 
This is the full text of a post from "The Obsolete Newsletter," a Substack that I write about the intersection of capitalism, geopolitics, and artificial intelligence. I’m a freelance journalist and the author of a forthcoming book called Obsolete: Power, Profit, and the Race to build Machine Superintelligence. Consider subscribing to stay up to date with my work. Wow. The Wall Street Journal just reported that, "a consortium of investors led by Elon Musk is offering $97.4 billion to buy the nonprofit that controls OpenAI." Technically, they can't actually do that, so I'm going to assume that Musk is trying to buy all of the nonprofit's assets, which include governing control over OpenAI's for-profit, as well as all the profits above the company's profit caps. OpenAI CEO Sam Altman already tweeted, "no thank you but we will buy twitter for $9.74 billion if you want." (Musk, for his part, replied with just the word: "Swindler.") Even if Altman were willing, it's not clear if this bid could even go through. It can probably best be understood as an attempt to throw a wrench in OpenAI's ongoing plan to restructure fully into a for-profit company. To complete the transition, OpenAI needs to compensate its nonprofit for the fair market value of what it is giving up. In October, The Information reported that OpenAI was planning to give the nonprofit at least 25 percent of the new company, at the time, worth $37.5 billion. But in late January, the Financial Times reported that the nonprofit might only receive around $30 billion, "but a final price is yet to be determined." That's still a lot of money, but many experts I've spoken with think it drastically undervalues what the nonprofit is giving up. Musk has sued to block OpenAI's conversion, arguing that he would be irreparably harmed if it went through. But while Musk's suit seems unlikely to succeed, his latest gambit might significantly drive up the price OpenAI has to pay. (My guess is that Altman will still ma
 ·  · 5m read
 · 
When we built a calculator to help meat-eaters offset the animal welfare impact of their diet through donations (like carbon offsets), we didn't expect it to become one of our most effective tools for engaging new donors. In this post we explain how it works, why it seems particularly promising for increasing support for farmed animal charities, and what you can do to support this work if you think it’s worthwhile. In the comments I’ll also share our answers to some frequently asked questions and concerns some people have when thinking about the idea of an ‘animal welfare offset’. Background FarmKind is a donation platform whose mission is to support the animal movement by raising funds from the general public for some of the most effective charities working to fix factory farming. When we built our platform, we directionally estimated how much a donation to each of our recommended charities helps animals, to show users.  This also made it possible for us to calculate how much someone would need to donate to do as much good for farmed animals as their diet harms them – like carbon offsetting, but for animal welfare. So we built it. What we didn’t expect was how much something we built as a side project would capture peoples’ imaginations!  What it is and what it isn’t What it is:  * An engaging tool for bringing to life the idea that there are still ways to help farmed animals even if you’re unable/unwilling to go vegetarian/vegan. * A way to help people get a rough sense of how much they might want to give to do an amount of good that’s commensurate with the harm to farmed animals caused by their diet What it isn’t:  * A perfectly accurate crystal ball to determine how much a given individual would need to donate to exactly offset their diet. See the caveats here to understand why you shouldn’t take this (or any other charity impact estimate) literally. All models are wrong but some are useful. * A flashy piece of software (yet!). It was built as
Omnizoid
 ·  · 9m read
 · 
Crossposted from my blog which many people are saying you should check out!    Imagine that you came across an injured deer on the road. She was in immense pain, perhaps having been mauled by a bear or seriously injured in some other way. Two things are obvious: 1. If you could greatly help her at small cost, you should do so. 2. Her suffering is bad. In such a case, it would be callous to say that the deer’s suffering doesn’t matter because it’s natural. Things can both be natural and bad—malaria certainly is. Crucially, I think in this case we’d see something deeply wrong with a person who thinks that it’s not their problem in any way, that helping the deer is of no value. Intuitively, we recognize that wild animals matter! But if we recognize that wild animals matter, then we have a problem. Because the amount of suffering in nature is absolutely staggering. Richard Dawkins put it well: > The total amount of suffering per year in the natural world is beyond all decent contemplation. During the minute that it takes me to compose this sentence, thousands of animals are being eaten alive, many others are running for their lives, whimpering with fear, others are slowly being devoured from within by rasping parasites, thousands of all kinds are dying of starvation, thirst, and disease. It must be so. If there ever is a time of plenty, this very fact will automatically lead to an increase in the population until the natural state of starvation and misery is restored. In fact, this is a considerable underestimate. Brian Tomasik a while ago estimated the number of wild animals in existence. While there are about 10^10 humans, wild animals are far more numerous. There are around 10 times that many birds, between 10 and 100 times as many mammals, and up to 10,000 times as many both of reptiles and amphibians. Beyond that lie the fish who are shockingly numerous! There are likely around a quadrillion fish—at least thousands, and potentially hundreds of thousands o