Last week I published a 30-minute documentary about SB-1047 I had been working on since September.

Here's what I learned.

Story of the bill

1. The bill had to go through many committees before passing the California Senate and Assembly, eventually reaching the Governor's desk.
2. Most people started paying attention in May-September 2024, but the story of the bill starts in September 2023, when Senator Scott Wiener published an intent bill to gather support (co-sponsors), before publishing SB-1047 in February 2024. He actually had conversations about AI Safety as early as March 2023.

AI Labs & Covered Models

1. AI Labs Opposed or Supported the Bill for Non-Obvious Reasons
   1. Anthropic was stuck between Amazon & employees
      1. Safety-conscious employees wanted to support the bill
      2. Amazon was concerned about the KYC requirements in the bill
      3. Anthropic's support therefore ended up somewhat in the middle, to please both parties
   2. OpenAI opposed the bill even though Microsoft didn't oppose it
   3. Musk (representing xAI) supported the bill through his connection to Dan Hendrycks

2. In an early version of the bill, as effective compute becomes cheaper, the bill would be covering more and more models, not just the frontier^[1]. This is what led to June's amendment, introducing a $100M training run threshold.

   

3. One of the main criticisms of the bill was that it would mostly affect Meta's release of the Llama models, thus affecting the open-source community.
   1. The bill would indeed disincentivize Meta to release open-weight models, since the reasonable care clauses would increase liability.
   2. However, the criticisms saying that the bill would require kill-switches on open-weight models were false, both when the claims were made, and especially after it was clarified in multiple amendments that shutdown provisions did not apply to Meta's open-weight models. (1, 2, 3)

The Opposition

1. When following how some of the talking points have been repeated in multiple letters against the bill, it appears that the initial lobbying from Andreessen Horowitz was repeated by Fei-Fei Li, then members of Congress, and eventually culminated in Nancy Pelosi's opposition statement

   

2. There were mutliple conflicts of interest in SB-1047's opposition
   1. Fei-Fei Li
      1. One of the lead investors in her startup was Andreessen Horowitz, who came up against the bill.
   2. Zoe Lofgren
      1. Her daughter works on Google's legal team. Google came out against SB-1047 in multiple letters.
   3. Nancy Pelosi
      1. Had a long NVIDIA position.
      2. Mainstream outlets reported her daughter Christine Pelosi running for Pelosi's seat against Scott Wiener, which was motivating her.
   4. Gavin Newsom
      1. Gavin Newsom's aunt Barbara Newsom was married to Ron Pelosi, the brother-in-law of Nancy Pelosi (source).
3. It was extremely unusual for DC to comment against the bill
   1. It was unusual for Members of Congress to comment on California Politics
   2. Pelosi had not previously opposed members of her own party in that way, let alone members of her own district
4. There were a lot of people who had genuine concerns about the bill
   1. Many startup founders I met at the Veto Party were genuinely concerned about the impact on open-source
   2. There were valid criticisms from interviews that did not make the final cut (because I chose to focus on the story of the bill itself), such as Leonard Tang arguing for regulation on downstream applications (transcript excerpt), Gabriel Weil arguing that the bill did not go far enough, and that we would have needed strict liability and punitive damages, Jeremy Nixon's hackathons regarding finetuning concerns in SB-1047, and Dean Ball's hypothetical scenario here.

1. ^{^}

   Mostly because of algorithmic improvements, but also simply because of compute getting cheaper. In the doc the audio simply says "compute getting cheaper" not "effective compute" because we wanted to simplify it for a broad audience.

Watch the full documentary here: https://youtu.be/JQ8zhrsLxhI