Note: if you want to discuss some of the content of this episode, or one of the above quotes, I'll be at EAG DC this weekend chatting about AI Governance–feel free to book a meeting!

Agreed!

As Zach pointed out below there might be some mistakes left in the precise numbers, for any quantitative analysis I would suggest reading AI Impacts' write-up: https://aiimpacts.org/what-do-ml-researchers-think-about-ai-in-2022/

Thanks for the corrections!

Can you tell me exactly which numbers I should change and where?

Sorry about that! The AI generating the transcript was not conscious of the pain created by his terrible typos.

Thanks for the quotes and the positive feedback on the interview/series!

Re Gato: we also mention it as a reason why training across multiple domains does not increase performance in narrow domains, so there is also evidence against generality (in the sense of generality being useful). From the transcript:

"And there’s been some funny work that shows that it can even transfer to some out-of-domain stuff a bit, but there hasn’t been any convincing demonstration that it transfers to anything you want. And in fact, I think that the recent paper… The Gato paper from DeepMind actually shows, if you look at their data, that they’re still getting better transfer effects if you train in domain than if you train across all possible tasks."

I think he would agree with "we wouldn't have GPT-3 from an economical perspective".  I am not sure whether he would agree with a theoretical impossibility. From the transcript:

"Because a lot of the current models are based on diffusion stuff, not just bigger transformers. If you didn’t have diffusion models [and] you didn’t have transformers, both of which were invented in the last five years, you wouldn’t have GPT-3 or DALL-E. And so I think it’s silly to say that scale was the only thing that was necessary because that’s just clearly not true."

To be clear, the part about the credit assignment problem was mostly when discussing the research at his lab, and he did not explicitly argue that the long-term credit assignment problem was evidence that training powerful AI systems is hard. I included the quote because it was relevant, but it was not an "argument" per se.

Thanks for the reminder on the open-minded epistemics ideal of the movement. To clarify, I do spend a lot of time reading posts from people who are concerned about AI Alignment, and talking to multiple "skeptics" made me realize things that I had not properly considered before, learning where AI Alignment arguments might be wrong or simply overconfident.

(FWIW I did not feel any pushback in suggesting that skeptics might be right on the EAF, and, to be clear, that was not my intention. The goal was simply to showcase a methodology to facilitate a constructive dialogue between the Machine Learning and AI Alignment community.)

LessWrong has been A/B testing for a voting system separate from karma for  "agree/disagree". I would suggest contacting the LW team to know 1) the results from their experiments 2) how easy it would be to just copy the feature to the EAF (since codebases used to be the same).

Thanks for the thoughtful post. (Cross-posting a comment I made on Nick's recent post.)

My understanding is that people were mostly speculating on the EAF about the rejection rate for the FTX future fund's grants and distribution of $ per grantee. What might have caused the propagation of "free-spending" EA stories:

• the selection bias at EAG(X) conferences where there was a high % of  grantees.
• the fact that the FTX future fund did not (afaik) released their rejection rate publicly
• other grants made by other orgs happening concurrently (eg. CEA)

This post helped me clarify my thoughts on this. In particular, I found this sentence useful to shed light on the rejection rate situation:

 "For example, Future Fund is trying to scale up its giving rapidly, but in the recent open call it rejected over 95% of applications" 

My understanding is that people were mostly speculating on the EAF about the rejection rate and distribution of $ per grantee. What might have caused the propagation of "free-spending" EA stories:

• the selection bias at EAG(X) conferences where there was a high % of  grantees.
• the fact that FTX did not (afaik) release their rejection rate publicly
• other grants made by other orgs happening concurrently (eg. CEA)

I found this sentence in Will's recent post "For example, Future Fund is trying to scale up its giving rapidly, but in the recent open call it rejected over 95% of applications" useful to shed light on the rejection rate situation.