All of Vael Gates's Comments + Replies

Great idea, thank you Vaidehi! I'm pulling this from the Forum and will repost once I get that done.

I haven't received much feedback on this video yet, so I'm very curious to know how it's received! I'm interested in critiques and things that it does well, so I can refine future descriptions and know who to send this to.

I've been finding "A Bird's Eye View of the ML Field [Pragmatic AI Safety #2]" to have a lot of content that would likely be interesting to the audience reading these transcripts. For example, the incentives section rhymes with the type of things interviewees would sometimes say. I think the post generally captures and analyzes a lot of the flavor / contextualizes what it was like to talk to researchers.

This isn't particularly helpful since it's not sorted, but some transcripts with ML researchers: https://www.lesswrong.com/posts/LfHWhcfK92qh2nwku/transcripts-of-interviews-with-ai-researchers

My argument structure within these interviews was basically to ask them these three questions in order, then respond from there. I chose the questions initially, but the details of the spiels were added to as I talked to researchers and started trying to respond to their comments before they made them.

1. “When do you think we’ll get AGI / capable / generalizable AI / ... (read more)

Indeed! I've actually found that in most of my interviews people haven't thought about the 50+ year future much or heard of AI alignment, given that my large sample is researchers who had papers at NeurIPS or ICML. (The five researchers who were individually selected here had thought about AI alignment uncommonly much, which didn't particularly surprise me given how they were selected.)

A nice followup direction to take this would be to get a list of common arguments used by AI researchers to be less worried about AI safety (or about working on capabilities

I just did a fast-and-dirty version of this study with some of the students I'm TAing for, in a freshman class at Stanford called "Preventing Human Extinction". No promises I got all the details right, in either the survey or the analysis.

—————————————————————————————————

QUICK SUMMARY OF DATA FROM https://forum.effectivealtruism.org/posts/7f3sq7ZHcRsaBBeMD/what-psychological-traits-predict-interest-in-effective

MTurkers (n=~250, having a hard time extracting it from 1-3? different samples):
- expansive altruism (M = 4.4, SD = 1.1)
- effectiveness-focus scale ... (read more)

It's super cool :). I think SERI's funded by a bunch of places (including some university funding, and for sure OpenPhil), but it definitely feels incredible! 

Just wanted to mention that if you were planning on standardizing an accelerated fellowship retreat, it seems definitely worth reaching out to CFAR folks (as mentioned), since they spent a lot of time testing models, including for post-workshop engagement, afaik! Happy to provide names / introductions if desired.

Awesome, thanks! Title is updated. 

Update on my post "Seeking social science students / collaborators interested in AI existential risks" from ~1.5 months ago: 

I've been running a two-month "program" with eight of the students who reached out to me! We've come up with research questions from my original list, and the expectation is that individuals work 9h/week as volunteer research assistants. I've been meeting with each person / group for 30min per week to discuss progress. We're halfway through this experiment, with a variety of projects and progress states-- hopefully you'll see at... (read more)

Update: I've been running a two-month "program" with eight of the students who reached out to me! We've come up with research questions from my original list, and the expectation is that individuals work 9h/week as volunteer RAs. I've been meeting with each person / group for 30min per week to discuss progress. We're halfway through this experiment, with a variety of projects and progress states-- hopefully you'll see at least one EA Forum post up from those students! 

--

I was quite surprised by the interest that this post generated; ~30 people reached... (read more)

I think classes are great given they're targeting something you want to learn, and you're not uncommonly self-motivated. They add a lot of structure and force engagement (i.e. homework, problem sets) in a way that's hard to find time / energy for by yourself. You also get a fair amount of guidance and scaffolding information, plus information presented in a pedagogical order!  With a lot of variance due to the skill and time investment of the instructor, size of class and quality of the curriculum etc. 

But if you DO happen to be very self-driven,... (read more)

(How to independent study) 

Stephen Casper (https://stephencasper.com/) was giving advice today in how to upskill in research, and suggested doing a "deep dive". 

Deep dive: read 40-50 papers in a specific research area you're interested in going into (e.g. adversarial examples in deep NNs). Take notes on each paper. You'll then have comparable knowledge  to people working in the area, after which you do a synthesis project at the end where you write something up (could be lit review, could be more original than that). 

He said he'd trade any class he'd ever taken for one of these deep dives, and they're worth doing even if it takes like 4 months.  

*cool idea

The comment about counterfactuals makes me think about computational cognitive scientist Tobias Gerstenberg's research (https://cicl.stanford.edu), where his research focuses a lot on counterfactual reasoning in the physical domain, but he also has work in the social domain. 

I confess to only a surface-level understanding of MIRI's research agenda, so I'm not quite able to connect my understanding of counterfactual reasoning in the social domain to a concrete research question within MIRI's agenda. I'd be happy to hear more though if you had more detail! 

Thanks so much; I'd be excited to talk! Emailed.

Got my post up :). https://forum.effectivealtruism.org/posts/dKgWZ8GMNkXfRwjqH/seeking-social-science-students-collaborators-interested-in

Also "Artificial Intelligence and Global Security Initiative Research Agenda - Centre for a New American Security, no date" was published in July 2017, according to the embedded pdf in that link!

Additions under "Less technical / AI strategy / AI governance"?

-  https://forum.effectivealtruism.org/posts/WdMnmmqqiP5zCtSfv/cognitive-science-psychology-as-a-neglected-approach-to-ai 
- https://forum.effectivealtruism.org/posts/9kNqYzEAYtvLg2BbR/baobao-zhang-how-social-science-research-can-inform-ai (though this one only has three research questions and isn't focused on generating questions)