All of Ryan Kidd's Comments + Replies

Thanks for publishing this, Arb! I have some thoughts, mostly pertaining to MATS:

1. MATS believes a large part of our impact comes via accelerating researchers who might still enter AI safety, but would otherwise take significantly longer to spin up as competent researchers, rather than converting people into AIS researchers. MATS highly recommends that applicants have already completed AI Safety Fundamentals and most of our applicants come from personal recommendations or AISF alumni (though we are considering better targeted advertising to professional engi

Cheers, Nick! We decided to change the title to "retrospective" based on this and some LessWrong comments.

Buck Shlegeris, Ethan Perez, Evan Hubinger, and Owain Evans are mentoring in both programs. The links show their MATS projects, "personal fit" for applicants, and (where applicable) applicant selection questions, designed to mimic the research experience.

Astra seems like an obviously better choice for applicants principally interested in:

• AI governance: MATS has no AI governance mentors in the Winter 2023-24 Program, whereas Astra has Daniel Kokotajlo, Richard Ngo, and associated staff at ARC Evals and Open Phil;
• Worldview investigations: Astra has Ajeya Cot

MATS has the following features that might be worth considering:

1. Empowerment: Emphasis on empowering scholars to develop as future "research leads" (think accelerated PhD-style program rather than a traditional internship), including research strategy workshops, significant opportunities for scholar project ownership (though the extent of this varies between mentors), and a 4-month extension program;
2. Diversity: Emphasis on a broad portfolio of AI safety research agendas and perspectives with a large, diverse cohort (50-60) and comprehensive seminar program;
3. S

Speaking on behalf of MATS, we offered support to the following AI governance/strategy mentors in Summer 2023: Alex Gray, Daniel Kokotajlo, Jack Clark, Jesse Clifton, Lennart Heim, Richard Ngo, and Yonadav Shavit. Of these people, only Daniel and Jesse decided to be included in our program. After reviewing the applicant pool, Jesse took on three scholars and Daniel took on zero.

I think that one's level of risk aversion in grantmaking should depend on the upside and the downside risk of grantees' action space. I see a potentially high upside to AI safety standards or compute governance projects that are specific, achievable, and verifiable and are rigorously determined by AI safety and policy experts. I see a potentially high downside to low-context and high-bandwidth efforts to slow down AI development that are unspecific, unachievable, or unverifiable and generate controversy or opposition that could negatively affect later, bet... (read more)

Thanks Joseph! Adding to this, our ideal applicant has:

• an understanding of the AI alignment research landscape equivalent to having completed the AGI Safety Fundamentals course;
• previous experience with technical research (e.g. ML, CS, maths, physics, neuroscience, etc.), ideally at a postgraduate level;
• strong motivation to pursue a career in AI alignment research, particularly to reduce global catastrophic risk.

MATS alumni have gone on to publish safety research (LW posts here), join alignment research teams (including at Anthropic and MIRI), and found ali... (read more)

• We broadened our advertising approach for the Summer 2023 Cohort, including a Twitter post and a shout-out on Rob Miles' YouTube and TikTok channels. We expected some lowering of average applicant quality as a result but have yet to see a massive influx of applicants from these sources. We additionally focused more on targeted advertising to AI safety student groups, given their recent growth. We will publish updated applicant statistics after our applications close.
• In addition to applicant selection and curriculum elements, our Scholar Support staff, intr

Copying over the Facebook comments I just made.

Response to Kat, intended as a devil's advocate stance:

1. As Tyler said, funders can already query other funders regarding projects they think might have been rejected. I think the unilateralist's curse argument holds if the funding platform has at least one risk-averse big spender. I'm particularly scared about random entrepreneurs with typical entrepreneurial risk tolerance entering this space and throwing money at projects without concern for downsides, not about e.g. Open Phil + LTFF + Longview accessing a ce

We hope to hold another cohort starting in Nov. However, applying for the summer cohort might be good practice, and if the mentor is willing, you could just defer to winter!

I'm not advocating a stock HR department with my comment. I used "HR" as a shorthand for "community health agent who is focused on support over evaluation." This is why I didn't refer to HR departments in my post. Corporate HR seems flawed in obvious ways, though I think it's probably usually better than nothing, at least for tail risks.

In my management role, I have to juggle these responsibilities. I think a HR department should generally exist, even if management is really fair and only wants the best for the world, we promise (not bad faith, just humour).

This post is mainly explaining part of what I'm currently thinking about regarding community health in EA and at MATS. If I think of concrete, shareable examples of concerns regarding insufficient air-gapping in EA or AI safety, I'll share them here.

Yeah, I think that EA is far better at encouraging and supporting disclosure to evaluators than, for example, private industry. I also think EAs are more likely to genuinely report their failures (and I take pride in doing this myself, to the extent I'm able). However, I feel that there is still room for more support in the EA community that is decoupled from evaluation, for individuals that might benefit from this.

I think the distinction you make is real. In the language of this post, I consider the first type of evaluation you mention as a form of "support." Whether someone desires comfort or criticism, they might prefer this to be decoupled from evaluation that might disadvantage them.

The official channel for general questions about MATS is the contact form on our website.

Hi Viktoria! I'm sorry; we dropped the ball on emailing all the applicants who had previously submitted. In hindsight, this was an obvious first thing to do. We did post the extension on the EA Forum and LessWrong posts, and a host of Slack workplaces and Facebook groups, but we should also have sent that email.

Application deadlines have been extended to May 22! Feel free message me or Victor if you have any questions.

I wrote a short blog post a little while ago on preventing low back pain with exercise. I think your problem area report might have missed several important meta-analyses on low back pain. In particular, Huang et al., 2018 and Shiri, Coggon and Hassani, 2017 seem to supersede Steffens et al., 2017, and Lin et al., 2018 seems broader and more recent than NICE, 2016. I think your assessment of the quality of evidence in favour of exercise interventions for low back pain might reasonably update with respect to these references.

Open again for 20k people at $50 each.

https://redefinegifting2021.tisbest.org/

Regarding Magnus' post, which you linked, I partly wrote this article as a response. The evidence base for preventing low back pain with exercise seems much greater than that for adjusting posture, stretching and using ergonomic furniture, which his post also recommends. I wanted to emphasise the importance of exercise as the primary intervention.

Regarding global health, the Happier Lives Institute produced a report on pain in 2020 that identified low back pain as a focus area, but missed my references [3-5]. In particular, my reference [4] seems to supersede their reference (Steffens et al., 2017) and my reference [3] seems broader and more recent than their reference (NICE, 2016). I think their recommendations might reasonably update with respect to these references. https://forum.effectivealtruism.org/posts/3MiMJYwYrhPmNcNBM/problem-area-report-pain-1