All of a_e_r&#x27;s Comments + Replies

Oh sorry! Thank you for clarifying.

What Do AI Safety Pitches Not Get About Your Field?

I'm not sure I understand which part of this post your comment is referring to

Nathan Young

It's a dumb joke about the notion of a Global Warming Contest, as if your aim was to cause global warming.

Is there a “What We Owe The Future” fellowship study guide?

It's been a while since, but from what I remember, my questions were generally in the same range as the framing highlighted by user seanrson's above!
I've also heard objections from people who've felt that predictions about AGI from biological anchors don't understand the biology of a brain well enough to be making calculations. Ajeya herself even caveats "Technical advisor Paul Christiano originally proposed this way of thinking about brain computation; neither he nor I have a background in neuroscience and I have not attempted to talk to neuros... (read more)

Answer by a_e_rSep 01, 20224

Hi! Yes, I created a 6-week reading guide with discussion questions, quotes, recommended readings, and a "take action" section with more resources.

Jordan Arel

This is awesome, thanks Aris!!

Thanks! I'll check it out and add it as a resource!

Thank you! I'm super glad to hear about modifications going wrong! I can make that more explicit on the website.

WWOTF: Is there an existing reading guide or reading group syllabus?

Should be fixed! Thank you!

Most students who would agree with EA ideas haven't heard of EA yet (results of a large-scale survey)

Thank you!

abier

Thanks Pablo and Aris! Currently we don't have plans to do a WWOTF-specific reading group syllabus in time for the start of school, but I'd be excited about possibly working with you on one if you're interested — let's continue over email :)

a_e_r2y7

This is super cool research and it’s great that you all were able to conduct this survey!!

I think it’s great that you all renamed to survey so that it didn’t specifically attract people who looked for words like “effective” or “altruism” or “existential risk.”

Calling for Student Submissions: AI Safety Distillation Contest

Calling for Student Submissions: AI Safety Distillation Contest

Sure :)

Calling for Student Submissions: AI Safety Distillation Contest

I wasn’t anticipating releasing the list (in some part because people may try to pander to a certain judge’s background and in some part to allow myself and the judges more flexibility in adding people last second).

Sending some judge recommendations my way would be great! I think having a variety of readers would be helpful :) Thank you!

Calling for Student Submissions: AI Safety Distillation Contest

Great point! Early on, I had someone more connected than me make a list of potential judges. We have 15 names brainstormed and sectioned off by how much they know about alignment. I can say with pretty high certainty that I imagine we will at least have someone whose full-time job is alignment reading the submissions (likely a person with a CS doctorate), but hopefully, we could get even more expertise :)

ChanaMessinger

Awesome! Will you be releasing the list at some point? Also, if you're still on the lookout for more judges, I can potentially send people your way! If not, great!

Calling for Student Submissions: AI Safety Distillation Contest

I do think it's possible that we might award more prizes retroactively if we recognize that we receive a lot of valuable submissions! Maybe an "honorable mentions" category.

Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality. If I give a couple of examples, people may feel limited to having their work look like those examples. I might say "make your distillation 1,000 words and explain two papers and I'll give you a prize" but 1,500 words on one paper might have made an optimal submission and I wou... (read more)

MaxRa

If they already get a price, I wouldn't call it "honorable mentions" because that unnecessarily diminishes it in my eyes. Just have anything that seems that would get at B- in school be in the same category as the 250$ price? Ah, interesting, I have the opposite intuition!:D I completely agree that you shouldn't give advice about the length of the distillations, but the criteria you mention here just seem really useful and like I'd be surprised if e.g. you find something clearly presented and accessible, and I wouldn't. And I feel like somebody who has spend like ~40 hours reading and discussing AI Safety material (e.g. as part AGI Safety Fundamentals course) could do a reasonably coherent job at rating the understanding and rigor. Originality seems maybe the trickiest, as you probably have to have some grasp of what ideas/framings are already in the water and which aren't.

Calling for Student Submissions: AI Safety Distillation Contest

Hi! I’ve been thinking about this a bit more and I do think I want graduate students to be able to submit! However, since the main audience is meant to be undergraduate students, I may have to be harsher in evaluation or, more excitingly, maybe I could create a new tier for graduate students? For now I’d say feel free to submit and I’ll work out more specifics on my end and make an edit (+ reply to this) if I make official changes!

mariushobbhahn

That sounds very reasonable. Thanks for the swift reply.

Calling for Student Submissions: AI Safety Distillation Contest

The primary audience for this contests is undergrad, but Master’s students are allowed!

Calling for Student Submissions: AI Safety Distillation Contest

Oh, great idea! If nothing else, distill.pub is a great resource for me to list!

mlc

Thanks for pointing that out!

ThomasW

Distill was never really about distillations in the sense this post is referring to. It was a journal that focused on having very high-quality presentation/visualizations. It's also no longer active: https://distill.pub/2021/distill-hiatus/

Calling for Student Submissions: AI Safety Distillation Contest

Thank you! These are thoughtful comments! I think I will try to add more texts and find more readers, as you suggest.

I've been thinking of going into working on creating contests in the future as a potentially serious work project, so I hope to create some contests that can be larger scale then! Right now, I'm rather limited in capacity. Thankfully, I'm connected with some other great university organizers who I've let know about advertising at their schools.

I think it would be tricky to have clear baseline cutoffs for distillation that still c... (read more)

MaxRa

You probably already have seen that the contest was featured on AstralCodexTen, so you might get more obviously good submissions than you have prices for and it would kinda feel like a wasted opportunity to not clearly signal (i.e. with money) to those authors that their work is highly appreciated and that we would love for them to do more of this work.

MaxRa

Nice and nice! :) Hmm, is your worry that distillations that in hindsight seem to be fairly sub-optimal (e.g. with major mistakes or confusing explanations) end up receiving the lowest tier price because there is some noise introduced by the people who rate the distillations? I think this might happen only rarely, for maybe 2 in 100 distillations? I think your list of scoring criteria already goes a long way giving raters a good idea for what solid work looks like. The money for the lowest tier would also not be a lot, maybe 200$. Giving a price to in-hindsight subpar quality work would maybe reduce the prestige of the price a little bit, but I think it's a fairly junior price anyway that mostly encourages and rewards initial solid efforts. Also you still would have the higher tiers for especially good work which would lose little prestige.

UK University Admissions Support Programme

Unfortunately, I created this contest to help build up university groups, so I think keeping the contest limited to enrolled students (including students who are entering college later this year and students who will graduate before the contest ends) would be the best way to ensure that students feel like they have an advantage in the contest. Thank you for clarifying!

mlc

Thanks for the explanation. other thoughts: Abram’s decision theory and Vanessa’s infrabayes work might be good for distillation. Also, might be worth thinking about some type of collab with current distillers, such as Robert Miles or Mark Xu, and the site distill.pub?

Calling for Student Submissions: AI Safety Distillation Contest

This sounds like a really great idea! I'd be curious to know, what are the factors limiting this to being a UK-only project? Would something like capacity or knowledge of other university systems in other countries be the issue? Because this sounds like a project I'd love to see in the US too and think could generally scale quite well!

Hannah Rowberry

Thanks Aris! The limiting factors are as you guess - I’m limited by my own capacity and starting out with the UK purely because that’s my own strongest area of expertise. But likewise hope to scale in both capacity and global reach - thank you for your encouragement! I’m currently effectively in a pilot phase assessing demand and impact prior to potentially scaling further. If so, I’d be looking to either develop my own global admissions expertise and/or find regional admissions specialists - I’d love to hear from anyone who might be interested in getting involved!

Thank you!

Could you clarify what you mean? Do you mean students who are on a break from college, newly admitted students who aren’t yet attending, or something else?

mlc

I am referring to people that chose alternative career paths to AI, autodidacts and independent ML researchers for example.

a_e_r2y4

This is a very good point! We’ll strongly consider those additions in our EAGL app update.

Building off of your ideas of prediction market integration, I think a next step for UpDating could be a developing a community-wide forecasting platform so that EAs could predict the longevity of relationships started on the app. People aren’t well calibrated in relationship predictions and at UpDating we’d like to help with that.

Beyond micromarriages

a_e_r2y6

Hey! As a representative for UpDating, I'm passionate about helping EAs find loving relationships. I'm wondering if I'd be able to partner with you to implement QAWYs as a complementary unit for our app's micromarriage calculator? We currently add 100 micromarriages per one-on-one, but I'd be excited to figure out an estimate in QAWYs as well! Let's get in touch soon!