All of Chris Leong's Comments + Replies
Should our EA residential program prioritize structured programming or open-ended residencies?
There's more information value in exploring structured programming.
That said, I'd be wary duplicating existing programs; ie. if the AI Safety Fellowship became a knock-off MATS.
What the School of Moral Ambition has achieved is impressive, but it's unclear whether EA should aim for mainstream appeal insofar as SoMA could potentially fill that niche.
"~70% male and ~75% white" — I'm increasingly feel that the way to be cool is to not be so self-conscious about this kind of stuff. Would it be great to have more women on our team? Of course! And for EA to be more global? Again, that'd be great! But talking about your demographics like it's a failure will never be cool. Instead EA should just back itself. Are our demographics ideal? No...
Create nice zones for spontaneous conversations (not sure how to do this well)
I've tried pushing for this without much success unfortunately.
It really is a lot more effort to have spontaneous conversations when almost all pairs are a one-on-one and almost all people by themselves are waiting for a one-on-one.
I've seen attempts to declare a space an area that's not for one-on-ones, but people have one-on-ones there anyway. Then again, organisers normally put up one or two small signs.
Honestly, the only way to stop people having one-on-ones in the area for s...
For most fellowships you're applying to a mentor rather than pursuing your own project (ERA is an exception). And, on the most common fellowships of a few months it's pretty much go, go, go, with little time to explore.
Thanks for the detailed comments.
Maybe the only way to really push for x-safety is with If Anyone Builds It style "you too should believe in and seek to stop the impending singularity" outreach. That just feels like such a tough sell even if people would believe in the x-safety conditional on believing in the singularity. Agh. I'm conflicted here. No idea.
I wish I had more strategic clarity here.
I believe there was a recent UN general assembly where world leaders were literally asking around for, like, ideas for AI red lines.
I would be surprised if a...
I agree that EA might be somewhat “intellectually adrift”, and yes the forum could be more vibrant, but I don’t think these are the only metric for EA success or progress - and maybe not even the most important.
The EA movement attracted a bunch of talent by being intellectually vibrant. If I thought that the EA movement was no longer intellectually vibrant, but it was attracting a different kind of talent (such as the doers you mention) instead, this would be less of a concern, but I don't think that's the case.
(To be clear, I'm talking about the EA ...
Very excited to read this post. I strongly agree with both the concrete direction and with the importance of making EA more intellectually vibrant
Then again, I'm rather biased since I made a similar argument a few years back.
Here's the main differences between what I was suggesting back then and what Will is suggesting here:
- I suggested that it might make sense for virtual programs to create a new course rather than just changing the intro fellowship content. My current intuition is that splitting the intro fellowship would likely be the best option for now
Honestly, I don't care enough to post any further replies. I've spent too much time on this whole Epoch thing already (not just through this post, but through other comments). I've been reflecting recently on how I spend my time and I've realised that I often make poor decisions here. I've shared my opinion, if your opinion is different, that's perfectly fine, but I'm out.
I took a look at the post announcing Epoch.
It was interesting noting this comment by Ofer:
Jaime Sevilla replied:
Additionally, looking at the post itself:
It's up to the reader to form their own judgement, but it certainly seems to me that the AI Safety community was too ready to trust Epoch.
Weak-downvoted; I think it's fair game to say an org acted in an untrustworthy way, but I think it's pretty essential to actually sketch the argument rather than screenshotting their claims and not specifying what they've done that contradicts the claims. It seems bad to leave the reader in a position of being like, "I don't know what the author means, but I guess Epoch must have done something flagrantly contradictory to these goals and I shouldn't trust them," rather than elucidating the evidence so the reader can actually "form their own judgment." Ben_...
The overall summary is pretty good, however:
"Concern for broader implications" - the broader implications of having the discussion.
"Requires considering possible social or political consequences" - of the speech act.
"Look like bias or deflection" - would likely be clearer to say political bias.
"Strict decoupling can enable harmful speech" - not just 'harmful' in the sense of someone's feelings being hurt, but in the worst case, it means standing aside as people begin co-ordinating on genocide.
"What counts as “charged”" - not just charged, but excessively charged.
in many employers’ eyes they would not look as value aligned as someone who did MATS, something which is part of a researcher’s career path anyway.
Yeah, I also found this sentence somewhat surprising. I likely care more about value alignment more than you, but I expect that the main way for people to signal this is by participating in multiple activities over time rather than by engaging in any particular program. I do agree with the OP's larger point though: that it is easier for researchers to demonstrate value alignment given that there are more program...
Great post! I agree with your core point about a shortfall in non-researcher pipelines creating unnecessary barriers and I really appreciated how well you've articulated these issues. Excited to see any future work!
But I spent the next 6 months floundering; I thought and thought about cause prioritisation, I read lots of 80k and I applied to fellowship after fellowship without success
Were you mostly applying to the highly competitive paid fellowships? I don't exactly know what Nontrivial entails (though my impression was that it covered a few differen...
I'm skeptical of your analysis of scenario 3, as I generally buy the orthogonality thesis, leading me to believe that it's possible to be both wise and evil.
At the same time, emergent misalignment seems to suggest that it might be reasonable to expect that an AI that has been nudged to become wise will also be nudged somewhat towards being moral.
On the contrary, I think we should be very careful about imposing morality taxes. I'm not going to say we should never impose them, but not even attempting to think through the unintended consequences is the height of arrogance. I see this bad both from the perspect of leading to bad policy and also bad from the perspective of class relations.
This post does a good job of highlighting the harms from alcohol.
However, I'm strongly suspicious of the implicit framing:
If you talk either about crime policy or drug policy, that’s got to be the number 1 recommendation — just because it’s so easy. It doesn’t cost you anything. You don’t have to kick in anybody’s door. You just have to change a number in the tax code and crime goes down.
This is a quote - rather than the author - but I think the article does the same thing.
Namely, that it takes a very naive view of the subject by focusing on the immediate ...
I would like to suggest that folk not downvote this post below zero. I'm generally in favour of allowing people to defend themselves, unless their response is clearly in bad faith. I'm sure many folk strongly disagree with the OP's desired social norms, but this is different from bad faith.
Additionally, I suspect most of us have very little insight into how community health operates and this post provides some much needed visibility. Regardless of whether you think their response was just right, too harsh or too lenient, this post opens up a rare opportuni...
Unfortunately, we got very little feedback from funders, and frequently none at all. It could be the risks of reaching out to this more senior audience (i.e. the risk of making them less interested in AI safety if the outreach is poor), but this is just a guess. I expect there are a number of factors at play.
Thinking this through: what's novel is not so much the idea that the path AI takes affects non-human welfare, but that it's worth developing this as its own subfield.
And the argument for this is much stronger in the current context: the arguments for rapid AI progress, AI companies not being responsible by default and AI not being aligned by default are much more legible these days.
And that makes it much easier to build energy around this as there seem to be folks in the EA animal welfare crowd who were skeptical about AI/AI risk before, but now see that t...
I agree with this problem, that there are many folk who lack courage, particularly within AI governance, but I don't really see these kinds of comments as being that likely to change things.
There are times when it can make sense to shout at people, but it normally works better when there's a clear and realistic path that's been sketched out[1]. And when this isn't the case, it tends to simply create strife for very little gain.
- ^
Someone really needs to write an article along the lines of "The case for being bold".
If your complaint is that the default assumption is that these are more or less true, well, my claim is that even though normies tend to see this as a negative signal, it's actually a positive signal for those with good epistemics.
That said, it's important to keep in mind that these aren't directly talking about actual systems (the Orthogonality Thesis is about possible systems and Instrumental Convergence is about incentives).
Interesting post.
I think it did a good job of explaining why the metacrisis might be relevant from an EA standpoint (the dialogue format was a great choice!). I made a similar (but different) argument - that Less Wrong should be paying attention to the sensemaking space - back in 2021[1] and it may still be helpful for readers who want to get better sense of the scene[2].
Unfortunately, I'm with Amina. Short AI timelines are looking increasingly likely and culture change tends to take a long time, so the argument for prioritising this isn't looking as ...
As soon as you start charging a fee to be a member, people will become suspicious that you're trying to sign them up because you want their cash, rather than being purely dedicated to charity. It'll also cost you members because they'll have the choice to either spend the money or not feel fully part of the community. This effect will be worse in some countries than others.
If you don't get enough members signing up, then the organisation becomes vulnerable to capture by someone who signs up a bunch of straw members who never show up to meetings except to v...
There's also Founder's Pledge.