Without saying much about the merits of various commenters' arguments, I wanted to check if this is a rhetorical question:
Is anyone on this forum in a better position than the Secretary-General of the UN to analyze, for example, the impact of Israel's actions on future, unrelated conflicts?
If so, this is an appeal to authority that isn't very helpful in advancing this discussion. If it's an actual question, never mind.
My understanding, though I'm not sure the board ever publicly confirmed this, was they decided that Larissa was acting on behalf of Leverage Research, and hence contrary to the best interests of CEA, and they wanted to stop the entryism.
Thanks for writing this up, I was skeptical about Scott‘s strong take but didn’t take the time to check the links he provided as proof.
Thanks for the write-up. Just adding a note on how this distinction has practical implications for how to design databases containing hazardous sequences that are required for gene synthesis screening systems.
With gene synthesis screening, companies want to stop bad actors from getting access to the physical DNA or RNA of potential pandemic pathogens. Now, let's say researchers find the sequence of a novel pathogen that would likely spark a pandemic if released. Most would want this sequence to be added to synthesis screening databases. But some also want ...
I skimmed it, and it looks good to me. Thanks for the work! A separate post on this would be cool.
Flagging that I approve this post; I do believe that the relevant biosecurity actors within EA are thinking about this (though I'd love a more public write-up of this topic). Get in touch if you are thinking about this!
I'm excited that more people are looking into this area!
Flagging that I only read the intro and the conclusion, which might mean I missed something.
High-skilled immigration
From my current understanding, high-skilled immigration reform seems promising not so much because of the effects on the migrants (though they are positive) but mostly due to the effect on the destination country's GDP and technological progress. The latter has sizeable positive spillover effects (that also accrue to poorer countries).
Advocacy for high-skilled immigration is less c...
Comment by Paul Christiano on Lesswrong:
...""RLHF and Fine-Tuning have not worked well so far. Models are often unhelpful, untruthful, inconsistent, in many ways that had been theorized in the past. We also witness goal misspecification, misalignment, etc. Worse than this, as models become more powerful, we expect more egregious instances of misalignment, as more optimization will push for more and more extreme edge cases and pseudo-adversarial examples.""
These three links are:
- The first is Mysteries of mode collapse, which claims tha
This post reads like it wants to convince its readers that AGI is near/will spell doom, picking and spelling out arguments in a biased way.
Just because many ppl on the Forum and LW (including myself) believe that AI Safety is very important and isn't given enough attention by important actors, I don't want to lower our standards for good arguments in favor of more AI Safety.
Some parts of the post that I find lacking:
..."We don’t have any obstacle left in mind that we don’t expect to get overcome in more than 6 months after efforts are invested to
Thanks for writing this up. I just wanted to note, the OWID graph that appears while hovering over a hyperlink is neat! @JP Addison or whoever created that, cool work.
Flagging that I'm only about 1/3 in.
Regarding this paragraph:
" An epistemically healthy community seems to be created by acquiring maximally-rational, intelligent, and knowledgeable individuals, with social considerations given second place. Unfortunately, the science does not bear this out. The quality of an epistemic community does not boil down to the de-biasing and training of individuals;[3] more important factors appear to be the community’s composition, its socio-economic structure, and its cultural norms.[4]"
When saying that the science doesn't bea...
Appreciated this post! Have you considered crossposting this to Lesswrong? Seems like an important audience for this.
I just wanted to note that I appreciated this post and the subsequent discussion, as it quickly allowed me to get a better model of the value of antivirals. Publicly visible discussions around biosecurity interventions are rare, making it hard to understand other people's models.
I appreciate that there are infohazards considerations here, but I feel it's too hard for people to scrutinize the views of others because of this.
Appreciated the 5-minute summary; I think more reports of this length should have two summaries, one TL;DR, the other similar to your 5 min summary.
Let's phrase it even more explicitly: You trust EVF to always make the right calls, even in 10 years from now.
The quote above (emphasis mine) reads like a strawman; I don't think Michael would say that they always make the right call. My personal view is that individuals steering GWWC will mostly make the right decisions and downside risks are small enough not to warrant costly governance interventions.
This point is boring, but I don't think Twitter gives an accurate picture of what the world thinks about EA. I still think there is a point in sometimes reacting to bad-faith arguments and continuing to i) put out good explanations of EA-ish ideas and ii) writing up thoughts on what went wrong. But communicating too fast, before, e.g., we have an improved understanding of the FTX situation, seems bad.
Also, as a semi-good analogy for the Wytham question, the World Economic Forum draws massive protests every year but is still widely respected among important circles.
Probably fits max 50-100 people, though I have low certainty and this might change in the future. I think it‘s designed to host smaller events than the above, e.g., cause-area specific conferences/retreats.
Agreed that their research is decent, but they are post-graduate institutes and have no undergraduate students.
Thanks, I saw a similar graph on Twitter! Wondering what kind of measurements would most clearly indicate more in-depth-engagement with EA—though traffic to the Forum likely comes close to that.
I liked it a lot. Given that he probably wasn't involved beforehand, the author got a detailed picture of EA's current state.
I appreciate that many EA's focus on high IQ and general mental ability can be hard to deal with. For instance, I found this quite aversive when I first got into EA.
But I'm unsure why your comment has 10 upvotes, given that you do not give many arguments for your statements.
Please let me know if anything below is uncharitable of if I misread something!
[...] why EA's obsession with elite universities is sickening.
The share of highly talented students at elite universities is higher. Thus, given the limited number of indiv...
Noting my excitement that you picked up on the idea and will actually make this happen!
The structure you lay out sounds good.
Regarding the winning team, will there be financial rewards? I’d give it >70% that someone would fund at least a ~$1000 award for the best team.
Maybe you’re already considering this but here it goes anyway:
I‘d advise against the name ‚longtermist hub‘. I wouldn‘t want longtermism to also become an identity, just as EA is one.
It also has reputational risks—which is why new EA-oriented orgs do not have EA in their name.
As far as I understand sessions will be fully subsidised by TfG. If you can’t afford them you can choose to pay 0$—unsure if this is standard among EA coaches.
I also think centralisation of psychological services might be valuable as it makes it easier to match fitting coaches/coachees and assess coaching performance.
Hey, I just wanted to leave a note of thanks for this excellent write-up!
I and some other EAs are planning an event with a similar format—your advice is super helpful to structure our planning and avoid obvious mistakes.
In general, these kinds of project management retrospectives provide a lot of value (e.g., EAF's hiring retrospective).
This could be easier, yes. I know of one person who models the defensive potential of different metagenomic sequencing approaches, but I think there is space for at least 3-5 additional people doing this.
I think he was explicitly addressing your question of sexually-transmitted diseases being capable of triggering pandemics, not if they can end civilization.
Discussing the latter in detail would quickly get into infohazards—but I think we should spend some of our efforts (10%) on defending against non-respiratory viruses. But I haven't thought about this in detail.
I do mean EAs with a longtermist focus. While writing about highly-engaged EAs, I had Benjamin Todd's EAG talk in mind, in which he pointed out that only around 4% of highly-engaged EAs are working in bio.
And thanks for pointing out I should be more precise. To qualify my statement, I'm 75% confident that this should happen.
Despite how promising and scalable we think some biosecurity interventions are, we don’t necessarily think that biosecurity should grow to be a substantially larger fraction of longtermist effort than it is currently.
Agreed that it shouldn't grow substantially, but ~doubling the share of highly-engaged EAs working on biosecurity feels reasonable to me.
I have only been involved in biosecurity for 1.5 years, but the focus on purely defensive projects (sterilization, refuges, some sequencing tech) feels relatively recent. It's a lot less risky to openly talk about those than about technologies like antivirals or vaccines.
I'm happy to see this shift, as concrete lists like this will likely motivate more people to enter the space.
@CarlaZoeC or Luke Kemp, could you create another forum post solely focused on your article? This might lead to more focused discussions, separating debate on community norms vs discussing arguments within your piece.
I also wanted to express that I'm sorry this experience has been so stressful. It's crucial to facilitate internal critique of EA, especially as the movement is becoming more powerful, and I feel pieces like yours are very useful to launch constructive discussions.
I particularly agree with the last point on focussing on purely defensive (not net-defensive) pathogen-agnostic technologies, such as metagenomic sequencing and resilience measures like PPE, air filters and shelters.
If others share this biodefense model in the longtermist biosecurity community, I think it'd be important to point towards these countermeasures in introductory materials (80k website, reading lists, future podcast episodes)
I do wonder what the downside is here. It's a fleeting, low-fidelity impression of EA that will probably not stick in most minds. However, if 10-20 people donate money after hearing about it through Patrick, it might already be positive in sum.
Do you specifically object to the term megaproject, or rather to the idea of launching larger organizations and projects that could potentially absorb a lot of money?
If it's the latter, the case for megaprojects is that they are bigger bets, with which funders could have an impact using larger sums of money, i.e., ~1-2 order of magnitudes bigger than current large longtermist grants. It is generally understood that EA has a funding overhang, which is even more true if you buy into longtermism, given that there are few obvious investment opportunities...
Hey Ludwig, happy to collaborate on this. A bunch of other EAs and I analyzed the initial party programs under EA considerations; this should be easily adapted to the final agreement and turned into a forum post.
Caveat: I work in Biosecurity.
I agree with the last point. Based on Ben Todd's presentation at EAG,
Based on Toby Ord's estimates in the Precipice, the risk of extinction in the next 100 years from
So, the stock of people in AI is 4.5x higher than Biosecurity, while AI is only 3x as important.
There is a lot of nuance missing here, but I'm moderately confident that this dysbalance...
Is there a historical precedent for social movements buying media? If so, it'd be interesting to know how that influenced the outlet's public perception/readership.
As of now, it seems like movements "merely" influence media, such as the NYTimes turning more leftward in the last few years or Vox employing more EA-oriented journalists.
Spencer Greenberg also comes to mind; he once noted that his agreeableness is in the 77th percentile. I'd consider him a generator.
Thanks for flagging, fixed!