This is a special post for quick takes by Rohin Shah. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
Reasonably often (maybe once or twice a month?) I see fairly highly upvoted posts that I think are basically wrong in something like "how they are reasoning", which I'll call epistemics. In particular, I think these are cases where it is pretty clear that the argument is wrong, and that this determination can be made using only knowledge that the author probably had (so it is more about reasoning correctly given a base of knowledge).
Sometimes I write a comment explaining why. If I reliably did this on all of the posts then you could still rely on karma as an indicator of epistemic soundness, but sadly I don't, because it's actually a fair amount of work and my time has high opportunity cost. So here is your PSA: for any particular high-karma post, knowing nothing else about the post besides that it is high karma, there is a non-trivial probability that I would find significant reasoning issues in that post. You can't rely solely on karma as a strong signal of epistemics.
Clear, strong examples from the EA Forum:
- Comment on How much current animal suffering does longtermism let us ignore?
- Comment on Aligning Recommender Systems as Cause Area
Moderate examples from the EA Forum (either they were lower karma, or only one particular thing was off instead of most of the post, or something else):
- Comment on Can money buy happiness? A review of new data
- Comment on How to think about an uncertain future: lessons from other sectors & mistakes of longtermist EAs
Weak examples from the EA Forum that are still some evidence (for these ones it's pretty likely someone would have made the points I made if I hadn't; that's not true for the others):
- Comment on Against the "smarts fetish"
- Comment on Critical Review of 'The Precipice': A Reassessment of the Risks of AI and Pandemics
- Comment on Does 80,000 Hours focus too much on AI risk?
Clear, strong examples from LessWrong:
Agreed. To some extent it's OK for bad posts to get upvoted. But I think the fact that posting volume is so much higher now means we should be able to trade off some of that volume for greater post quality. This could be by having a review process for posts, or reinstating the minimum upvote requirement before a user is allowed to post. I also think there may be some achievable gains that don't require trading off volume, such as improving the upvote strength algorithm.
I'd assume that forum members don't notice that the reasoning is bad.
As evidence in favor of this view, at least sometimes after I post such a comment, the post's karma starts to go down, suggesting that the comment informed voters about bad reasoning that they hadn't previously noticed. (Possibly this happened in most of the examples above, I wasn't carefully tracking this and don't know of any way to check now.)
I'd assume that forum members don't notice that the reasoning is bad.
Probably yeah, at least in part. Sometimes they may notice it a bit but put insufficient weight on it relative to the fact that they agree with the conclusion. But some may also miss it altogether.
My comment was in response to the claim that "to some extent it's OK for bad posts to get upvoted".
Hot take: strong upvoting things without great reasoning that also have conclusions I disagree with could be good for improving epistemics. At least, I think this gives us an opportunity to demonstrate common thinking processes in EA and what reasoning transparency looks like to newer people to the community. [1]
My best guess is that it also makes it more likely quality divergent thinking from established ideas happens in EA community spaces like the EA forum.
My reasoning is in a footnote in my comment here.
- ^
I'm aware that people on this thread might think my thinking processes and reasoning abilities aren't stellar,* but I still think my point stands.
*My personal view is that this impression would be less because I'm bad at thinking clearly and more because our views are quite different.
A large inferential distance means it's harder to diagnose epistemics accurately (but I'm not exactly an unbiased observer when it comes to judging my own ability to think clearly).
This leads me to another hot take: footnotes within footnotes are fun.
Maybe I want silent upvoting and downvoting to be disincentivized (or commenting with reasoning to be more incentivized). Commenting with reasoning is valuable but also hard work.
After 2 seconds of thought, I think I'd be massively in favour of a forum feature where any upvotes or downvotes count for more (e.g. double or triple the karma) once you've commented.[1]
Just having this incentive might make more people try and articulate what they think and why they think it. This extra incentive to stop and think might possibly make people change their votes even if they don't end up submitting their comments.
- ^
Me commenting on my own comment shouldn't mean the default upvote on my comment counts for more though: only the first reply should give extra voting power (I'm sure there are other ways to game it that I haven't thought of yet but I feel like there could be something salvageable from the idea anyway).
[This comment is no longer endorsed by its author]
I know this is just a small detail and not what you wrote about, but: much of your comment on the recommender systems post hinged on news articles being uncorrelated with the truth. Do you have data to back that up?
I'm replying here because it's a strong claim that's relevant to many things beyond that specific post.
I have data in the sense that when I read news articles and check how correct they are, they are usually not very correct. (You can have more nuance than this, e.g. facts about what mundane stuff happened in the world tend to be correct.)
I don't have data in the sense that I don't have a convenient list of articles and ways they were wrong such that I could easily persuade someone else of this belief of mine. (Though here's one example of an article that you at least have to read closely if you want to not be misled.)
Also, I could justify ignoring those two particular news articles without this general claim, at least to myself. I did briefly look at them before I wrote that comment; I didn't particularly expect to believe them but if they were the rare good kind of news article I would have noticed.
For radicalization, I know specific people who have looked into it and come away unconvinced; Stefan Schubert links to some of this work in a different comment on that post.
The article about social media being addictive is basically just a bunch of quotes from people rather than particular studies / data. It generally seems pretty easy to find people saying things you want so I don't update much on "such-and-such person said X". I've also once experienced and many times heard stories of journalists adversarially quoting people to make it sound like their position was very different than it actually was, so I usually don't even update on "such-and-such person believes X".
I'm wondering if it'd be good to have something special happen to posts where a comment has more karma than the OP. Like, decrease the font size of the OP and increase the font size of the comment, or display the comment first, or have a red warning light emoji next to the post's title or ...
Or maybe the commenter gets a $1,000 prize whenever that happens.
Good versions of "something special" would also incentivize the public service of pointing out significant flaws in posts by making comments that have a shot at exceeding the OP's karma score.
Obviously "there exists a comment that has higher karma than the OP" is an imperfect proxy of what we're after here, but anecdotally it seems to me this proxy works surprisingly well (though maybe it would stop due to Goodhart issues if we did any of the above) and it has the upside that it can be evaluated automatically.
There's been a few posts recently about how there should be more EA failures, since we're trying a bunch of high-risk, high-reward projects, and some of them should fail or we're not being ambitious enough.
I think this is a misunderstanding of what high-EV bets look like. Most projects do not either produce wild success or abject failure, there's usually a continuity of outcomes in between, and that's what you hit. This doesn't look like "failure", it looks like moderate success.
For example, consider the MineRL BASALT competition that I organized. The low-probability, high-value outcome would have had hundreds or thousands of entries to the competition, several papers produced as a result, and the establishment of BASALT as a standard benchmark and competition in the field.
What actually happened was that we got ~11 submissions, of which maybe ~5 were serious, made decent progress on the problem, produced a couple of vaguely interesting papers, some people in the field have heard about the benchmark and occasionally use it, and we built enough excitement in the team that the competition will (very likely) run again this year.
Is this failure? It certainly isn't what normally comes to mind from the normal meaning of "failure". But it was:
- Below my median expectation for what the competition would accomplish
- Not something I would have put time into if someone had told me in advance exactly what it would accomplish so far, and the time cost needed to get it.
One hopes that roughly 50% of the things I do meet the first criterion, and probably 90% of the things I'd do would meet the second. But also maybe 90% of the work I do is something people would say was "successful" even ex post.
If you are actually seeing failures for relatively large projects that look like "failures" in the normal English sense of the word, where basically nothing was accomplished at all, I'd be a lot more worried that actually your project was not in fact high-EV even ex ante, and you should be updating a lot more on your failure, and it is a good sign that we don't see that many EA "failures" in this sense.
(One exception to this is earning-to-give entrepreneurship, where "we had to shut the company down and made ~no money after a year of effort" seems reasonably likely and it still would plausibly be high-EV ex ante.)
I sometimes see people arguing for people to work in area A, and declaring a conflict of interest that they are personally working on area A.
If they already were working in area A for unrelated reasons, and then they produced these arguments, it seems reasonable to be worried about motivated reasoning.
On the other hand, if because of these arguments they switched to working in area A, this is in some sense a signal of sincerity ("I'm putting my career where my mouth is").
I don't like the norm of declaring your career as a "conflict of interest", because it implies that you are in the former rather than latter category, regardless of which one is actually true. (And the latter is especially common in EA.) However, I don't really have a candidate alternative norm.
I share your feeling towards it... but I also often say that one's "skin in the game" (your latter example) is someone else's "conflict of interest."
I don't think that the listener / reader is usually in a good position to distinguish between your first and your second example; that's enough to justify the practice of disclosing this as a potential "conflict of interest." In addition, by knowing you already work for cause X, I might consider if your case is affected by some kind of cognitive bias.
I'm not objecting to providing the information (I think that is good), I'm objecting to calling it a "conflict of interest".
I'd be much more keen on something like this (source):
For transparency, note that the reports for the latter three rows are all Open Philanthropy analyses, and I am co-CEO of Open Philanthropy.
Curated and popular this week
·
Advanced AI could unlock an era of enlightened and competent government action. But without smart, active investment, we’ll squander that opportunity and barrel blindly into danger.
Executive summary
See also a summary on Twitter / X.
The US federal government is falling behind the private sector on AI adoption. As AI improves, a growing gap would leave the government unable to effectively respond to AI-driven existential challenges and threaten the legitimacy of its democratic institutions.
A dual imperative
→ Government adoption of AI can’t wait. Making steady progress is critical to:
* Boost the government’s capacity to effectively respond to AI-driven existential challenges
* Help democratic oversight keep up with the technological power of other groups
* Defuse the risk of rushed AI adoption in a crisis
→ But hasty AI adoption could backfire. Without care, integration of AI could:
* Be exploited, subverting independent government action
* Lead to unsafe deployment of AI systems
* Accelerate arms races or compress safety research timelines
Summary of the recommendations
1. Work with the US federal government to help it effectively adopt AI
Simplistic “pro-security” or “pro-speed” attitudes miss the point. Both are important — and many interventions would help with both. We should:
* Invest in win-win measures that both facilitate adoption and reduce the risks involved, e.g.:
* Build technical expertise within government (invest in AI and technical talent, ensure NIST is well resourced)
* Streamline procurement processes for AI products and related tech (like cloud services)
* Modernize the government’s digital infrastructure and data management practices
* Prioritize high-leverage interventions that have strong adoption-boosting benefits with minor security costs or vice versa, e.g.:
* On the security side: investing in cyber security, pre-deployment testing of AI in high-stakes areas, and advancing research on mitigating the ris
·
In our recent strategy retreat, the GWWC Leadership Team recognised that by spreading our limited resources across too many projects, we are unable to deliver the level of excellence and impact that our mission demands.
True to our value of being mission accountable, we've therefore made the difficult but necessary decision to discontinue a total of 10 initiatives. By focusing our energy on fewer, more strategically aligned initiatives, we think we’ll be more likely to ultimately achieve our Big Hairy Audacious Goal of 1 million pledgers donating $3B USD to high-impact charities annually. (See our 2025 strategy.)
We’d like to be transparent about the choices we made, both to hold ourselves accountable and so other organisations can take the gaps we leave into account when planning their work. As such, this post aims to:
* Inform the broader EA community about changes to projects & highlight opportunities to carry these projects forward
* Provide timelines for project transitions
* Explain our rationale for discontinuing certain initiatives
What’s changing
We've identified 10 initiatives[1] to wind down or transition. These are:
* GWWC Canada
* Effective Altruism Australia funding partnership
* GWWC Groups
* Giving Games
* Charity Elections
* Effective Giving Meta evaluation and grantmaking
* The Donor Lottery
* Translations
* Hosted Funds
* New licensing of the GWWC brand
Each of these is detailed in the sections below, with timelines and transition plans where applicable.
How this is relevant to you
We still believe in the impact potential of many of these projects. Our decision doesn’t necessarily reflect their lack of value, but rather our need to focus at this juncture of GWWC's development.
Thus, we are actively looking for organisations and individuals interested in taking on some of these projects. If that’s you, please do reach out: see each project's section for specific contact details.
Thank you for your continued support as we
·
We are excited to share a summary of our 2025 strategy, which builds on our work in 2024 and provides a vision through 2027 and beyond!
Background
Giving What We Can (GWWC) is working towards a world without preventable suffering or existential risk, where everyone is able to flourish. We do this by making giving effectively and significantly a cultural norm.
Focus on pledges
Based on our last impact evaluation[1], we have made our pledges – and in particular the 🔸10% Pledge – the core focus of GWWC’s work.[2] We know the 🔸10% Pledge is a powerful institution, as we’ve seen almost 10,000 people take it and give nearly $50M USD to high-impact charities annually. We believe it could become a norm among at least the richest 1% — and likely a much wider segment of the population — which would cumulatively direct an enormous quantity of financial resources towards tackling the world’s most pressing problems.
We initiated this focus on pledges in early 2024, and are doubling down on it in 2025. In line with this, we are retiring various other initiatives we were previously running and which are not consistent with our new strategy.
Introducing our BHAG
We are setting ourselves a long-term Big Hairy Audacious Goal (BHAG) of 1 million pledgers donating $3B USD to high-impact charities annually, which we will start working towards in 2025.
1 million pledgers donating $3B USD to high-impact charities annually would be roughly equivalent to ~100x GWWC’s current scale, and could be achieved by 1% of the world’s richest 1% pledging and giving effectively. Achieving this would imply the equivalent of nearly 1 million lives being saved[3] every year.
See the BHAG FAQ for more info.
Working towards our BHAG
Over the coming years, we expect to test various growth pathways and interventions that could get us to our BHAG, including digital marketing, partnerships with aligned organisations, community advocacy, media/PR, and direct outreach to potential pledgers. We thin
Yes. But people are sticky, so you need to instill more vetting power in people who evaluate appropriately. The question is how to do that.